Class InputChannel
- java.lang.Object
-
- org.apache.flink.runtime.io.network.partition.consumer.InputChannel
-
- Direct Known Subclasses:
LocalInputChannel
,RecoveredInputChannel
,RemoteInputChannel
public abstract class InputChannel extends Object
An input channel consumes a singleResultSubpartitionView
.For each channel, the consumption life cycle is as follows:
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
InputChannel.BufferAndAvailability
A combination of aBuffer
and a flag indicating availability of further buffers, and the backlog length indicating how many non-event buffers are available in the subpartitions.
-
Field Summary
Fields Modifier and Type Field Description protected InputChannelInfo
channelInfo
The info of the input channel to identify it globally within a task.protected ResultSubpartitionIndexSet
consumedSubpartitionIndexSet
The indexes of the subpartitions consumed by this channel.protected int
currentBackoff
The current backoff (in ms).protected int
initialBackoff
The initial backoff (in ms).protected SingleInputGate
inputGate
protected int
maxBackoff
The maximum backoff (in ms).protected Counter
numBuffersIn
protected Counter
numBytesIn
protected ResultPartitionID
partitionId
The parent partition of the subpartitions consumed by this channel.
-
Constructor Summary
Constructors Modifier Constructor Description protected
InputChannel(SingleInputGate inputGate, int channelIndex, ResultPartitionID partitionId, ResultSubpartitionIndexSet consumedSubpartitionIndexSet, int initialBackoff, int maxBackoff, Counter numBytesIn, Counter numBuffersIn)
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description abstract void
acknowledgeAllRecordsProcessed()
When receivedEndOfData
from one channel, it need to acknowledge after this event get processed.protected void
checkError()
Checks for an error and rethrows it if one was reported.void
checkpointStarted(CheckpointBarrier barrier)
Called by task thread when checkpointing is started (e.g., any input channel received barrier).void
checkpointStopped(long checkpointId)
Called by task thread on cancel/complete to clean-up temporary data.void
convertToPriorityEvent(int sequenceNumber)
int
getChannelIndex()
Returns the index of this channel within itsSingleInputGate
.InputChannelInfo
getChannelInfo()
Returns the info of this channel, which uniquely identifies the channel in respect to its operator instance.ResultSubpartitionIndexSet
getConsumedSubpartitionIndexSet()
protected int
getCurrentBackoff()
Returns the current backoff in ms.abstract Optional<InputChannel.BufferAndAvailability>
getNextBuffer()
Returns the next buffer from the consumed subpartitions orOptional.empty()
if there is no data to return.ResultPartitionID
getPartitionId()
protected boolean
increaseBackoff()
Increases the current backoff and returns whether the operation was successful.protected void
notifyBufferAvailable(int numAvailableBuffers)
protected void
notifyChannelNonEmpty()
Notifies the owningSingleInputGate
that this channel became non-empty.void
notifyPriorityEvent(int priorityBufferNumber)
void
notifyRequiredSegmentId(int subpartitionId, int segmentId)
Notify the upstream the id of required segment that should be sent to netty connection.int
peekNextBufferSubpartitionId()
Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available and the subpartition to be consumed is not determined.protected abstract int
peekNextBufferSubpartitionIdInternal()
Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available and the subpartition to be consumed is not determined.abstract void
resumeConsumption()
After sending aCheckpointBarrier
of exactly-once mode, the upstream will be blocked and become unavailable.protected void
setError(Throwable cause)
Atomically sets an error for this channel and notifies the input gate about available data to trigger querying this channel by the task thread.int
unsynchronizedGetNumberOfQueuedBuffers()
long
unsynchronizedGetSizeOfQueuedBuffers()
-
-
-
Field Detail
-
channelInfo
protected final InputChannelInfo channelInfo
The info of the input channel to identify it globally within a task.
-
partitionId
protected final ResultPartitionID partitionId
The parent partition of the subpartitions consumed by this channel.
-
consumedSubpartitionIndexSet
protected final ResultSubpartitionIndexSet consumedSubpartitionIndexSet
The indexes of the subpartitions consumed by this channel.
-
inputGate
protected final SingleInputGate inputGate
-
initialBackoff
protected final int initialBackoff
The initial backoff (in ms).
-
maxBackoff
protected final int maxBackoff
The maximum backoff (in ms).
-
numBytesIn
protected final Counter numBytesIn
-
numBuffersIn
protected final Counter numBuffersIn
-
currentBackoff
protected int currentBackoff
The current backoff (in ms).
-
-
Constructor Detail
-
InputChannel
protected InputChannel(SingleInputGate inputGate, int channelIndex, ResultPartitionID partitionId, ResultSubpartitionIndexSet consumedSubpartitionIndexSet, int initialBackoff, int maxBackoff, Counter numBytesIn, Counter numBuffersIn)
-
-
Method Detail
-
getChannelIndex
public int getChannelIndex()
Returns the index of this channel within itsSingleInputGate
.
-
getChannelInfo
public InputChannelInfo getChannelInfo()
Returns the info of this channel, which uniquely identifies the channel in respect to its operator instance.
-
getPartitionId
public ResultPartitionID getPartitionId()
-
getConsumedSubpartitionIndexSet
public ResultSubpartitionIndexSet getConsumedSubpartitionIndexSet()
-
resumeConsumption
public abstract void resumeConsumption() throws IOException
After sending aCheckpointBarrier
of exactly-once mode, the upstream will be blocked and become unavailable. This method tries to unblock the corresponding upstream and resume data consumption.- Throws:
IOException
-
acknowledgeAllRecordsProcessed
public abstract void acknowledgeAllRecordsProcessed() throws IOException
When receivedEndOfData
from one channel, it need to acknowledge after this event get processed.- Throws:
IOException
-
notifyChannelNonEmpty
protected void notifyChannelNonEmpty()
Notifies the owningSingleInputGate
that this channel became non-empty.This is guaranteed to be called only when a Buffer was added to a previously empty input channel. The notion of empty is atomically consistent with the flag
InputChannel.BufferAndAvailability.moreAvailable()
when polling the next buffer from this channel.Note: When the input channel observes an exception, this method is called regardless of whether the channel was empty before. That ensures that the parent InputGate will always be notified about the exception.
-
notifyPriorityEvent
public void notifyPriorityEvent(int priorityBufferNumber)
-
notifyBufferAvailable
protected void notifyBufferAvailable(int numAvailableBuffers) throws IOException
- Throws:
IOException
-
peekNextBufferSubpartitionId
public int peekNextBufferSubpartitionId() throws IOException
Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available and the subpartition to be consumed is not determined.- Throws:
IOException
-
peekNextBufferSubpartitionIdInternal
protected abstract int peekNextBufferSubpartitionIdInternal() throws IOException
Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available and the subpartition to be consumed is not determined.- Throws:
IOException
-
getNextBuffer
public abstract Optional<InputChannel.BufferAndAvailability> getNextBuffer() throws IOException, InterruptedException
Returns the next buffer from the consumed subpartitions orOptional.empty()
if there is no data to return.- Throws:
IOException
InterruptedException
-
checkpointStarted
public void checkpointStarted(CheckpointBarrier barrier) throws CheckpointException
Called by task thread when checkpointing is started (e.g., any input channel received barrier).- Throws:
CheckpointException
-
checkpointStopped
public void checkpointStopped(long checkpointId)
Called by task thread on cancel/complete to clean-up temporary data.
-
convertToPriorityEvent
public void convertToPriorityEvent(int sequenceNumber) throws IOException
- Throws:
IOException
-
checkError
protected void checkError() throws IOException
Checks for an error and rethrows it if one was reported.Note: Any
PartitionException
instances should not be transformed and make sure they are always visible in task failure cause.- Throws:
IOException
-
setError
protected void setError(Throwable cause)
Atomically sets an error for this channel and notifies the input gate about available data to trigger querying this channel by the task thread.
-
getCurrentBackoff
protected int getCurrentBackoff()
Returns the current backoff in ms.
-
increaseBackoff
protected boolean increaseBackoff()
Increases the current backoff and returns whether the operation was successful.- Returns:
true
, iff the operation was successful. Otherwise,false
.
-
unsynchronizedGetNumberOfQueuedBuffers
public int unsynchronizedGetNumberOfQueuedBuffers()
-
unsynchronizedGetSizeOfQueuedBuffers
public long unsynchronizedGetSizeOfQueuedBuffers()
-
notifyRequiredSegmentId
public void notifyRequiredSegmentId(int subpartitionId, int segmentId) throws IOException
Notify the upstream the id of required segment that should be sent to netty connection.- Parameters:
subpartitionId
- The id of the corresponding subpartition.segmentId
- The id of required segment.- Throws:
IOException
-
-