org.apache.flink.runtime.io.network.partition.consumer

Class InputChannel

java.lang.Object

org.apache.flink.runtime.io.network.partition.consumer.InputChannel

Direct Known Subclasses:

LocalInputChannel, RecoveredInputChannel, RemoteInputChannel

public abstract class InputChannel
extends Object

An input channel consumes a single ResultSubpartitionView.

For each channel, the consumption life cycle is as follows:

1. requestSubpartitions()
2. getNextBuffer()
3. releaseAllResources()

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	InputChannel.BufferAndAvailability A combination of a Buffer and a flag indicating availability of further buffers, and the backlog length indicating how many non-event buffers are available in the subpartitions.

Field Summary

Fields

Modifier and Type	Field and Description
protected InputChannelInfo	channelInfo The info of the input channel to identify it globally within a task.
protected ResultSubpartitionIndexSet	consumedSubpartitionIndexSet The indexes of the subpartitions consumed by this channel.
protected int	currentBackoff The current backoff (in ms).
protected int	initialBackoff The initial backoff (in ms).
protected SingleInputGate	inputGate
protected int	maxBackoff The maximum backoff (in ms).
protected Counter	numBuffersIn
protected Counter	numBytesIn
protected ResultPartitionID	partitionId The parent partition of the subpartitions consumed by this channel.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	InputChannel(SingleInputGate inputGate, int channelIndex, ResultPartitionID partitionId, ResultSubpartitionIndexSet consumedSubpartitionIndexSet, int initialBackoff, int maxBackoff, Counter numBytesIn, Counter numBuffersIn)

Method Summary

Modifier and Type	Method and Description
abstract void	acknowledgeAllRecordsProcessed() When received EndOfData from one channel, it need to acknowledge after this event get processed.
protected void	checkError() Checks for an error and rethrows it if one was reported.
void	checkpointStarted(CheckpointBarrier barrier) Called by task thread when checkpointing is started (e.g., any input channel received barrier).
void	checkpointStopped(long checkpointId) Called by task thread on cancel/complete to clean-up temporary data.
void	convertToPriorityEvent(int sequenceNumber)
int	getChannelIndex() Returns the index of this channel within its SingleInputGate.
InputChannelInfo	getChannelInfo() Returns the info of this channel, which uniquely identifies the channel in respect to its operator instance.
ResultSubpartitionIndexSet	getConsumedSubpartitionIndexSet()
protected int	getCurrentBackoff() Returns the current backoff in ms.
abstract Optional<InputChannel.BufferAndAvailability>	getNextBuffer() Returns the next buffer from the consumed subpartitions or Optional.empty() if there is no data to return.
ResultPartitionID	getPartitionId()
protected boolean	increaseBackoff() Increases the current backoff and returns whether the operation was successful.
protected void	notifyBufferAvailable(int numAvailableBuffers)
protected void	notifyChannelNonEmpty() Notifies the owning SingleInputGate that this channel became non-empty.
void	notifyPriorityEvent(int priorityBufferNumber)
void	notifyRequiredSegmentId(int subpartitionId, int segmentId) Notify the upstream the id of required segment that should be sent to netty connection.
int	peekNextBufferSubpartitionId() Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available and the subpartition to be consumed is not determined.
protected abstract int	peekNextBufferSubpartitionIdInternal() Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available and the subpartition to be consumed is not determined.
abstract void	resumeConsumption() After sending a CheckpointBarrier of exactly-once mode, the upstream will be blocked and become unavailable.
protected void	setError(Throwable cause) Atomically sets an error for this channel and notifies the input gate about available data to trigger querying this channel by the task thread.
int	unsynchronizedGetNumberOfQueuedBuffers()
long	unsynchronizedGetSizeOfQueuedBuffers()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

channelInfo

protected final InputChannelInfo channelInfo

The info of the input channel to identify it globally within a task.

partitionId

protected final ResultPartitionID partitionId

The parent partition of the subpartitions consumed by this channel.

consumedSubpartitionIndexSet

protected final ResultSubpartitionIndexSet consumedSubpartitionIndexSet

The indexes of the subpartitions consumed by this channel.

inputGate

protected final SingleInputGate inputGate

initialBackoff

protected final int initialBackoff

The initial backoff (in ms).

maxBackoff

protected final int maxBackoff

The maximum backoff (in ms).

numBytesIn

protected final Counter numBytesIn

numBuffersIn

protected final Counter numBuffersIn

currentBackoff

protected int currentBackoff

The current backoff (in ms).

Constructor Detail

InputChannel

protected InputChannel(SingleInputGate inputGate,
                       int channelIndex,
                       ResultPartitionID partitionId,
                       ResultSubpartitionIndexSet consumedSubpartitionIndexSet,
                       int initialBackoff,
                       int maxBackoff,
                       Counter numBytesIn,
                       Counter numBuffersIn)

Method Detail

getChannelIndex

public int getChannelIndex()

Returns the index of this channel within its SingleInputGate.

getChannelInfo

public InputChannelInfo getChannelInfo()

Returns the info of this channel, which uniquely identifies the channel in respect to its operator instance.

getPartitionId

public ResultPartitionID getPartitionId()

getConsumedSubpartitionIndexSet

public ResultSubpartitionIndexSet getConsumedSubpartitionIndexSet()

resumeConsumption

public abstract void resumeConsumption()
                                throws IOException

After sending a CheckpointBarrier of exactly-once mode, the upstream will be blocked and become unavailable. This method tries to unblock the corresponding upstream and resume data consumption.

Throws:

IOException

acknowledgeAllRecordsProcessed

public abstract void acknowledgeAllRecordsProcessed()
                                             throws IOException

When received EndOfData from one channel, it need to acknowledge after this event get processed.

Throws:

IOException

notifyChannelNonEmpty

protected void notifyChannelNonEmpty()

Notifies the owning SingleInputGate that this channel became non-empty.

This is guaranteed to be called only when a Buffer was added to a previously empty input channel. The notion of empty is atomically consistent with the flag InputChannel.BufferAndAvailability.moreAvailable() when polling the next buffer from this channel.

Note: When the input channel observes an exception, this method is called regardless of whether the channel was empty before. That ensures that the parent InputGate will always be notified about the exception.

notifyPriorityEvent

public void notifyPriorityEvent(int priorityBufferNumber)

notifyBufferAvailable

protected void notifyBufferAvailable(int numAvailableBuffers)
                              throws IOException

Throws:

IOException

peekNextBufferSubpartitionId

public int peekNextBufferSubpartitionId()
                                 throws IOException

Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available and the subpartition to be consumed is not determined.

Throws:

IOException

peekNextBufferSubpartitionIdInternal

protected abstract int peekNextBufferSubpartitionIdInternal()
                                                     throws IOException

Returns the index of the subpartition where the next buffer locates, or -1 if there is no buffer available and the subpartition to be consumed is not determined.

Throws:

IOException

getNextBuffer

public abstract Optional<InputChannel.BufferAndAvailability> getNextBuffer()
                                                                    throws IOException,
                                                                           InterruptedException

Returns the next buffer from the consumed subpartitions or Optional.empty() if there is no data to return.

Throws:

IOException

InterruptedException

checkpointStarted

public void checkpointStarted(CheckpointBarrier barrier)
                       throws CheckpointException

Called by task thread when checkpointing is started (e.g., any input channel received barrier).

Throws:

CheckpointException

checkpointStopped

public void checkpointStopped(long checkpointId)

Called by task thread on cancel/complete to clean-up temporary data.

convertToPriorityEvent

public void convertToPriorityEvent(int sequenceNumber)
                            throws IOException

Throws:

IOException

checkError

protected void checkError()
                   throws IOException

Checks for an error and rethrows it if one was reported.

Note: Any PartitionException instances should not be transformed and make sure they are always visible in task failure cause.

Throws:

IOException

setError

protected void setError(Throwable cause)

Atomically sets an error for this channel and notifies the input gate about available data to trigger querying this channel by the task thread.

getCurrentBackoff

protected int getCurrentBackoff()

Returns the current backoff in ms.

increaseBackoff

protected boolean increaseBackoff()

Increases the current backoff and returns whether the operation was successful.

Returns:

true, iff the operation was successful. Otherwise, false.

unsynchronizedGetNumberOfQueuedBuffers

public int unsynchronizedGetNumberOfQueuedBuffers()

unsynchronizedGetSizeOfQueuedBuffers

public long unsynchronizedGetSizeOfQueuedBuffers()

notifyRequiredSegmentId

public void notifyRequiredSegmentId(int subpartitionId,
                                    int segmentId)
                             throws IOException

Notify the upstream the id of required segment that should be sent to netty connection.

Parameters:

subpartitionId - The id of the corresponding subpartition.

segmentId - The id of required segment.

Throws:

IOException