org.apache.flink.runtime.io.network.partition.consumer

Class UnionInputGate

java.lang.Object

org.apache.flink.runtime.io.network.partition.consumer.InputGate

org.apache.flink.runtime.io.network.partition.consumer.UnionInputGate

All Implemented Interfaces:

AutoCloseable, AvailabilityProvider, PullingAsyncDataInput<BufferOrEvent>

public class UnionInputGate
extends InputGate

Input gate wrapper to union the input from multiple input gates.

Each input gate has input channels attached from which it reads data. At each input gate, the input channels have unique IDs from 0 (inclusive) to the number of input channels (exclusive).

 +---+---+      +---+---+---+
 | 0 | 1 |      | 0 | 1 | 2 |
 +--------------+--------------+
 | Input gate 0 | Input gate 1 |
 +--------------+--------------+
 

The union input gate maps these IDs from 0 to the *total* number of input channels across all unioned input gates, e.g. the channels of input gate 0 keep their original indexes and the channel indexes of input gate 1 are set off by 2 to 2--4.

 +---+---++---+---+---+
 | 0 | 1 || 2 | 3 | 4 |
 +--------------------+
 | Union input gate   |
 +--------------------+
 

It is NOT possible to recursively union union input gates.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.runtime.io.network.partition.consumer.InputGate

InputGate.InputWithData<INPUT,DATA>

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AvailabilityProvider.AvailabilityHelper

Field Summary

Fields inherited from class org.apache.flink.runtime.io.network.partition.consumer.InputGate

availabilityHelper

Fields inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AVAILABLE

Constructor Summary

Constructors

Constructor and Description
UnionInputGate(IndexedInputGate... inputGates)

Method Summary

Modifier and Type	Method and Description
void	close()
InputChannel	getChannel(int channelIndex) Returns the channel of this gate.
Optional<BufferOrEvent>	getNext() Blocking call waiting for next BufferOrEvent.
int	getNumberOfInputChannels() Returns the total number of input channels across all unioned input gates.
boolean	isFinished()
Optional<BufferOrEvent>	pollNext() Poll the BufferOrEvent.
CompletableFuture<?>	readRecoveredState(ExecutorService executor, ChannelStateReader reader) Reads the previous unaligned checkpoint states before requesting partition data.
void	registerBufferReceivedListener(BufferReceivedListener listener)
void	requestPartitions()
void	resumeConsumption(int channelIndex)
void	sendTaskEvent(TaskEvent event)
void	setup() Setup gate, potentially heavy-weight, blocking operation comparing to just creation.

Methods inherited from class org.apache.flink.runtime.io.network.partition.consumer.InputGate

getAvailableFuture, getChannelInfos

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

isApproximatelyAvailable, isAvailable

Constructor Detail

UnionInputGate

public UnionInputGate(IndexedInputGate... inputGates)

Method Detail

getNumberOfInputChannels

public int getNumberOfInputChannels()

Returns the total number of input channels across all unioned input gates.

Specified by:

getNumberOfInputChannels in class InputGate

getChannel

public InputChannel getChannel(int channelIndex)

Description copied from class: InputGate

Returns the channel of this gate.

Specified by:

getChannel in class InputGate

isFinished

public boolean isFinished()

Specified by:

isFinished in interface PullingAsyncDataInput<BufferOrEvent>

Specified by:

isFinished in class InputGate

Returns:

true if is finished and for example end of input was reached, false otherwise.

getNext

public Optional<BufferOrEvent> getNext()
                                throws IOException,
                                       InterruptedException

Description copied from class: InputGate

Blocking call waiting for next BufferOrEvent.

Note: It should be guaranteed that the previous returned buffer has been recycled before getting next one.

Specified by:

getNext in class InputGate

Returns:

Optional.empty() if InputGate.isFinished() returns true.

Throws:

IOException

InterruptedException

pollNext

public Optional<BufferOrEvent> pollNext()
                                 throws IOException,
                                        InterruptedException

Description copied from class: InputGate

Poll the BufferOrEvent.

Note: It should be guaranteed that the previous returned buffer has been recycled before polling next one.

Specified by:

pollNext in interface PullingAsyncDataInput<BufferOrEvent>

Specified by:

pollNext in class InputGate

Returns:

Optional.empty() if there is no data to return or if InputGate.isFinished() returns true.

Throws:

IOException

InterruptedException

sendTaskEvent

public void sendTaskEvent(TaskEvent event)
                   throws IOException

Specified by:

sendTaskEvent in class InputGate

Throws:

IOException

resumeConsumption

public void resumeConsumption(int channelIndex)
                       throws IOException

Specified by:

resumeConsumption in class InputGate

Throws:

IOException

setup

public void setup()

Description copied from class: InputGate

Setup gate, potentially heavy-weight, blocking operation comparing to just creation.

Specified by:

setup in class InputGate

readRecoveredState

public CompletableFuture<?> readRecoveredState(ExecutorService executor,
                                               ChannelStateReader reader)

Description copied from class: InputGate

Reads the previous unaligned checkpoint states before requesting partition data.

Specified by:

readRecoveredState in class InputGate

Parameters:

executor - the dedicated executor for performing this action for all the internal channels.

reader - the dedicated reader for unspilling the respective channel state from snapshots.

Returns:

the future indicates whether the recovered states have already been drained or not.

requestPartitions

public void requestPartitions()
                       throws IOException

Specified by:

requestPartitions in class InputGate

Throws:

IOException

close

public void close()
           throws IOException

Throws:

IOException

registerBufferReceivedListener

public void registerBufferReceivedListener(BufferReceivedListener listener)

Specified by:

registerBufferReceivedListener in class InputGate