org.apache.flink.runtime.io.network.partition.consumer

Interface InputGate

All Known Implementing Classes:

SingleInputGate, UnionInputGate

public interface InputGate

An input gate consumes one or more partitions of a single produced intermediate result.

Each intermediate result is partitioned over its producing parallel subtasks; each of these partitions is furthermore partitioned into one or more subpartitions.

As an example, consider a map-reduce program, where the map operator produces data and the reduce operator consumes the produced data.

 +-----+              +---------------------+              +--------+
 | Map | = produce => | Intermediate Result | <= consume = | Reduce |
 +-----+              +---------------------+              +--------+
 

When deploying such a program in parallel, the intermediate result will be partitioned over its producing parallel subtasks; each of these partitions is furthermore partitioned into one or more subpartitions.

                            Intermediate result
               +-----------------------------------------+
               |                      +----------------+ |              +-----------------------+
 +-------+     | +-------------+  +=> | Subpartition 1 | | <=======+=== | Input Gate | Reduce 1 |
 | Map 1 | ==> | | Partition 1 | =|   +----------------+ |         |    +-----------------------+
 +-------+     | +-------------+  +=> | Subpartition 2 | | <==+    |
               |                      +----------------+ |    |    | Subpartition request
               |                                         |    |    |
               |                      +----------------+ |    |    |
 +-------+     | +-------------+  +=> | Subpartition 1 | | <==+====+
 | Map 2 | ==> | | Partition 2 | =|   +----------------+ |    |         +-----------------------+
 +-------+     | +-------------+  +=> | Subpartition 2 | | <==+======== | Input Gate | Reduce 2 |
               |                      +----------------+ |              +-----------------------+
               +-----------------------------------------+
 

In the above example, two map subtasks produce the intermediate result in parallel, resulting in two partitions (Partition 1 and 2). Each of these partitions is further partitioned into two subpartitions -- one for each parallel reduce subtask. As shown in the Figure, each reduce task will have an input gate attached to it. This will provide its input, which will consist of one subpartition from each partition of the intermediate result.

Method Summary

Modifier and Type	Method and Description
Optional<BufferOrEvent>	getNextBufferOrEvent() Blocking call waiting for next BufferOrEvent.
int	getNumberOfInputChannels()
String	getOwningTaskName()
int	getPageSize()
boolean	isFinished()
Optional<BufferOrEvent>	pollNextBufferOrEvent() Poll the BufferOrEvent.
void	registerListener(InputGateListener listener)
void	requestPartitions()
void	sendTaskEvent(TaskEvent event)

Method Detail

getNumberOfInputChannels

int getNumberOfInputChannels()

getOwningTaskName

String getOwningTaskName()

isFinished

boolean isFinished()

requestPartitions

void requestPartitions()
                throws IOException,
                       InterruptedException

Throws:

IOException

InterruptedException

getNextBufferOrEvent

Optional<BufferOrEvent> getNextBufferOrEvent()
                                      throws IOException,
                                             InterruptedException

Blocking call waiting for next BufferOrEvent.

Returns:

Optional.empty() if isFinished() returns true.

Throws:

IOException

InterruptedException

pollNextBufferOrEvent

Optional<BufferOrEvent> pollNextBufferOrEvent()
                                       throws IOException,
                                              InterruptedException

Poll the BufferOrEvent.

Returns:

Optional.empty() if there is no data to return or if isFinished() returns true.

Throws:

IOException

InterruptedException

sendTaskEvent

void sendTaskEvent(TaskEvent event)
            throws IOException

Throws:

IOException

registerListener

void registerListener(InputGateListener listener)

getPageSize

int getPageSize()