org.apache.flink.runtime.io.network.partition

Class BoundedBlockingResultPartition

java.lang.Object

org.apache.flink.runtime.io.network.partition.ResultPartition

org.apache.flink.runtime.io.network.partition.BufferWritingResultPartition

org.apache.flink.runtime.io.network.partition.BoundedBlockingResultPartition

All Implemented Interfaces:

AutoCloseable, AvailabilityProvider, ResultPartitionWriter

public class BoundedBlockingResultPartition
extends BufferWritingResultPartition

A output data result of an individual task (one partition of an intermediate result), produced and communicated in a batch manner: The result must be produced completely before it can be consumed.

In this particular implementation, the batch result is written to (and read from) one file per sub-partition. This implementation hence requires at least as many files (file handles) and memory buffers as the parallelism of the target task that the data is shuffled to.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AvailabilityProvider.AvailabilityHelper

Field Summary

Fields inherited from class org.apache.flink.runtime.io.network.partition.BufferWritingResultPartition

subpartitions

Fields inherited from class org.apache.flink.runtime.io.network.partition.ResultPartition

bufferCompressor, bufferPool, LOG, numBuffersOut, numBytesOut, numSubpartitions, partitionId, partitionManager, partitionType, resultPartitionBytes

Fields inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

AVAILABLE

Constructor Summary

Constructors

Constructor and Description
BoundedBlockingResultPartition(String owningTaskName, int partitionIndex, ResultPartitionID partitionId, ResultPartitionType partitionType, ResultSubpartition[] subpartitions, int numTargetKeyGroups, ResultPartitionManager partitionManager, BufferCompressor bufferCompressor, SupplierWithException<BufferPool,IOException> bufferPoolFactory)

Method Summary

Modifier and Type	Method and Description
void	flush(int targetSubpartition) Manually trigger the consumption of data from the given subpartitions.
void	flushAll() Manually trigger the consumption of data from all subpartitions.
void	notifyEndOfData(StopMode mode) Notifies the downstream tasks that this ResultPartitionWriter have emitted all the user records.

Methods inherited from class org.apache.flink.runtime.io.network.partition.BufferWritingResultPartition

abortCheckpoint, addToSubpartition, alignedBarrierTimeout, broadcastEvent, broadcastRecord, close, createSubpartitionView, emitRecord, finish, flushAllSubpartitions, flushSubpartition, getAllPartitions, getHardBackPressuredTimeMsPerSecond, getNumberOfQueuedBuffers, getNumberOfQueuedBuffers, getSizeOfQueuedBuffersUnsafe, releaseInternal, setMetricGroup, setupInternal

Methods inherited from class org.apache.flink.runtime.io.network.partition.ResultPartition

canBeCompressed, checkInProduceState, createSubpartitionView, fail, getAllDataProcessedFuture, getAvailableFuture, getBufferPool, getFailureCause, getNumberOfSubpartitions, getNumTargetKeyGroups, getOwningTaskName, getPartitionId, getPartitionIndex, getPartitionManager, getPartitionType, isFinished, isNumberOfPartitionConsumerUndefined, isNumberOfPartitionConsumerUndefined, isReleased, onSubpartitionAllDataProcessed, release, release, setMaxOverdraftBuffersPerGate, setup, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.io.AvailabilityProvider

and, isApproximatelyAvailable, isAvailable, or

Constructor Detail

BoundedBlockingResultPartition

public BoundedBlockingResultPartition(String owningTaskName,
                                      int partitionIndex,
                                      ResultPartitionID partitionId,
                                      ResultPartitionType partitionType,
                                      ResultSubpartition[] subpartitions,
                                      int numTargetKeyGroups,
                                      ResultPartitionManager partitionManager,
                                      @Nullable
                                      BufferCompressor bufferCompressor,
                                      SupplierWithException<BufferPool,IOException> bufferPoolFactory)

Method Detail

notifyEndOfData

public void notifyEndOfData(StopMode mode)
                     throws IOException

Description copied from interface: ResultPartitionWriter

Notifies the downstream tasks that this ResultPartitionWriter have emitted all the user records.

Specified by:

notifyEndOfData in interface ResultPartitionWriter

Overrides:

notifyEndOfData in class ResultPartition

Parameters:

mode - tells if we should flush all records or not (it is false in case of stop-with-savepoint (--no-drain))

Throws:

IOException

flush

public void flush(int targetSubpartition)

Description copied from interface: ResultPartitionWriter

Manually trigger the consumption of data from the given subpartitions.

flushAll

public void flushAll()

Description copied from interface: ResultPartitionWriter

Manually trigger the consumption of data from all subpartitions.