public class BoundedBlockingResultPartition extends BufferWritingResultPartition
In this particular implementation, the batch result is written to (and read from) one file per sub-partition. This implementation hence requires at least as many files (file handles) and memory buffers as the parallelism of the target task that the data is shuffled to.
AvailabilityProvider.AvailabilityHelper
subpartitions
bufferCompressor, bufferPool, LOG, numBuffersOut, numBytesOut, numSubpartitions, partitionId, partitionManager, partitionType, resultPartitionBytes
AVAILABLE
Constructor and Description |
---|
BoundedBlockingResultPartition(String owningTaskName,
int partitionIndex,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
ResultSubpartition[] subpartitions,
int numTargetKeyGroups,
ResultPartitionManager partitionManager,
BufferCompressor bufferCompressor,
SupplierWithException<BufferPool,IOException> bufferPoolFactory) |
Modifier and Type | Method and Description |
---|---|
void |
flush(int targetSubpartition)
Manually trigger the consumption of data from the given subpartitions.
|
void |
flushAll()
Manually trigger the consumption of data from all subpartitions.
|
void |
notifyEndOfData(StopMode mode)
Notifies the downstream tasks that this
ResultPartitionWriter have emitted all the
user records. |
abortCheckpoint, addToSubpartition, alignedBarrierTimeout, broadcastEvent, broadcastRecord, close, createSubpartitionView, emitRecord, finish, flushAllSubpartitions, flushSubpartition, getAllPartitions, getHardBackPressuredTimeMsPerSecond, getNumberOfQueuedBuffers, getNumberOfQueuedBuffers, getSizeOfQueuedBuffersUnsafe, releaseInternal, setMetricGroup, setupInternal
canBeCompressed, checkInProduceState, createSubpartitionView, fail, getAllDataProcessedFuture, getAvailableFuture, getBufferPool, getFailureCause, getNumberOfSubpartitions, getNumTargetKeyGroups, getOwningTaskName, getPartitionId, getPartitionIndex, getPartitionManager, getPartitionType, getResultPartitionBytes, isFinished, isNumberOfPartitionConsumerUndefined, isNumberOfPartitionConsumerUndefined, isReleased, onSubpartitionAllDataProcessed, release, release, setMaxOverdraftBuffersPerGate, setup, toString
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
and, isApproximatelyAvailable, isAvailable, or
public BoundedBlockingResultPartition(String owningTaskName, int partitionIndex, ResultPartitionID partitionId, ResultPartitionType partitionType, ResultSubpartition[] subpartitions, int numTargetKeyGroups, ResultPartitionManager partitionManager, @Nullable BufferCompressor bufferCompressor, SupplierWithException<BufferPool,IOException> bufferPoolFactory)
public void notifyEndOfData(StopMode mode) throws IOException
ResultPartitionWriter
ResultPartitionWriter
have emitted all the
user records.notifyEndOfData
in interface ResultPartitionWriter
notifyEndOfData
in class ResultPartition
mode
- tells if we should flush all records or not (it is false in case of
stop-with-savepoint (--no-drain))IOException
public void flush(int targetSubpartition)
ResultPartitionWriter
public void flushAll()
ResultPartitionWriter
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.