public class ResultPartition extends Object implements BufferPoolOwner
This class is the runtime part of a logical IntermediateResultPartition
. Essentially,
a result partition is a collection of Buffer
instances. The buffers are organized in one
or more ResultSubpartition
instances, which further partition the data depending on the
number of consuming tasks and the data DistributionPattern
.
Tasks, which consume a result partition have to request one of its subpartitions. The request
happens either remotely (see RemoteInputChannel
) or locally (see LocalInputChannel
)
Constructor and Description |
---|
ResultPartition(String owningTaskName,
JobID jobId,
ResultPartitionID partitionId,
ResultPartitionType partitionType,
boolean eagerlyDeployConsumers,
int numberOfSubpartitions,
ResultPartitionManager partitionManager,
ResultPartitionConsumableNotifier partitionConsumableNotifier,
IOManager ioManager,
IOManager.IOMode defaultIoMode) |
Modifier and Type | Method and Description |
---|---|
void |
add(Buffer buffer,
int subpartitionIndex)
Adds a buffer to the subpartition with the given index.
|
ResultSubpartitionView |
createSubpartitionView(int index,
BufferProvider bufferProvider)
Returns the requested subpartition.
|
void |
destroyBufferPool() |
void |
finish()
Finishes the result partition.
|
BufferProvider |
getBufferProvider() |
boolean |
getEagerlyDeployConsumers()
Returns whether consumers should be deployed eagerly (as soon as they
are registered at the result manager of the task manager).
|
Throwable |
getFailureCause() |
JobID |
getJobId() |
int |
getNumberOfSubpartitions() |
ResultPartitionID |
getPartitionId() |
int |
getTotalNumberOfBuffers() |
long |
getTotalNumberOfBytes() |
void |
registerBufferPool(BufferPool bufferPool)
Registers a buffer pool with this result partition.
|
void |
release() |
void |
release(Throwable cause)
Releases the result partition.
|
void |
releaseMemory(int toRelease)
Releases buffers held by this result partition.
|
String |
toString() |
public ResultPartition(String owningTaskName, JobID jobId, ResultPartitionID partitionId, ResultPartitionType partitionType, boolean eagerlyDeployConsumers, int numberOfSubpartitions, ResultPartitionManager partitionManager, ResultPartitionConsumableNotifier partitionConsumableNotifier, IOManager ioManager, IOManager.IOMode defaultIoMode)
public void registerBufferPool(BufferPool bufferPool)
There is one pool for each result partition, which is shared by all its sub partitions.
The pool is registered with the partition *after* it as been constructed in order to conform
to the life-cycle of task registrations in the TaskManager
.
public JobID getJobId()
public ResultPartitionID getPartitionId()
public int getNumberOfSubpartitions()
public boolean getEagerlyDeployConsumers()
public BufferProvider getBufferProvider()
public int getTotalNumberOfBuffers()
public long getTotalNumberOfBytes()
public void add(Buffer buffer, int subpartitionIndex) throws IOException
For PIPELINED results, this will trigger the deployment of consuming tasks after the first buffer has been added.
IOException
public void finish() throws IOException
After this operation, it is not possible to add further data to the result partition.
For BLOCKING results, this will trigger the deployment of consuming tasks.
IOException
public void release()
public void release(Throwable cause)
public void destroyBufferPool()
public ResultSubpartitionView createSubpartitionView(int index, BufferProvider bufferProvider) throws IOException
IOException
public Throwable getFailureCause()
public void releaseMemory(int toRelease) throws IOException
This is a callback from the buffer pool, which is registered for result partitions, which are back pressure-free.
releaseMemory
in interface BufferPoolOwner
IOException
Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.