Package org.apache.flink.runtime.io.network.partition

Class ResultPartition

    • Field Detail

      • LOG

        protected static final org.slf4j.Logger LOG

      • partitionType

        protected final ResultPartitionType partitionType
        Type of this partition. Defines the concrete subpartition implementation to use.

      • numSubpartitions

        protected final int numSubpartitions

      • bufferCompressor

        @Nullable
protected final BufferCompressor bufferCompressor
        Used to compress buffer to reduce IO.

      • numBytesOut

        protected Counter numBytesOut

      • numBuffersOut

        protected Counter numBuffersOut

    • Method Detail

      • setup

        public void setup()
           throws IOException
        Registers a buffer pool with this result partition.

        There is one pool for each result partition, which is shared by all its sub partitions.

        The pool is registered with the partition *after* it as been constructed in order to conform to the life-cycle of task registrations in the TaskExecutor.

        Specified by:
        setup in interface ResultPartitionWriter
        Throws:
        IOException

      • setupInternal

        protected abstract void setupInternal()
                               throws IOException
        Do the subclass's own setup operation.
        Throws:
        IOException

      • getOwningTaskName

        public String getOwningTaskName()

      • getPartitionIndex

        public int getPartitionIndex()

      • getBufferPool

        public BufferPool getBufferPool()

      • isNumberOfPartitionConsumerUndefined

        public void isNumberOfPartitionConsumerUndefined​(boolean isNumberOfPartitionConsumerUndefined)

      • isNumberOfPartitionConsumerUndefined

        public boolean isNumberOfPartitionConsumerUndefined()

      • getNumberOfQueuedBuffers

        public abstract int getNumberOfQueuedBuffers()
        Returns the total number of queued buffers of all subpartitions.

      • getSizeOfQueuedBuffersUnsafe

        public abstract long getSizeOfQueuedBuffersUnsafe()
        Returns the total size in bytes of queued buffers of all subpartitions.

      • getNumberOfQueuedBuffers

        public abstract int getNumberOfQueuedBuffers​(int targetSubpartition)
        Returns the number of queued buffers of the given target subpartition.

      • getPartitionType

        public ResultPartitionType getPartitionType()
        Returns the type of this result partition.
        Returns:
        result partition type

      • notifyEndOfData

        public void notifyEndOfData​(StopMode mode)
                     throws IOException
        Description copied from interface: ResultPartitionWriter
        Notifies the downstream tasks that this ResultPartitionWriter have emitted all the user records.
        Specified by:
        notifyEndOfData in interface ResultPartitionWriter
        Parameters:
        mode - tells if we should flush all records or not (it is false in case of stop-with-savepoint (--no-drain))
        Throws:
        IOException

      • onSubpartitionAllDataProcessed

        public void onSubpartitionAllDataProcessed​(int subpartition)
        The subpartition notifies that the corresponding downstream task have processed all the user records.
        Parameters:
        subpartition - The index of the subpartition sending the notification.
        See Also:
        EndOfData

      • finish

        public void finish()
            throws IOException
        Finishes the result partition.

        After this operation, it is not possible to add further data to the result partition.

        For BLOCKING results, this will trigger the deployment of consuming tasks.

        Specified by:
        finish in interface ResultPartitionWriter
        Throws:
        IOException

      • release

        public void release()

      • releaseInternal

        protected abstract void releaseInternal()
        Releases all produced data including both those stored in memory and persisted on disk.

      • fail

        public void fail​(@Nullable
                 Throwable throwable)
        Description copied from interface: ResultPartitionWriter
        Fail the production of the partition.

        This method propagates non-null failure causes to consumers on a best-effort basis. This call also leads to the release of all resources associated with the partition. Closing of the partition is still needed afterwards if it has not been done before.

        Specified by:
        fail in interface ResultPartitionWriter
        Parameters:
        throwable - failure cause

      • getFailureCause

        public Throwable getFailureCause()

      • isReleased

        public boolean isReleased()
        Whether this partition is released.

        A partition is released when each subpartition is either consumed and communication is closed by consumer or failed. A partition is also released if task is cancelled.

        Specified by:
        isReleased in interface ResultPartitionWriter

      • canBeCompressed

        protected boolean canBeCompressed​(Buffer buffer)
        Whether the buffer can be compressed or not. Note that event is not compressed because it is usually small and the size can become even larger after compression.