Class HashBasedDataBuffer
- java.lang.Object
-
- org.apache.flink.runtime.io.network.partition.HashBasedDataBuffer
-
- All Implemented Interfaces:
DataBuffer
public class HashBasedDataBuffer extends Object implements DataBuffer
* ADataBuffer
implementation which sorts all appended records only by subpartition index. Records of the same subpartition keep the appended order.Different from the
SortBasedDataBuffer
, in thisDataBuffer
implementation, memory segment boundary serves as the nature data boundary of different subpartitions, which means that one memory segment can never contain data from different subpartitions.
-
-
Constructor Summary
Constructors Constructor Description HashBasedDataBuffer(LinkedList<MemorySegment> freeSegments, BufferRecycler bufferRecycler, int numSubpartitions, int bufferSize, int numGuaranteedBuffers, int[] customReadOrder)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description boolean
append(ByteBuffer source, int targetSubpartition, Buffer.DataType dataType)
Partial data of the target record can be written if thisHashBasedDataBuffer
is full.void
finish()
Finishes thisDataBuffer
which means no record can be appended anymore.BufferWithSubpartition
getNextBuffer(MemorySegment transitBuffer)
Copies data in thisDataBuffer
to the targetMemorySegment
in subpartition index order and returnsBufferWithSubpartition
which contains the copied data and the corresponding subpartition index.boolean
hasRemaining()
Returns true if not all data appended to thisDataBuffer
is consumed.boolean
isFinished()
Whether thisDataBuffer
is finished or not.boolean
isReleased()
Whether thisDataBuffer
is released or not.long
numTotalBytes()
Returns the total number of bytes written to thisDataBuffer
.long
numTotalRecords()
Returns the total number of records written to thisDataBuffer
.void
release()
Releases thisDataBuffer
which releases all resources.
-
-
-
Constructor Detail
-
HashBasedDataBuffer
public HashBasedDataBuffer(LinkedList<MemorySegment> freeSegments, BufferRecycler bufferRecycler, int numSubpartitions, int bufferSize, int numGuaranteedBuffers, @Nullable int[] customReadOrder)
-
-
Method Detail
-
append
public boolean append(ByteBuffer source, int targetSubpartition, Buffer.DataType dataType) throws IOException
Partial data of the target record can be written if thisHashBasedDataBuffer
is full. The remaining data of the target record will be written to the next data region (a new data buffer or this data buffer after reset).- Specified by:
append
in interfaceDataBuffer
- Throws:
IOException
-
getNextBuffer
public BufferWithSubpartition getNextBuffer(MemorySegment transitBuffer)
Description copied from interface:DataBuffer
Copies data in thisDataBuffer
to the targetMemorySegment
in subpartition index order and returnsBufferWithSubpartition
which contains the copied data and the corresponding subpartition index.- Specified by:
getNextBuffer
in interfaceDataBuffer
-
numTotalRecords
public long numTotalRecords()
Description copied from interface:DataBuffer
Returns the total number of records written to thisDataBuffer
.- Specified by:
numTotalRecords
in interfaceDataBuffer
-
numTotalBytes
public long numTotalBytes()
Description copied from interface:DataBuffer
Returns the total number of bytes written to thisDataBuffer
.- Specified by:
numTotalBytes
in interfaceDataBuffer
-
hasRemaining
public boolean hasRemaining()
Description copied from interface:DataBuffer
Returns true if not all data appended to thisDataBuffer
is consumed.- Specified by:
hasRemaining
in interfaceDataBuffer
-
finish
public void finish()
Description copied from interface:DataBuffer
Finishes thisDataBuffer
which means no record can be appended anymore.- Specified by:
finish
in interfaceDataBuffer
-
isFinished
public boolean isFinished()
Description copied from interface:DataBuffer
Whether thisDataBuffer
is finished or not.- Specified by:
isFinished
in interfaceDataBuffer
-
release
public void release()
Description copied from interface:DataBuffer
Releases thisDataBuffer
which releases all resources.- Specified by:
release
in interfaceDataBuffer
-
isReleased
public boolean isReleased()
Description copied from interface:DataBuffer
Whether thisDataBuffer
is released or not.- Specified by:
isReleased
in interfaceDataBuffer
-
-