public interface PartitionFileReader
PartitionFileReader
defines the read logic for different types of shuffle files.Modifier and Type | Interface and Description |
---|---|
static class |
PartitionFileReader.ReadBufferResult
A wrapper class of the reading buffer result, including the read buffers, the hint of
continue reading, and the read progress, etc.
|
static interface |
PartitionFileReader.ReadProgress
This
PartitionFileReader.ReadProgress defines the read progress of the PartitionFileReader . |
Modifier and Type | Method and Description |
---|---|
long |
getPriority(TieredStoragePartitionId partitionId,
TieredStorageSubpartitionId subpartitionId,
int segmentId,
int bufferIndex,
PartitionFileReader.ReadProgress readProgress)
Get the priority for reading a particular buffer from the partitioned file.
|
PartitionFileReader.ReadBufferResult |
readBuffer(TieredStoragePartitionId partitionId,
TieredStorageSubpartitionId subpartitionId,
int segmentId,
int bufferIndex,
MemorySegment memorySegment,
BufferRecycler recycler,
PartitionFileReader.ReadProgress readProgress,
CompositeBuffer partialBuffer)
Read a buffer from the partition file.
|
void |
release()
Release the
PartitionFileReader . |
@Nullable PartitionFileReader.ReadBufferResult readBuffer(TieredStoragePartitionId partitionId, TieredStorageSubpartitionId subpartitionId, int segmentId, int bufferIndex, MemorySegment memorySegment, BufferRecycler recycler, @Nullable PartitionFileReader.ReadProgress readProgress, @Nullable CompositeBuffer partialBuffer) throws IOException
partitionId
- the partition id of the buffersubpartitionId
- the subpartition id of the buffersegmentId
- the segment id of the bufferbufferIndex
- the index of buffermemorySegment
- the empty buffer to store the read bufferrecycler
- the buffer recyclerreadProgress
- the current read progress. The progress comes from the previous
ReadBufferResult. Note that the read progress should be implemented and provided by
Flink, and it should be directly tied to the file format. The field can be null if the
current file reader has no the read progresspartialBuffer
- the previous partial buffer. The partial buffer is not null only when
the last read has a partial buffer, it will construct a full buffer during the read
process.IOException
- if an error happens.long getPriority(TieredStoragePartitionId partitionId, TieredStorageSubpartitionId subpartitionId, int segmentId, int bufferIndex, @Nullable PartitionFileReader.ReadProgress readProgress) throws IOException
Depending on the partition file implementation, following the suggestions should typically result in better performance and efficiency. This can be achieved by e.g. choosing preloaded data over others, optimizing the order of disk access to be more sequential, etc.
Note: Priorities are suggestions rather than a requirements. The caller can still read data in whichever order it wants.
partitionId
- the partition id of the buffersubpartitionId
- the subpartition id of the buffersegmentId
- the segment id of the bufferbufferIndex
- the index of bufferreadProgress
- the current read progress. The progress comes from the previous
ReadBufferResult. Note that the read progress should be implemented and provided by
Flink, and it should be directly tied to the file format. The field can be null if the
current file reader has no the read progressPartitionFileReader
.IOException
- if an error happens.void release()
PartitionFileReader
.Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.