public class ProducerMergedPartitionFileIndex extends Object
ProducerMergedPartitionFileIndex
is used by ProducerMergedPartitionFileWriter
and ProducerMergedPartitionFileReader
, to maintain the offset of each buffer in the
physical file.
For efficiency, buffers from the same subpartition that are both logically (i.e. index in the
subpartition) and physically (i.e. offset in the file) consecutive are combined into a ProducerMergedPartitionFileIndex.FixedSizeRegion
.
For example, the following buffers (indicated by subpartitionId-bufferIndex): 1-1, 1-2, 1-3, 2-1, 2-2, 2-5, 1-4, 1-5, 2-6 will be combined into 5 regions (separated by '|'): 1-1, 1-2, 1-3 | 2-1, 2-2 | 2-5 | 1-4, 1-5 | 2-6
Modifier and Type | Class and Description |
---|---|
static class |
ProducerMergedPartitionFileIndex.FixedSizeRegion
Represents a series of buffers that are:
From the same subpartition
Logically (i.e.
|
Constructor and Description |
---|
ProducerMergedPartitionFileIndex(int numSubpartitions,
Path indexFilePath,
int regionGroupSizeInBytes,
long numRetainedInMemoryRegionsMax) |
public ProducerMergedPartitionFileIndex(int numSubpartitions, Path indexFilePath, int regionGroupSizeInBytes, long numRetainedInMemoryRegionsMax)
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.