org.apache.flink.runtime.io.network.partition.hybrid

Interface HsFileDataIndex

All Known Implementing Classes:

HsFileDataIndexImpl

public interface HsFileDataIndex

Index of spilled data. For each spilled data buffer, this maintains the subpartition it belongs to, the buffer index within the subpartition, the offset in file it begin with, and its readability.

Note: Buffers added to this index should not be read until being explicitly marked READABLE.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	HsFileDataIndex.ReadableRegion Represents a series of physically continuous buffers in the file, which are readable, from the same subpartition, and has sequential buffer index.
static class	HsFileDataIndex.SpilledBuffer Represents a spilled buffer.

Method Summary

Modifier and Type	Method and Description
void	addBuffers(List<HsFileDataIndex.SpilledBuffer> spilledBuffers) Add buffers to the index.
Optional<HsFileDataIndex.ReadableRegion>	getReadableRegion(int subpartitionId, int bufferIndex, int consumingOffset) Get a HsFileDataIndex.ReadableRegion for the given subpartition that starts with the given buffer index, if existed.
void	markBufferReleased(int subpartitionId, int bufferIndex) Mark a buffer as RELEASED.

Method Detail

getReadableRegion

Optional<HsFileDataIndex.ReadableRegion> getReadableRegion(int subpartitionId,
                                                           int bufferIndex,
                                                           int consumingOffset)

Get a HsFileDataIndex.ReadableRegion for the given subpartition that starts with the given buffer index, if existed.

Note: Depending on the implementation, this method does not guarantee to always return the longest possible readable region.

Parameters:

subpartitionId - that the readable region belongs to

bufferIndex - that the readable region starts with

consumingOffset - of the downstream

Returns:

a HsFileDataIndex.ReadableRegion for the given subpartition that starts with the given buffer index, if exist; otherwise, Optional.empty().

addBuffers

void addBuffers(List<HsFileDataIndex.SpilledBuffer> spilledBuffers)

Add buffers to the index.

Attention: this method only called by spilling thread.

Parameters:

spilledBuffers - to be added. The buffers in the list are expected in the same order as in the spilled file.

markBufferReleased

void markBufferReleased(int subpartitionId,
                        int bufferIndex)

Mark a buffer as RELEASED.

Parameters:

subpartitionId - that the buffer belongs to

bufferIndex - of the buffer within the subpartition