org.apache.flink.runtime.io.network.partition

Class PartitionedFile

java.lang.Object

org.apache.flink.runtime.io.network.partition.PartitionedFile

public class PartitionedFile
extends Object

PartitionedFile is the persistent file type of sort-merge based blocking shuffle. Each PartitionedFile contains two physical files: one is the data file and the other is the index file. Both the data file and the index file have multiple regions. Data belonging to the same subpartition are stored together in each data region and the corresponding index region contains index entries of all subpartitions. Each index entry is a (long, integer) value tuple of which the long value represents the file offset of the target subpartition and the integer value is the number of buffers.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DATA_FILE_SUFFIX
static int	INDEX_ENTRY_SIZE Size of each index entry in the index file: 8 bytes for file offset and 8 bytes for data size in bytes.
static String	INDEX_FILE_SUFFIX

Constructor Summary

Constructors

Constructor and Description
PartitionedFile(int numRegions, int numSubpartitions, Path dataFilePath, Path indexFilePath, long dataFileSize, long indexFileSize, long numBuffers, ByteBuffer indexEntryCache)

Method Summary

Modifier and Type	Method and Description
void	deleteQuietly()
Path	getDataFilePath()
Path	getIndexFilePath()
int	getNumRegions()
boolean	isReadable()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DATA_FILE_SUFFIX

public static final String DATA_FILE_SUFFIX

See Also:

Constant Field Values

INDEX_FILE_SUFFIX

public static final String INDEX_FILE_SUFFIX

See Also:

Constant Field Values

INDEX_ENTRY_SIZE

public static final int INDEX_ENTRY_SIZE

Size of each index entry in the index file: 8 bytes for file offset and 8 bytes for data size in bytes.

See Also:

Constant Field Values

Constructor Detail

PartitionedFile

public PartitionedFile(int numRegions,
                       int numSubpartitions,
                       Path dataFilePath,
                       Path indexFilePath,
                       long dataFileSize,
                       long indexFileSize,
                       long numBuffers,
                       @Nullable
                       ByteBuffer indexEntryCache)

Method Detail

getDataFilePath

public Path getDataFilePath()

getIndexFilePath

public Path getIndexFilePath()

getNumRegions

public int getNumRegions()

isReadable

public boolean isReadable()

deleteQuietly

public void deleteQuietly()

toString

public String toString()

Overrides:

toString in class Object