org.apache.flink.runtime.io.network.partition

Interface ResourceManagerPartitionTracker

All Known Implementing Classes:

ResourceManagerPartitionTrackerImpl

public interface ResourceManagerPartitionTracker

Utility for tracking and releasing partitions on the ResourceManager.

Method Summary

Modifier and Type	Method and Description
List<ShuffleDescriptor>	getClusterPartitionShuffleDescriptors(IntermediateDataSetID dataSetID) Returns all the shuffle descriptors of cluster partitions for the intermediate dataset.
Map<IntermediateDataSetID,DataSetMetaInfo>	listDataSets() Returns all data sets for which partitions are being tracked.
void	processTaskExecutorClusterPartitionReport(ResourceID taskExecutorId, ClusterPartitionReport clusterPartitionReport) Processes ClusterPartitionReport of a task executor.
void	processTaskExecutorShutdown(ResourceID taskExecutorId) Processes the shutdown of task executor.
CompletableFuture<Void>	releaseClusterPartitions(IntermediateDataSetID dataSetId) Issues a release calls to all task executors that are hosting partitions of the given data set.

Method Detail

processTaskExecutorClusterPartitionReport

void processTaskExecutorClusterPartitionReport(ResourceID taskExecutorId,
                                               ClusterPartitionReport clusterPartitionReport)

Processes ClusterPartitionReport of a task executor. Updates the tracking information for the respective task executor. Any partition no longer being hosted on the task executor is considered lost, corrupting the corresponding data set. For any such data set this method issues partition release calls to all task executors that are hosting partitions of this data set.

Parameters:

taskExecutorId - origin of the report

clusterPartitionReport - partition report

processTaskExecutorShutdown

void processTaskExecutorShutdown(ResourceID taskExecutorId)

Processes the shutdown of task executor. Removes all tracking information for the given executor, determines datasets that may be corrupted by the shutdown (and implied loss of partitions). For any such data set this method issues partition release calls to all task executors that are hosting partitions of this data set, and issues release calls.

Parameters:

taskExecutorId - task executor that shut down

releaseClusterPartitions

CompletableFuture<Void> releaseClusterPartitions(IntermediateDataSetID dataSetId)

Issues a release calls to all task executors that are hosting partitions of the given data set.

Parameters:

dataSetId - data set to release

listDataSets

Map<IntermediateDataSetID,DataSetMetaInfo> listDataSets()

Returns all data sets for which partitions are being tracked.

Returns:

tracked datasets

getClusterPartitionShuffleDescriptors

List<ShuffleDescriptor> getClusterPartitionShuffleDescriptors(IntermediateDataSetID dataSetID)

Returns all the shuffle descriptors of cluster partitions for the intermediate dataset.

Parameters:

dataSetID - The id of the intermediate dataset.

Returns:

the shuffle descriptors.