public class ResourceManagerPartitionTrackerImpl extends Object implements ResourceManagerPartitionTracker
ResourceManagerPartitionTracker
implementation.
Internal tracking info must only be updated upon reception of a ClusterPartitionReport
, as the task executor state is the source of truth.
Constructor and Description |
---|
ResourceManagerPartitionTrackerImpl(TaskExecutorClusterPartitionReleaser taskExecutorClusterPartitionReleaser) |
Modifier and Type | Method and Description |
---|---|
List<ShuffleDescriptor> |
getClusterPartitionShuffleDescriptors(IntermediateDataSetID dataSetID)
Returns all the shuffle descriptors of cluster partitions for the intermediate dataset.
|
Map<IntermediateDataSetID,DataSetMetaInfo> |
listDataSets()
Returns all data sets for which partitions are being tracked.
|
void |
processTaskExecutorClusterPartitionReport(ResourceID taskExecutorId,
ClusterPartitionReport clusterPartitionReport)
Processes
ClusterPartitionReport of a task executor. |
void |
processTaskExecutorShutdown(ResourceID taskExecutorId)
Processes the shutdown of task executor.
|
CompletableFuture<Void> |
releaseClusterPartitions(IntermediateDataSetID dataSetId)
Issues a release calls to all task executors that are hosting partitions of the given data
set.
|
public ResourceManagerPartitionTrackerImpl(TaskExecutorClusterPartitionReleaser taskExecutorClusterPartitionReleaser)
public void processTaskExecutorClusterPartitionReport(ResourceID taskExecutorId, ClusterPartitionReport clusterPartitionReport)
ResourceManagerPartitionTracker
ClusterPartitionReport
of a task executor. Updates the tracking information
for the respective task executor. Any partition no longer being hosted on the task executor
is considered lost, corrupting the corresponding data set. For any such data set this method
issues partition release calls to all task executors that are hosting partitions of this data
set.processTaskExecutorClusterPartitionReport
in interface ResourceManagerPartitionTracker
taskExecutorId
- origin of the reportclusterPartitionReport
- partition reportpublic void processTaskExecutorShutdown(ResourceID taskExecutorId)
ResourceManagerPartitionTracker
processTaskExecutorShutdown
in interface ResourceManagerPartitionTracker
taskExecutorId
- task executor that shut downpublic CompletableFuture<Void> releaseClusterPartitions(IntermediateDataSetID dataSetId)
ResourceManagerPartitionTracker
releaseClusterPartitions
in interface ResourceManagerPartitionTracker
dataSetId
- data set to releasepublic List<ShuffleDescriptor> getClusterPartitionShuffleDescriptors(IntermediateDataSetID dataSetID)
ResourceManagerPartitionTracker
getClusterPartitionShuffleDescriptors
in interface ResourceManagerPartitionTracker
dataSetID
- The id of the intermediate dataset.public Map<IntermediateDataSetID,DataSetMetaInfo> listDataSets()
ResourceManagerPartitionTracker
listDataSets
in interface ResourceManagerPartitionTracker
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.