Interface ShuffleDescriptor
-
- All Superinterfaces:
Serializable
- All Known Implementing Classes:
NettyShuffleDescriptor
,UnknownShuffleDescriptor
public interface ShuffleDescriptor extends Serializable
Interface for shuffle deployment descriptor of result partition resource.The descriptor is used for the deployment of the partition producer/consumer and their data exchange
-
-
Method Summary
All Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description ResultPartitionID
getResultPartitionID()
default boolean
isUnknown()
Returns whether the partition is known and registered with theShuffleMaster
implementation.Optional<ResourceID>
storesLocalResourcesOn()
Returns the location of the producing task executor if the partition occupies local resources there.
-
-
-
Method Detail
-
getResultPartitionID
ResultPartitionID getResultPartitionID()
-
isUnknown
default boolean isUnknown()
Returns whether the partition is known and registered with theShuffleMaster
implementation.When a partition consumer is being scheduled, it can happen that the producer of the partition (consumer input channel) has not been scheduled and its location and other relevant data is yet to be defined. To proceed with the consumer deployment, currently unknown input channels have to be marked with placeholders. The placeholder is a special implementation of the shuffle descriptor:
UnknownShuffleDescriptor
.Note: this method is not supposed to be overridden in concrete shuffle implementation. The only class where it returns
true
isUnknownShuffleDescriptor
.- Returns:
- whether the partition producer has been ever deployed and the corresponding shuffle
descriptor is obtained from the
ShuffleMaster
implementation.
-
storesLocalResourcesOn
Optional<ResourceID> storesLocalResourcesOn()
Returns the location of the producing task executor if the partition occupies local resources there.Indicates that this partition occupies local resources in the producing task executor. Such partition requires that the task executor is running and being connected to be able to consume the produced data. This is mostly relevant for the batch jobs and blocking result partitions which can outlive the producer lifetime and be released externally.
ShuffleEnvironment.releasePartitionsLocally(Collection)
can be used to release such kind of partitions locally.- Returns:
- the resource id of the producing task executor if the partition occupies local resources there
-
-