org.apache.flink.runtime.shuffle

Class UnknownShuffleDescriptor

java.lang.Object

org.apache.flink.runtime.shuffle.UnknownShuffleDescriptor

All Implemented Interfaces:

Serializable, ShuffleDescriptor

public final class UnknownShuffleDescriptor
extends Object
implements ShuffleDescriptor

Unknown ShuffleDescriptor for which the producer has not been deployed yet.

When a partition consumer is being scheduled, it can happen that the producer of the partition (consumer input channel) has not been scheduled and its location and other relevant data is yet to be defined. To proceed with the consumer deployment, currently unknown input channels have to be marked with placeholders which are this special implementation of ShuffleDescriptor.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
UnknownShuffleDescriptor(ResultPartitionID resultPartitionID)

Method Summary

Modifier and Type	Method and Description
ResultPartitionID	getResultPartitionID()
boolean	isUnknown() Returns whether the partition is known and registered with the ShuffleMaster implementation.
Optional<ResourceID>	storesLocalResourcesOn() Returns the location of the producing task executor if the partition occupies local resources there.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

UnknownShuffleDescriptor

public UnknownShuffleDescriptor(ResultPartitionID resultPartitionID)

Method Detail

getResultPartitionID

public ResultPartitionID getResultPartitionID()

Specified by:

getResultPartitionID in interface ShuffleDescriptor

isUnknown

public boolean isUnknown()

Description copied from interface: ShuffleDescriptor

Returns whether the partition is known and registered with the ShuffleMaster implementation.

When a partition consumer is being scheduled, it can happen that the producer of the partition (consumer input channel) has not been scheduled and its location and other relevant data is yet to be defined. To proceed with the consumer deployment, currently unknown input channels have to be marked with placeholders. The placeholder is a special implementation of the shuffle descriptor: UnknownShuffleDescriptor.

Note: this method is not supposed to be overridden in concrete shuffle implementation. The only class where it returns true is UnknownShuffleDescriptor.

Specified by:

isUnknown in interface ShuffleDescriptor

Returns:

whether the partition producer has been ever deployed and the corresponding shuffle descriptor is obtained from the ShuffleMaster implementation.

storesLocalResourcesOn

public Optional<ResourceID> storesLocalResourcesOn()

Description copied from interface: ShuffleDescriptor

Returns the location of the producing task executor if the partition occupies local resources there.

Indicates that this partition occupies local resources in the producing task executor. Such partition requires that the task executor is running and being connected to be able to consume the produced data. This is mostly relevant for the batch jobs and blocking result partitions which can outlive the producer lifetime and be released externally. ShuffleEnvironment.releasePartitionsLocally(Collection) can be used to release such kind of partitions locally.

Specified by:

storesLocalResourcesOn in interface ShuffleDescriptor

Returns:

the resource id of the producing task executor if the partition occupies local resources there