public enum NettyShuffleMaster extends Enum<NettyShuffleMaster> implements ShuffleMaster<NettyShuffleDescriptor>
ShuffleMaster
for netty and local file based shuffle implementation.Enum Constant and Description |
---|
INSTANCE |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<NettyShuffleDescriptor> |
registerPartitionWithProducer(PartitionDescriptor partitionDescriptor,
ProducerDescriptor producerDescriptor)
Asynchronously register a partition and its producer with the shuffle service.
|
void |
releasePartitionExternally(ShuffleDescriptor shuffleDescriptor)
Release any external resources occupied by the given partition.
|
static NettyShuffleMaster |
valueOf(String name)
Returns the enum constant of this type with the specified name.
|
static NettyShuffleMaster[] |
values()
Returns an array containing the constants of this enum type, in
the order they are declared.
|
public static final NettyShuffleMaster INSTANCE
public static NettyShuffleMaster[] values()
for (NettyShuffleMaster c : NettyShuffleMaster.values()) System.out.println(c);
public static NettyShuffleMaster valueOf(String name)
name
- the name of the enum constant to be returned.IllegalArgumentException
- if this enum type has no constant with the specified nameNullPointerException
- if the argument is nullpublic CompletableFuture<NettyShuffleDescriptor> registerPartitionWithProducer(PartitionDescriptor partitionDescriptor, ProducerDescriptor producerDescriptor)
ShuffleMaster
The returned shuffle descriptor is an internal handle which identifies the partition internally within the shuffle service. The descriptor should provide enough information to read from or write data to the partition.
registerPartitionWithProducer
in interface ShuffleMaster<NettyShuffleDescriptor>
partitionDescriptor
- general job graph information about the partitionproducerDescriptor
- general producer information (location, execution id, connection info)public void releasePartitionExternally(ShuffleDescriptor shuffleDescriptor)
ShuffleMaster
This call triggers release of any resources which are occupied by the given partition in the external systems
outside of the producer executor. This is mostly relevant for the batch jobs and blocking result partitions.
The producer local resources are managed by ShuffleDescriptor.storesLocalResourcesOn()
and
ShuffleEnvironment.releasePartitionsLocally(Collection)
.
releasePartitionExternally
in interface ShuffleMaster<NettyShuffleDescriptor>
shuffleDescriptor
- shuffle descriptor of the result partition to release externally.Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.