org.apache.flink.api.common

Enum BatchShuffleMode

java.lang.Object

java.lang.Enum<BatchShuffleMode>

org.apache.flink.api.common.BatchShuffleMode

All Implemented Interfaces:

Serializable, Comparable<BatchShuffleMode>, DescribedEnum

@PublicEvolving
public enum BatchShuffleMode
extends Enum<BatchShuffleMode>
implements DescribedEnum

Defines how data is exchanged between tasks in batch ExecutionOptions.RUNTIME_MODE if the shuffling behavior has not been set explicitly for an individual exchange.

With pipelined exchanges, upstream and downstream tasks run simultaneously. In order to achieve lower latency, a result record is immediately sent to and processed by the downstream task. Thus, the receiver back-pressures the sender. The streaming mode always uses this exchange.

With blocking exchanges, upstream and downstream tasks run in stages. Records are persisted to some storage between stages. Downstream tasks then fetch these records after the upstream tasks finished. Such an exchange reduces the resources required to execute the job as it does not need to run upstream and downstream tasks simultaneously.

Enum Constant Summary

Enum Constants

Enum Constant and Description
ALL_EXCHANGES_BLOCKING Upstream and downstream tasks run subsequently.
ALL_EXCHANGES_HYBRID_FULL Downstream can start running anytime, as long as the upstream has started.
ALL_EXCHANGES_HYBRID_SELECTIVE Downstream can start running anytime, as long as the upstream has started.
ALL_EXCHANGES_PIPELINED Upstream and downstream tasks run simultaneously.

Method Summary

Modifier and Type	Method and Description
InlineElement	getDescription() Returns the description for the enum constant.
static BatchShuffleMode	valueOf(String name) Returns the enum constant of this type with the specified name.
static BatchShuffleMode[]	values() Returns an array containing the constants of this enum type, in the order they are declared.

Methods inherited from class java.lang.Enum

clone, compareTo, equals, finalize, getDeclaringClass, hashCode, name, ordinal, toString, valueOf

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Enum Constant Detail

ALL_EXCHANGES_PIPELINED

public static final BatchShuffleMode ALL_EXCHANGES_PIPELINED

Upstream and downstream tasks run simultaneously.

This leads to lower latency and more evenly distributed (but higher) resource usage across tasks.

ALL_EXCHANGES_BLOCKING

public static final BatchShuffleMode ALL_EXCHANGES_BLOCKING

Upstream and downstream tasks run subsequently.

This reduces the resource usage as downstream tasks are started after upstream tasks finished.

ALL_EXCHANGES_HYBRID_FULL

public static final BatchShuffleMode ALL_EXCHANGES_HYBRID_FULL

Downstream can start running anytime, as long as the upstream has started.

This adapts the resource usage to whatever is available.

This type will spill all data to disk to support re-consume.

ALL_EXCHANGES_HYBRID_SELECTIVE

public static final BatchShuffleMode ALL_EXCHANGES_HYBRID_SELECTIVE

Downstream can start running anytime, as long as the upstream has started.

This adapts the resource usage to whatever is available.

This type will selective spilling data to reduce disk writes as much as possible.

Method Detail

values

public static BatchShuffleMode[] values()

Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows:

for (BatchShuffleMode c : BatchShuffleMode.values())
    System.out.println(c);

Returns:

an array containing the constants of this enum type, in the order they are declared

valueOf

public static BatchShuffleMode valueOf(String name)

Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)

Parameters:

name - the name of the enum constant to be returned.

Returns:

the enum constant with the specified name

Throws:

IllegalArgumentException - if this enum type has no constant with the specified name

NullPointerException - if the argument is null

getDescription

public InlineElement getDescription()

Description copied from interface: DescribedEnum

Returns the description for the enum constant.

If you want to include links or code blocks, use TextElement.wrap(InlineElement...) to wrap multiple inline elements into a single one.

Specified by:

getDescription in interface DescribedEnum