Package org.apache.flink.api.common

Enum BatchShuffleMode

  • All Implemented Interfaces:
    Serializable, Comparable<BatchShuffleMode>, DescribedEnum
    @PublicEvolving
public enum BatchShuffleMode
extends Enum<BatchShuffleMode>
implements DescribedEnum
    Defines how data is exchanged between tasks in batch ExecutionOptions.RUNTIME_MODE if the shuffling behavior has not been set explicitly for an individual exchange.

    With pipelined exchanges, upstream and downstream tasks run simultaneously. In order to achieve lower latency, a result record is immediately sent to and processed by the downstream task. Thus, the receiver back-pressures the sender. The streaming mode always uses this exchange.

    With blocking exchanges, upstream and downstream tasks run in stages. Records are persisted to some storage between stages. Downstream tasks then fetch these records after the upstream tasks finished. Such an exchange reduces the resources required to execute the job as it does not need to run upstream and downstream tasks simultaneously.

    • Enum Constant Detail

      • ALL_EXCHANGES_PIPELINED

        public static final BatchShuffleMode ALL_EXCHANGES_PIPELINED
        Upstream and downstream tasks run simultaneously.

        This leads to lower latency and more evenly distributed (but higher) resource usage across tasks.

      • ALL_EXCHANGES_BLOCKING

        public static final BatchShuffleMode ALL_EXCHANGES_BLOCKING
        Upstream and downstream tasks run subsequently.

        This reduces the resource usage as downstream tasks are started after upstream tasks finished.

      • ALL_EXCHANGES_HYBRID_FULL

        public static final BatchShuffleMode ALL_EXCHANGES_HYBRID_FULL
        Downstream can start running anytime, as long as the upstream has started.

        This adapts the resource usage to whatever is available.

        This type will spill all data to disk to support re-consume.

      • ALL_EXCHANGES_HYBRID_SELECTIVE

        public static final BatchShuffleMode ALL_EXCHANGES_HYBRID_SELECTIVE
        Downstream can start running anytime, as long as the upstream has started.

        This adapts the resource usage to whatever is available.

        This type will selective spilling data to reduce disk writes as much as possible.

    • Method Detail

      • values

        public static BatchShuffleMode[] values()
        Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows: 
        for (BatchShuffleMode c : BatchShuffleMode.values())
    System.out.println(c);
        Returns:
        an array containing the constants of this enum type, in the order they are declared

      • valueOf

        public static BatchShuffleMode valueOf​(String name)
        Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)
        Parameters:
        name - the name of the enum constant to be returned.
        Returns:
        the enum constant with the specified name
        Throws:
        IllegalArgumentException - if this enum type has no constant with the specified name
        NullPointerException - if the argument is null