public enum ResultPartitionType extends Enum<ResultPartitionType>
Enum Constant and Description |
---|
BLOCKING
Blocking partitions represent blocking data exchanges, where the data stream is first fully
produced and then consumed.
|
BLOCKING_PERSISTENT
BLOCKING_PERSISTENT partitions are similar to
BLOCKING partitions, but have a
user-specified life cycle. |
HYBRID_FULL
Hybrid partitions with a bounded (local) buffer pool to support downstream task to
simultaneous reading and writing shuffle data.
|
HYBRID_SELECTIVE
HYBRID_SELECTIVE partitions are similar to
HYBRID_FULL partitions, but it is not
re-consumable. |
PIPELINED
A pipelined streaming data exchange.
|
PIPELINED_APPROXIMATE
Pipelined partitions with a bounded (local) buffer pool to support downstream task to
continue consuming data after reconnection in Approximate Local-Recovery.
|
PIPELINED_BOUNDED
Pipelined partitions with a bounded (local) buffer pool.
|
Modifier and Type | Method and Description |
---|---|
boolean |
canBePipelinedConsumed()
return if this partition's upstream and downstream support scheduling in the same time.
|
boolean |
isBlockingOrBlockingPersistentResultPartition()
isBlockingOrBlockingPersistentResultPartition() is used to judge whether it is the
specified BLOCKING or BLOCKING_PERSISTENT resultPartitionType. |
boolean |
isBounded()
Whether this partition uses a limited number of (network) buffers or not.
|
boolean |
isHybridResultPartition()
isHybridResultPartition() is used to judge whether it is the specified HYBRID_FULL or HYBRID_SELECTIVE resultPartitionType. |
boolean |
isPersistent() |
boolean |
isPipelinedOrPipelinedBoundedResultPartition()
isPipelinedOrPipelinedBoundedResultPartition() is used to judge whether it is the
specified PIPELINED or PIPELINED_BOUNDED resultPartitionType. |
boolean |
isReconsumable() |
boolean |
isReleaseByScheduler() |
boolean |
isReleaseByUpstream() |
boolean |
mustBePipelinedConsumed()
return if this partition's upstream and downstream must be scheduled in the same time.
|
boolean |
supportCompression() |
static ResultPartitionType |
valueOf(String name)
Returns the enum constant of this type with the specified name.
|
static ResultPartitionType[] |
values()
Returns an array containing the constants of this enum type, in
the order they are declared.
|
public static final ResultPartitionType BLOCKING
Blocking partitions can be consumed multiple times and concurrently.
The partition is not automatically released after being consumed (like for example the
PIPELINED
partitions), but only released through the scheduler, when it determines
that the partition is no longer needed.
public static final ResultPartitionType BLOCKING_PERSISTENT
BLOCKING
partitions, but have a
user-specified life cycle.
BLOCKING_PERSISTENT partitions are dropped upon explicit API calls to the JobManager or ResourceManager, rather than by the scheduler.
Otherwise, the partition may only be dropped by safety-nets during failure handling scenarios, like when the TaskManager exits or when the TaskManager loses connection to JobManager / ResourceManager for too long.
public static final ResultPartitionType PIPELINED
Pipelined results can be consumed only once by a single consumer and are automatically disposed when the stream has been consumed.
This result partition type may keep an arbitrary amount of data in-flight, in contrast to
the PIPELINED_BOUNDED
variant.
public static final ResultPartitionType PIPELINED_BOUNDED
For streaming jobs, a fixed limit on the buffer pool size should help avoid that too much data is being buffered and checkpoint barriers are delayed. In contrast to limiting the overall network buffer pool size, this, however, still allows to be flexible with regards to the total number of partitions by selecting an appropriately big network buffer pool size.
For batch jobs, it will be best to keep this unlimited (PIPELINED
) since there
are no checkpoint barriers.
public static final ResultPartitionType PIPELINED_APPROXIMATE
Pipelined results can be consumed only once by a single consumer at one time. PIPELINED_APPROXIMATE
is different from PIPELINED
and PIPELINED_BOUNDED
in that PIPELINED_APPROXIMATE
partition can be reconnected after down stream task
fails.
public static final ResultPartitionType HYBRID_FULL
Hybrid partitions can be consumed any time, whether fully produced or not.
HYBRID_FULL partitions is re-consumable, so double calculation can be avoided during failover.
public static final ResultPartitionType HYBRID_SELECTIVE
HYBRID_FULL
partitions, but it is not
re-consumable.public static ResultPartitionType[] values()
for (ResultPartitionType c : ResultPartitionType.values()) System.out.println(c);
public static ResultPartitionType valueOf(String name)
name
- the name of the enum constant to be returned.IllegalArgumentException
- if this enum type has no constant with the specified nameNullPointerException
- if the argument is nullpublic boolean mustBePipelinedConsumed()
public boolean canBePipelinedConsumed()
public boolean isReleaseByScheduler()
public boolean isReleaseByUpstream()
public boolean isBlockingOrBlockingPersistentResultPartition()
isBlockingOrBlockingPersistentResultPartition()
is used to judge whether it is the
specified BLOCKING
or BLOCKING_PERSISTENT
resultPartitionType.
this method suitable for judgment conditions related to the specific implementation of
ResultPartitionType
.
this method not related to data consumption and partition release. As for the logic
related to partition release, use isReleaseByScheduler()
instead, and as consume
type, use mustBePipelinedConsumed()
or canBePipelinedConsumed()
instead.
public boolean isHybridResultPartition()
isHybridResultPartition()
is used to judge whether it is the specified HYBRID_FULL
or HYBRID_SELECTIVE
resultPartitionType.
this method suitable for judgment conditions related to the specific implementation of
ResultPartitionType
.
this method not related to data consumption and partition release. As for the logic
related to partition release, use isReleaseByScheduler()
instead, and as consume
type, use mustBePipelinedConsumed()
or canBePipelinedConsumed()
instead.
public boolean isPipelinedOrPipelinedBoundedResultPartition()
isPipelinedOrPipelinedBoundedResultPartition()
is used to judge whether it is the
specified PIPELINED
or PIPELINED_BOUNDED
resultPartitionType.
This method suitable for judgment conditions related to the specific implementation of
ResultPartitionType
.
This method not related to data consumption and partition release. As for the logic
related to partition release, use isReleaseByScheduler()
instead, and as consume
type, use mustBePipelinedConsumed()
or canBePipelinedConsumed()
instead.
public boolean isBounded()
public boolean isPersistent()
public boolean supportCompression()
public boolean isReconsumable()
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.