org.apache.flink.runtime.executiongraph

Class ExecutionJobVertex

java.lang.Object

org.apache.flink.runtime.executiongraph.ExecutionJobVertex

All Implemented Interfaces:

Archiveable<ArchivedExecutionJobVertex>, AccessExecutionJobVertex

public class ExecutionJobVertex
extends Object
implements AccessExecutionJobVertex, Archiveable<ArchivedExecutionJobVertex>

An ExecutionJobVertex is part of the ExecutionGraph, and the peer to the JobVertex.

The ExecutionJobVertex corresponds to a parallelized operation. It contains an ExecutionVertex for each parallel instance of that operation.

Field Summary

Fields

Modifier and Type	Field and Description
static int	VALUE_NOT_SET

Constructor Summary

Constructors

Constructor and Description
ExecutionJobVertex(ExecutionGraph graph, JobVertex jobVertex, int defaultParallelism, Time timeout, long initialGlobalModVersion, long createTimestamp)

Method Summary

Modifier and Type	Method and Description
Collection<CompletableFuture<Execution>>	allocateResourcesForAll(SlotProvider resourceProvider, boolean queued, LocationPreferenceConstraint locationPreferenceConstraint, Set<AllocationID> allPreviousExecutionGraphAllocationIds, Time allocationTimeout) Acquires a slot for all the execution vertices of this ExecutionJobVertex.
ArchivedExecutionJobVertex	archive()
void	cancel() Cancels all currently running vertex executions.
CompletableFuture<Void>	cancelWithFuture() Cancels all currently running vertex executions.
void	connectToPredecessors(Map<IntermediateDataSetID,IntermediateResult> intermediateDataSets)
void	fail(Throwable t)
StringifiedAccumulatorResult[]	getAggregatedUserAccumulatorsStringified() Returns the aggregated user-defined accumulators as strings.
static ExecutionState	getAggregateJobVertexState(int[] verticesPerState, int parallelism) A utility function that computes an "aggregated" state for the vertex.
ExecutionState	getAggregateState() Returns the aggregated ExecutionState for this job vertex.
CoLocationGroup	getCoLocationGroup()
ExecutionGraph	getGraph()
InputDependencyConstraint	getInputDependencyConstraint()
List<IntermediateResult>	getInputs()
JobID	getJobId()
JobVertex	getJobVertex()
JobVertexID	getJobVertexId() Returns the JobVertexID for this job vertex.
int	getMaxParallelism() Returns the max parallelism for this job vertex.
String	getName() Returns the name for this job vertex.
List<OperatorID>	getOperatorIDs() Returns a list containing the IDs of all operators contained in this execution job vertex.
int	getParallelism() Returns the parallelism for this job vertex.
IntermediateResult[]	getProducedDataSets()
SlotSharingGroup	getSlotSharingGroup()
InputSplitAssigner	getSplitAssigner()
Either<SerializedValue<TaskInformation>,PermanentBlobKey>	getTaskInformationOrBlobKey()
ExecutionVertex[]	getTaskVertices() Returns all execution vertices for this job vertex.
List<OperatorID>	getUserDefinedOperatorIDs() Returns a list containing the alternative IDs of all operators contained in this execution job vertex.
static Map<OperatorID,ExecutionJobVertex>	includeAlternativeOperatorIDs(Map<OperatorID,ExecutionJobVertex> operatorMapping)
static Map<JobVertexID,ExecutionJobVertex>	includeLegacyJobVertexIDs(Map<JobVertexID,ExecutionJobVertex> tasks)
boolean	isMaxParallelismConfigured()
void	resetForNewExecution(long timestamp, long expectedGlobalModVersion)
CompletableFuture<Void>	scheduleAll(SlotProvider slotProvider, boolean queued, LocationPreferenceConstraint locationPreferenceConstraint, Set<AllocationID> allPreviousExecutionGraphAllocationIds) Schedules all execution vertices of this ExecutionJobVertex.
void	setMaxParallelism(int maxParallelismDerived)
CompletableFuture<Void>	suspend()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

VALUE_NOT_SET

public static final int VALUE_NOT_SET

See Also:

Constant Field Values

Constructor Detail

ExecutionJobVertex

public ExecutionJobVertex(ExecutionGraph graph,
                          JobVertex jobVertex,
                          int defaultParallelism,
                          Time timeout,
                          long initialGlobalModVersion,
                          long createTimestamp)
                   throws JobException

Throws:

JobException

Method Detail

getOperatorIDs

public List<OperatorID> getOperatorIDs()

Returns a list containing the IDs of all operators contained in this execution job vertex.

Returns:

list containing the IDs of all contained operators

getUserDefinedOperatorIDs

public List<OperatorID> getUserDefinedOperatorIDs()

Returns a list containing the alternative IDs of all operators contained in this execution job vertex.

Returns:

list containing alternative the IDs of all contained operators

setMaxParallelism

public void setMaxParallelism(int maxParallelismDerived)

getGraph

public ExecutionGraph getGraph()

getJobVertex

public JobVertex getJobVertex()

getName

public String getName()

Description copied from interface: AccessExecutionJobVertex

Returns the name for this job vertex.

Specified by:

getName in interface AccessExecutionJobVertex

Returns:

name for this job vertex.

getParallelism

public int getParallelism()

Description copied from interface: AccessExecutionJobVertex

Returns the parallelism for this job vertex.

Specified by:

getParallelism in interface AccessExecutionJobVertex

Returns:

parallelism for this job vertex.

getMaxParallelism

public int getMaxParallelism()

Description copied from interface: AccessExecutionJobVertex

Returns the max parallelism for this job vertex.

Specified by:

getMaxParallelism in interface AccessExecutionJobVertex

Returns:

max parallelism for this job vertex.

isMaxParallelismConfigured

public boolean isMaxParallelismConfigured()

getJobId

public JobID getJobId()

getJobVertexId

public JobVertexID getJobVertexId()

Description copied from interface: AccessExecutionJobVertex

Returns the JobVertexID for this job vertex.

Specified by:

getJobVertexId in interface AccessExecutionJobVertex

Returns:

JobVertexID for this job vertex.

getTaskVertices

public ExecutionVertex[] getTaskVertices()

Description copied from interface: AccessExecutionJobVertex

Returns all execution vertices for this job vertex.

Specified by:

getTaskVertices in interface AccessExecutionJobVertex

Returns:

all execution vertices for this job vertex

getProducedDataSets

public IntermediateResult[] getProducedDataSets()

getSplitAssigner

public InputSplitAssigner getSplitAssigner()

getSlotSharingGroup

public SlotSharingGroup getSlotSharingGroup()

getCoLocationGroup

public CoLocationGroup getCoLocationGroup()

getInputs

public List<IntermediateResult> getInputs()

getInputDependencyConstraint

public InputDependencyConstraint getInputDependencyConstraint()

getTaskInformationOrBlobKey

public Either<SerializedValue<TaskInformation>,PermanentBlobKey> getTaskInformationOrBlobKey()
                                                                                      throws IOException

Throws:

IOException

getAggregateState

public ExecutionState getAggregateState()

Description copied from interface: AccessExecutionJobVertex

Returns the aggregated ExecutionState for this job vertex.

Specified by:

getAggregateState in interface AccessExecutionJobVertex

Returns:

aggregated state for this job vertex

connectToPredecessors

public void connectToPredecessors(Map<IntermediateDataSetID,IntermediateResult> intermediateDataSets)
                           throws JobException

Throws:

JobException

scheduleAll

public CompletableFuture<Void> scheduleAll(SlotProvider slotProvider,
                                           boolean queued,
                                           LocationPreferenceConstraint locationPreferenceConstraint,
                                           @Nonnull
                                           Set<AllocationID> allPreviousExecutionGraphAllocationIds)

Schedules all execution vertices of this ExecutionJobVertex.

Parameters:

slotProvider - to allocate the slots from

queued - if the allocations can be queued

locationPreferenceConstraint - constraint for the location preferences

allPreviousExecutionGraphAllocationIds - set with all previous allocation ids in the job graph. Can be empty if the allocation ids are not required for scheduling.

Returns:

Future which is completed once all Execution could be deployed

allocateResourcesForAll

public Collection<CompletableFuture<Execution>> allocateResourcesForAll(SlotProvider resourceProvider,
                                                                        boolean queued,
                                                                        LocationPreferenceConstraint locationPreferenceConstraint,
                                                                        @Nonnull
                                                                        Set<AllocationID> allPreviousExecutionGraphAllocationIds,
                                                                        Time allocationTimeout)

Acquires a slot for all the execution vertices of this ExecutionJobVertex. The method returns pairs of the slots and execution attempts, to ease correlation between vertices and execution attempts.

If this method throws an exception, it makes sure to release all so far requested slots.

Parameters:

resourceProvider - The resource provider from whom the slots are requested.

queued - if the allocation can be queued

locationPreferenceConstraint - constraint for the location preferences

allPreviousExecutionGraphAllocationIds - the allocation ids of all previous executions in the execution job graph.

allocationTimeout - timeout for allocating the individual slots

cancel

public void cancel()

Cancels all currently running vertex executions.

cancelWithFuture

public CompletableFuture<Void> cancelWithFuture()

Cancels all currently running vertex executions.

Returns:

A future that is complete once all tasks have canceled.

suspend

public CompletableFuture<Void> suspend()

fail

public void fail(Throwable t)

resetForNewExecution

public void resetForNewExecution(long timestamp,
                                 long expectedGlobalModVersion)
                          throws GlobalModVersionMismatch

Throws:

GlobalModVersionMismatch

getAggregatedUserAccumulatorsStringified

public StringifiedAccumulatorResult[] getAggregatedUserAccumulatorsStringified()

Description copied from interface: AccessExecutionJobVertex

Returns the aggregated user-defined accumulators as strings.

Specified by:

getAggregatedUserAccumulatorsStringified in interface AccessExecutionJobVertex

Returns:

aggregated user-defined accumulators as strings.

archive

public ArchivedExecutionJobVertex archive()

Specified by:

archive in interface Archiveable<ArchivedExecutionJobVertex>

getAggregateJobVertexState

public static ExecutionState getAggregateJobVertexState(int[] verticesPerState,
                                                        int parallelism)

A utility function that computes an "aggregated" state for the vertex.

This state is not used anywhere in the coordination, but can be used for display in dashboards to as a summary for how the particular parallel operation represented by this ExecutionJobVertex is currently behaving.

For example, if at least one parallel task is failed, the aggregate state is failed. If not, and at least one parallel task is cancelling (or cancelled), the aggregate state is cancelling (or cancelled). If all tasks are finished, the aggregate state is finished, and so on.

Parameters:

verticesPerState - The number of vertices in each state (indexed by the ordinal of the ExecutionState values).

parallelism - The parallelism of the ExecutionJobVertex

Returns:

The aggregate state of this ExecutionJobVertex.

includeLegacyJobVertexIDs

public static Map<JobVertexID,ExecutionJobVertex> includeLegacyJobVertexIDs(Map<JobVertexID,ExecutionJobVertex> tasks)

includeAlternativeOperatorIDs

public static Map<OperatorID,ExecutionJobVertex> includeAlternativeOperatorIDs(Map<OperatorID,ExecutionJobVertex> operatorMapping)