org.apache.flink.runtime.executiongraph

Class ExecutionVertex

java.lang.Object

org.apache.flink.runtime.executiongraph.ExecutionVertex

All Implemented Interfaces:

Archiveable<ArchivedExecutionVertex>, AccessExecutionVertex

public class ExecutionVertex
extends Object
implements AccessExecutionVertex, Archiveable<ArchivedExecutionVertex>

The ExecutionVertex is a parallel subtask of the execution. It may be executed once, or several times, each of which time it spawns an Execution.

Constructor Summary

Constructors

Constructor and Description
ExecutionVertex(ExecutionJobVertex jobVertex, int subTaskIndex, IntermediateResult[] producedDataSets, Time timeout, long initialGlobalModVersion, long createTimestamp, int maxPriorExecutionHistoryLength) Creates an ExecutionVertex.

Method Summary

Modifier and Type	Method and Description
ArchivedExecutionVertex	archive()
void	cachePartitionInfo(PartialInputChannelDeploymentDescriptor partitionInfo)
CompletableFuture<?>	cancel() Cancels this ExecutionVertex.
void	connectSource(int inputNumber, IntermediateResult source, JobEdge edge, int consumerNumber)
void	deployToSlot(SimpleSlot slot)
void	fail(Throwable t)
LogicalSlot	getCurrentAssignedResource()
TaskManagerLocation	getCurrentAssignedResourceLocation() Returns the TaskManagerLocation for this execution vertex.
Execution	getCurrentExecutionAttempt() Returns the current execution for this execution vertex.
CompletableFuture<TaskManagerLocation>	getCurrentTaskManagerLocationFuture()
ExecutionGraph	getExecutionGraph()
ExecutionState	getExecutionState() Returns the current ExecutionState for this execution vertex.
Throwable	getFailureCause()
String	getFailureCauseAsString() Returns the exception that caused the job to fail.
ExecutionEdge[]	getInputEdges(int input)
JobID	getJobId()
ExecutionJobVertex	getJobVertex()
JobVertexID	getJobvertexId()
AllocationID	getLatestPriorAllocation()
ArchivedExecution	getLatestPriorExecution()
TaskManagerLocation	getLatestPriorLocation() Gets the location where the latest completed/canceled/failed execution of the vertex's task happened.
CoLocationConstraint	getLocationConstraint()
int	getMaxParallelism()
int	getNumberOfInputs()
int	getParallelSubtaskIndex() Returns the subtask index of this execution vertex.
Collection<CompletableFuture<TaskManagerLocation>>	getPreferredLocations() Gets the overall preferred execution location for this vertex's current execution.
Collection<CompletableFuture<TaskManagerLocation>>	getPreferredLocationsBasedOnInputs() Gets the location preferences of the vertex's current task execution, as determined by the locations of the predecessors from which it receives input data.
Collection<CompletableFuture<TaskManagerLocation>>	getPreferredLocationsBasedOnState() Gets the preferred location to execute the current task execution attempt, based on the state that the execution attempt will resume.
ArchivedExecution	getPriorExecutionAttempt(int attemptNumber) Returns the execution for the given attempt number.
Map<IntermediateResultPartitionID,IntermediateResultPartition>	getProducedPartitions()
long	getStateTimestamp(ExecutionState state) Returns the timestamp for the given ExecutionState.
String	getTaskName()
String	getTaskNameWithSubtaskIndex() Creates a simple name representation in the style 'taskname (x/y)', where 'taskname' is the name as returned by getTaskName(), 'x' is the parallel subtask index as returned by getParallelSubtaskIndex()+ 1, and 'y' is the total number of tasks, as returned by getTotalNumberOfParallelSubtasks().
int	getTotalNumberOfParallelSubtasks()
Execution	resetForNewExecution(long timestamp, long originatingGlobalModVersion) Archives the current Execution and creates a new Execution for this vertex.
CompletableFuture<Void>	scheduleForExecution(SlotProvider slotProvider, boolean queued, LocationPreferenceConstraint locationPreferenceConstraint) Schedules the current execution of this ExecutionVertex.
void	stop()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ExecutionVertex

public ExecutionVertex(ExecutionJobVertex jobVertex,
                       int subTaskIndex,
                       IntermediateResult[] producedDataSets,
                       Time timeout,
                       long initialGlobalModVersion,
                       long createTimestamp,
                       int maxPriorExecutionHistoryLength)

Creates an ExecutionVertex.

Parameters:

timeout - The RPC timeout to use for deploy / cancel calls

initialGlobalModVersion - The global modification version to initialize the first Execution with.

createTimestamp - The timestamp for the vertex creation, used to initialize the first Execution with.

maxPriorExecutionHistoryLength - The number of prior Executions (= execution attempts) to keep.

Method Detail

getJobId

public JobID getJobId()

getJobVertex

public ExecutionJobVertex getJobVertex()

getJobvertexId

public JobVertexID getJobvertexId()

getTaskName

public String getTaskName()

getTaskNameWithSubtaskIndex

public String getTaskNameWithSubtaskIndex()

Creates a simple name representation in the style 'taskname (x/y)', where 'taskname' is the name as returned by getTaskName(), 'x' is the parallel subtask index as returned by getParallelSubtaskIndex()+ 1, and 'y' is the total number of tasks, as returned by getTotalNumberOfParallelSubtasks().

Specified by:

getTaskNameWithSubtaskIndex in interface AccessExecutionVertex

Returns:

A simple name representation in the form 'myTask (2/7)'

getTotalNumberOfParallelSubtasks

public int getTotalNumberOfParallelSubtasks()

getMaxParallelism

public int getMaxParallelism()

getParallelSubtaskIndex

public int getParallelSubtaskIndex()

Description copied from interface: AccessExecutionVertex

Returns the subtask index of this execution vertex.

Specified by:

getParallelSubtaskIndex in interface AccessExecutionVertex

Returns:

subtask index of this execution vertex.

getNumberOfInputs

public int getNumberOfInputs()

getInputEdges

public ExecutionEdge[] getInputEdges(int input)

getLocationConstraint

public CoLocationConstraint getLocationConstraint()

getCurrentExecutionAttempt

public Execution getCurrentExecutionAttempt()

Description copied from interface: AccessExecutionVertex

Returns the current execution for this execution vertex.

Specified by:

getCurrentExecutionAttempt in interface AccessExecutionVertex

Returns:

current execution

getExecutionState

public ExecutionState getExecutionState()

Description copied from interface: AccessExecutionVertex

Returns the current ExecutionState for this execution vertex.

Specified by:

getExecutionState in interface AccessExecutionVertex

Returns:

execution state for this execution vertex

getStateTimestamp

public long getStateTimestamp(ExecutionState state)

Description copied from interface: AccessExecutionVertex

Returns the timestamp for the given ExecutionState.

Specified by:

getStateTimestamp in interface AccessExecutionVertex

Parameters:

state - state for which the timestamp should be returned

Returns:

timestamp for the given state

getFailureCauseAsString

public String getFailureCauseAsString()

Description copied from interface: AccessExecutionVertex

Returns the exception that caused the job to fail. This is the first root exception that was not recoverable and triggered job failure.

Specified by:

getFailureCauseAsString in interface AccessExecutionVertex

Returns:

failure exception as a string, or "(null)"

getFailureCause

public Throwable getFailureCause()

getCurrentTaskManagerLocationFuture

public CompletableFuture<TaskManagerLocation> getCurrentTaskManagerLocationFuture()

getCurrentAssignedResource

public LogicalSlot getCurrentAssignedResource()

getCurrentAssignedResourceLocation

public TaskManagerLocation getCurrentAssignedResourceLocation()

Description copied from interface: AccessExecutionVertex

Returns the TaskManagerLocation for this execution vertex.

Specified by:

getCurrentAssignedResourceLocation in interface AccessExecutionVertex

Returns:

taskmanager location for this execution vertex.

getPriorExecutionAttempt

public ArchivedExecution getPriorExecutionAttempt(int attemptNumber)

Description copied from interface: AccessExecutionVertex

Returns the execution for the given attempt number.

Specified by:

getPriorExecutionAttempt in interface AccessExecutionVertex

Parameters:

attemptNumber - attempt number of execution to be returned

Returns:

execution for the given attempt number

getLatestPriorExecution

public ArchivedExecution getLatestPriorExecution()

getLatestPriorLocation

public TaskManagerLocation getLatestPriorLocation()

Gets the location where the latest completed/canceled/failed execution of the vertex's task happened.

Returns:

The latest prior execution location, or null, if there is none, yet.

getLatestPriorAllocation

public AllocationID getLatestPriorAllocation()

getExecutionGraph

public ExecutionGraph getExecutionGraph()

getProducedPartitions

public Map<IntermediateResultPartitionID,IntermediateResultPartition> getProducedPartitions()

connectSource

public void connectSource(int inputNumber,
                          IntermediateResult source,
                          JobEdge edge,
                          int consumerNumber)

getPreferredLocations

public Collection<CompletableFuture<TaskManagerLocation>> getPreferredLocations()

Gets the overall preferred execution location for this vertex's current execution. The preference is determined as follows:

1. If the task execution has state to load (from a checkpoint), then the location preference is the location of the previous execution (if there is a previous execution attempt).
2. If the task execution has no state or no previous location, then the location preference is based on the task's inputs.

These rules should result in the following behavior:

• Stateless tasks are always scheduled based on co-location with inputs.
• Stateful tasks are on their initial attempt executed based on co-location with inputs.
• Repeated executions of stateful tasks try to co-locate the execution with its state.

Returns:

The preferred execution locations for the execution attempt.

See Also:

getPreferredLocationsBasedOnState(), getPreferredLocationsBasedOnInputs()

getPreferredLocationsBasedOnState

public Collection<CompletableFuture<TaskManagerLocation>> getPreferredLocationsBasedOnState()

Gets the preferred location to execute the current task execution attempt, based on the state that the execution attempt will resume.

Returns:

A size-one collection with the location preference, or null, if there is no location preference based on the state.

getPreferredLocationsBasedOnInputs

public Collection<CompletableFuture<TaskManagerLocation>> getPreferredLocationsBasedOnInputs()

Gets the location preferences of the vertex's current task execution, as determined by the locations of the predecessors from which it receives input data. If there are more than MAX_DISTINCT_LOCATIONS_TO_CONSIDER different locations of source data, this method returns null to indicate no location preference.

Returns:

The preferred locations based in input streams, or an empty iterable, if there is no input-based preference.

resetForNewExecution

public Execution resetForNewExecution(long timestamp,
                                      long originatingGlobalModVersion)
                               throws GlobalModVersionMismatch

Archives the current Execution and creates a new Execution for this vertex.

This method atomically checks if the ExecutionGraph is still of an expected global mod. version and replaces the execution if that is the case. If the ExecutionGraph has increased its global mod. version in the meantime, this operation fails.

This mechanism can be used to prevent conflicts between various concurrent recovery and reconfiguration actions in a similar way as "optimistic concurrency control".

Parameters:

timestamp - The creation timestamp for the new Execution

originatingGlobalModVersion -

Returns:

Returns the new created Execution.

Throws:

GlobalModVersionMismatch - Thrown, if the execution graph has a new global mod version than the one passed to this message.

scheduleForExecution

public CompletableFuture<Void> scheduleForExecution(SlotProvider slotProvider,
                                                    boolean queued,
                                                    LocationPreferenceConstraint locationPreferenceConstraint)

Schedules the current execution of this ExecutionVertex.

Parameters:

slotProvider - to allocate the slots from

queued - if the allocation can be queued

locationPreferenceConstraint - constraint for the location preferences

Returns:

Future which is completed once the execution is deployed. The future can also completed exceptionally.

deployToSlot

@VisibleForTesting
public void deployToSlot(SimpleSlot slot)
                                     throws JobException

Throws:

JobException

cancel

public CompletableFuture<?> cancel()

Cancels this ExecutionVertex.

Returns:

A future that completes once the execution has reached its final state.

stop

public void stop()

fail

public void fail(Throwable t)

cachePartitionInfo

public void cachePartitionInfo(PartialInputChannelDeploymentDescriptor partitionInfo)

toString

public String toString()

Overrides:

toString in class Object

archive

public ArchivedExecutionVertex archive()

Specified by:

archive in interface Archiveable<ArchivedExecutionVertex>