org.apache.flink.runtime.executiongraph

Class ExecutionVertex

java.lang.Object

org.apache.flink.runtime.executiongraph.ExecutionVertex

All Implemented Interfaces:

Archiveable<ArchivedExecutionVertex>, AccessExecutionVertex

public class ExecutionVertex
extends Object
implements AccessExecutionVertex, Archiveable<ArchivedExecutionVertex>

The ExecutionVertex is a parallel subtask of the execution. It may be executed once, or several times, each of which time it spawns an Execution.

Field Summary

Fields

Modifier and Type	Field and Description
static int	MAX_DISTINCT_LOCATIONS_TO_CONSIDER

Method Summary

Modifier and Type	Method and Description
ArchivedExecutionVertex	archive()
CompletableFuture<?>	cancel() Cancels this ExecutionVertex.
void	connectSource(int inputNumber, IntermediateResult source, JobEdge edge, int consumerNumber)
void	deploy()
void	deployToSlot(LogicalSlot slot)
void	fail(Throwable t)
ExecutionEdge[][]	getAllInputEdges()
LogicalSlot	getCurrentAssignedResource()
TaskManagerLocation	getCurrentAssignedResourceLocation() Returns the TaskManagerLocation for this execution vertex.
Execution	getCurrentExecutionAttempt() Returns the current execution for this execution vertex.
CompletableFuture<TaskManagerLocation>	getCurrentTaskManagerLocationFuture()
ExecutionGraph	getExecutionGraph()
ExecutionState	getExecutionState() Returns the current ExecutionState for this execution vertex.
Throwable	getFailureCause()
String	getFailureCauseAsString() Returns the exception that caused the job to fail.
ExecutionVertexID	getID()
InputDependencyConstraint	getInputDependencyConstraint()
ExecutionEdge[]	getInputEdges(int input)
JobID	getJobId()
ExecutionJobVertex	getJobVertex()
JobVertexID	getJobvertexId()
AllocationID	getLatestPriorAllocation()
ArchivedExecution	getLatestPriorExecution()
TaskManagerLocation	getLatestPriorLocation() Gets the location where the latest completed/canceled/failed execution of the vertex's task happened.
CoLocationConstraint	getLocationConstraint()
int	getMaxParallelism()
InputSplit	getNextInputSplit(String host)
int	getNumberOfInputs()
int	getParallelSubtaskIndex() Returns the subtask index of this execution vertex.
Optional<TaskManagerLocation>	getPreferredLocationBasedOnState() Gets the preferred location to execute the current task execution attempt, based on the state that the execution attempt will resume.
Collection<CompletableFuture<TaskManagerLocation>>	getPreferredLocations() Gets the overall preferred execution location for this vertex's current execution.
Collection<CompletableFuture<TaskManagerLocation>>	getPreferredLocationsBasedOnInputs() Gets the location preferences of the vertex's current task execution, as determined by the locations of the predecessors from which it receives input data.
Collection<CompletableFuture<TaskManagerLocation>>	getPreferredLocationsBasedOnState() Gets the preferred location to execute the current task execution attempt, based on the state that the execution attempt will resume.
ArchivedExecution	getPriorExecutionAttempt(int attemptNumber) Returns the execution for the given attempt number.
Map<IntermediateResultPartitionID,IntermediateResultPartition>	getProducedPartitions()
ResourceProfile	getResourceProfile()
long	getStateTimestamp(ExecutionState state) Returns the timestamp for the given ExecutionState.
String	getTaskName()
String	getTaskNameWithSubtaskIndex() Creates a simple name representation in the style 'taskname (x/y)', where 'taskname' is the name as returned by getTaskName(), 'x' is the parallel subtask index as returned by getParallelSubtaskIndex()+ 1, and 'y' is the total number of tasks, as returned by getTotalNumberOfParallelSubtasks().
int	getTotalNumberOfParallelSubtasks()
boolean	isLegacyScheduling()
void	markFailed(Throwable t) This method marks the task as failed, but will make no attempt to remove task execution from the task manager.
void	resetForNewExecution()
Execution	resetForNewExecution(long timestamp, long originatingGlobalModVersion) Archives the current Execution and creates a new Execution for this vertex.
CompletableFuture<Void>	scheduleForExecution(SlotProviderStrategy slotProviderStrategy, LocationPreferenceConstraint locationPreferenceConstraint, Set<AllocationID> allPreviousExecutionGraphAllocationIds) Schedules the current execution of this ExecutionVertex.
CompletableFuture<?>	suspend()
String	toString()
void	tryAssignResource(LogicalSlot slot)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

MAX_DISTINCT_LOCATIONS_TO_CONSIDER

public static final int MAX_DISTINCT_LOCATIONS_TO_CONSIDER

See Also:

Constant Field Values

Method Detail

getJobId

public JobID getJobId()

getJobVertex

public ExecutionJobVertex getJobVertex()

getJobvertexId

public JobVertexID getJobvertexId()

getTaskName

public String getTaskName()

getTaskNameWithSubtaskIndex

public String getTaskNameWithSubtaskIndex()

Creates a simple name representation in the style 'taskname (x/y)', where 'taskname' is the name as returned by getTaskName(), 'x' is the parallel subtask index as returned by getParallelSubtaskIndex()+ 1, and 'y' is the total number of tasks, as returned by getTotalNumberOfParallelSubtasks().

Specified by:

getTaskNameWithSubtaskIndex in interface AccessExecutionVertex

Returns:

A simple name representation in the form 'myTask (2/7)'

getTotalNumberOfParallelSubtasks

public int getTotalNumberOfParallelSubtasks()

getMaxParallelism

public int getMaxParallelism()

getResourceProfile

public ResourceProfile getResourceProfile()

getParallelSubtaskIndex

public int getParallelSubtaskIndex()

Description copied from interface: AccessExecutionVertex

Returns the subtask index of this execution vertex.

Specified by:

getParallelSubtaskIndex in interface AccessExecutionVertex

Returns:

subtask index of this execution vertex.

getID

public ExecutionVertexID getID()

getNumberOfInputs

public int getNumberOfInputs()

getInputEdges

public ExecutionEdge[] getInputEdges(int input)

getAllInputEdges

public ExecutionEdge[][] getAllInputEdges()

getLocationConstraint

public CoLocationConstraint getLocationConstraint()

getNextInputSplit

public InputSplit getNextInputSplit(String host)

getCurrentExecutionAttempt

public Execution getCurrentExecutionAttempt()

Description copied from interface: AccessExecutionVertex

Returns the current execution for this execution vertex.

Specified by:

getCurrentExecutionAttempt in interface AccessExecutionVertex

Returns:

current execution

getExecutionState

public ExecutionState getExecutionState()

Description copied from interface: AccessExecutionVertex

Returns the current ExecutionState for this execution vertex.

Specified by:

getExecutionState in interface AccessExecutionVertex

Returns:

execution state for this execution vertex

getStateTimestamp

public long getStateTimestamp(ExecutionState state)

Description copied from interface: AccessExecutionVertex

Returns the timestamp for the given ExecutionState.

Specified by:

getStateTimestamp in interface AccessExecutionVertex

Parameters:

state - state for which the timestamp should be returned

Returns:

timestamp for the given state

getFailureCauseAsString

public String getFailureCauseAsString()

Description copied from interface: AccessExecutionVertex

Returns the exception that caused the job to fail. This is the first root exception that was not recoverable and triggered job failure.

Specified by:

getFailureCauseAsString in interface AccessExecutionVertex

Returns:

failure exception as a string, or "(null)"

getFailureCause

public Throwable getFailureCause()

getCurrentTaskManagerLocationFuture

public CompletableFuture<TaskManagerLocation> getCurrentTaskManagerLocationFuture()

getCurrentAssignedResource

public LogicalSlot getCurrentAssignedResource()

getCurrentAssignedResourceLocation

public TaskManagerLocation getCurrentAssignedResourceLocation()

Description copied from interface: AccessExecutionVertex

Returns the TaskManagerLocation for this execution vertex.

Specified by:

getCurrentAssignedResourceLocation in interface AccessExecutionVertex

Returns:

taskmanager location for this execution vertex.

getPriorExecutionAttempt

@Nullable
public ArchivedExecution getPriorExecutionAttempt(int attemptNumber)

Description copied from interface: AccessExecutionVertex

Returns the execution for the given attempt number.

Specified by:

getPriorExecutionAttempt in interface AccessExecutionVertex

Parameters:

attemptNumber - attempt number of execution to be returned

Returns:

execution for the given attempt number

getLatestPriorExecution

public ArchivedExecution getLatestPriorExecution()

getLatestPriorLocation

public TaskManagerLocation getLatestPriorLocation()

Gets the location where the latest completed/canceled/failed execution of the vertex's task happened.

Returns:

The latest prior execution location, or null, if there is none, yet.

getLatestPriorAllocation

public AllocationID getLatestPriorAllocation()

getExecutionGraph

public ExecutionGraph getExecutionGraph()

getProducedPartitions

public Map<IntermediateResultPartitionID,IntermediateResultPartition> getProducedPartitions()

getInputDependencyConstraint

public InputDependencyConstraint getInputDependencyConstraint()

connectSource

public void connectSource(int inputNumber,
                          IntermediateResult source,
                          JobEdge edge,
                          int consumerNumber)

getPreferredLocations

public Collection<CompletableFuture<TaskManagerLocation>> getPreferredLocations()

Gets the overall preferred execution location for this vertex's current execution. The preference is determined as follows:

1. If the task execution has state to load (from a checkpoint), then the location preference is the location of the previous execution (if there is a previous execution attempt).
2. If the task execution has no state or no previous location, then the location preference is based on the task's inputs.

These rules should result in the following behavior:

• Stateless tasks are always scheduled based on co-location with inputs.
• Stateful tasks are on their initial attempt executed based on co-location with inputs.
• Repeated executions of stateful tasks try to co-locate the execution with its state.

Returns:

The preferred execution locations for the execution attempt.

See Also:

getPreferredLocationsBasedOnState(), getPreferredLocationsBasedOnInputs()

getPreferredLocationsBasedOnState

public Collection<CompletableFuture<TaskManagerLocation>> getPreferredLocationsBasedOnState()

Gets the preferred location to execute the current task execution attempt, based on the state that the execution attempt will resume.

Returns:

A size-one collection with the location preference, or null, if there is no location preference based on the state.

getPreferredLocationBasedOnState

public Optional<TaskManagerLocation> getPreferredLocationBasedOnState()

Gets the preferred location to execute the current task execution attempt, based on the state that the execution attempt will resume.

getPreferredLocationsBasedOnInputs

public Collection<CompletableFuture<TaskManagerLocation>> getPreferredLocationsBasedOnInputs()

Gets the location preferences of the vertex's current task execution, as determined by the locations of the predecessors from which it receives input data. If there are more than MAX_DISTINCT_LOCATIONS_TO_CONSIDER different locations of source data, this method returns null to indicate no location preference.

Returns:

The preferred locations based in input streams, or an empty iterable, if there is no input-based preference.

resetForNewExecution

public Execution resetForNewExecution(long timestamp,
                                      long originatingGlobalModVersion)
                               throws GlobalModVersionMismatch

Archives the current Execution and creates a new Execution for this vertex.

This method atomically checks if the ExecutionGraph is still of an expected global mod. version and replaces the execution if that is the case. If the ExecutionGraph has increased its global mod. version in the meantime, this operation fails.

This mechanism can be used to prevent conflicts between various concurrent recovery and reconfiguration actions in a similar way as "optimistic concurrency control".

Parameters:

timestamp - The creation timestamp for the new Execution

originatingGlobalModVersion -

Returns:

Returns the new created Execution.

Throws:

GlobalModVersionMismatch - Thrown, if the execution graph has a new global mod version than the one passed to this message.

resetForNewExecution

public void resetForNewExecution()

scheduleForExecution

public CompletableFuture<Void> scheduleForExecution(SlotProviderStrategy slotProviderStrategy,
                                                    LocationPreferenceConstraint locationPreferenceConstraint,
                                                    @Nonnull
                                                    Set<AllocationID> allPreviousExecutionGraphAllocationIds)

Schedules the current execution of this ExecutionVertex.

Parameters:

slotProviderStrategy - to allocate the slots from

locationPreferenceConstraint - constraint for the location preferences

allPreviousExecutionGraphAllocationIds - set with all previous allocation ids in the job graph. Can be empty if the allocation ids are not required for scheduling.

Returns:

Future which is completed once the execution is deployed. The future can also completed exceptionally.

tryAssignResource

public void tryAssignResource(LogicalSlot slot)

deploy

public void deploy()
            throws JobException

Throws:

JobException

deployToSlot

@VisibleForTesting
public void deployToSlot(LogicalSlot slot)
                                     throws JobException

Throws:

JobException

cancel

public CompletableFuture<?> cancel()

Cancels this ExecutionVertex.

Returns:

A future that completes once the execution has reached its final state.

suspend

public CompletableFuture<?> suspend()

fail

public void fail(Throwable t)

markFailed

public void markFailed(Throwable t)

This method marks the task as failed, but will make no attempt to remove task execution from the task manager. It is intended for cases where the task is known not to be deployed yet.

Parameters:

t - The exception that caused the task to fail.

toString

public String toString()

Overrides:

toString in class Object

archive

public ArchivedExecutionVertex archive()

Specified by:

archive in interface Archiveable<ArchivedExecutionVertex>

isLegacyScheduling

public boolean isLegacyScheduling()