public interface ExecutionGraph extends AccessExecutionGraph
The execution graph consists of the following constructs:
ExecutionJobVertex
represents one vertex from the JobGraph (usually one
operation like "map" or "join") during execution. It holds the aggregated state of all
parallel subtasks. The ExecutionJobVertex is identified inside the graph by the JobVertexID
, which it takes from the JobGraph's corresponding JobVertex.
ExecutionVertex
represents one parallel subtask. For each ExecutionJobVertex,
there are as many ExecutionVertices as the parallelism. The ExecutionVertex is identified
by the ExecutionJobVertex and the index of the parallel subtask
Execution
is one attempt to execute a ExecutionVertex. There may be multiple
Executions for the ExecutionVertex, in case of a failure, or in the case where some data
needs to be recomputed because it is no longer available when requested by later
operations. An Execution is always identified by an ExecutionAttemptID
. All
messages between the JobManager and the TaskManager about deployment of tasks and updates
in the task status always use the ExecutionAttemptID to address the message receiver.
getAccumulatorResultsStringified, getAccumulatorsSerialized, getArchivedExecutionConfig, getCheckpointCoordinatorConfiguration, getCheckpointStatsSnapshot, getCheckpointStorageName, getFailureInfo, getJobID, getJobName, getJsonPlan, getState, getStateBackendName, getStatusTimestamp, isStoppable
void start(@Nonnull ComponentMainThreadExecutor jobMasterMainThreadExecutor)
SchedulingTopology getSchedulingTopology()
void enableCheckpointing(CheckpointCoordinatorConfiguration chkConfig, List<MasterTriggerRestoreHook<?>> masterHooks, CheckpointIDCounter checkpointIDCounter, CompletedCheckpointStore checkpointStore, StateBackend checkpointStateBackend, CheckpointStorage checkpointStorage, CheckpointStatsTracker statsTracker, CheckpointsCleaner checkpointsCleaner)
@Nullable CheckpointCoordinator getCheckpointCoordinator()
KvStateLocationRegistry getKvStateLocationRegistry()
void setJsonPlan(String jsonPlan)
Configuration getJobConfiguration()
Throwable getFailureCause()
Iterable<ExecutionJobVertex> getVerticesTopologically()
AccessExecutionGraph
getVerticesTopologically
in interface AccessExecutionGraph
Iterable<ExecutionVertex> getAllExecutionVertices()
AccessExecutionGraph
getAllExecutionVertices
in interface AccessExecutionGraph
ExecutionJobVertex getJobVertex(JobVertexID id)
AccessExecutionGraph
JobVertexID
.getJobVertex
in interface AccessExecutionGraph
id
- id of job vertex to be returnednull
Map<JobVertexID,ExecutionJobVertex> getAllVertices()
AccessExecutionGraph
getAllVertices
in interface AccessExecutionGraph
long getNumberOfRestarts()
int getTotalNumberOfVertices()
Map<IntermediateDataSetID,IntermediateResult> getAllIntermediateResults()
Map<String,OptionalFailure<Accumulator<?,?>>> aggregateUserAccumulators()
void updateAccumulators(AccumulatorSnapshot accumulatorSnapshot)
accumulatorSnapshot
- The serialized flink and user-defined accumulatorsvoid setInternalTaskFailuresListener(InternalFailuresListener internalTaskFailuresListener)
void attachJobGraph(List<JobVertex> topologiallySorted) throws JobException
JobException
void transitionToRunning()
void cancel()
void suspend(Throwable suspensionCause)
The JobStatus will be directly set to JobStatus.SUSPENDED
iff the current state is
not a terminal state. All ExecutionJobVertices will be canceled and the onTerminalState() is
executed.
The JobStatus.SUSPENDED
state is a local terminal state which stops the execution
of the job but does not remove the job from the HA job store so that it can be recovered by
another JobManager.
suspensionCause
- Cause of the suspensionvoid failJob(Throwable cause, long timestamp)
CompletableFuture<JobStatus> getTerminationFuture()
ExecutionGraph
. The termination future is
completed with the terminal JobStatus
once the ExecutionGraph reaches this terminal
state and all Execution
have been terminated.ExecutionGraph
.@VisibleForTesting JobStatus waitUntilTerminal() throws InterruptedException
InterruptedException
void incrementRestarts()
void initFailureCause(Throwable t, long timestamp)
boolean updateState(TaskExecutionStateTransition state)
state
- The state update.void notifyPartitionDataAvailable(ResultPartitionID partitionId)
partitionId
- specifying the result partition whose data have become availableMap<ExecutionAttemptID,Execution> getRegisteredExecutions()
void registerJobStatusListener(JobStatusListener listener)
ResultPartitionAvailabilityChecker getResultPartitionAvailabilityChecker()
int getNumFinishedVertices()
@Nonnull ComponentMainThreadExecutor getJobMasterMainThreadExecutor()
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.