org.apache.flink.runtime.scheduler

Interface SchedulerNG

All Superinterfaces:

AutoCloseable, AutoCloseableAsync, GlobalFailureHandler

All Known Implementing Classes:

AdaptiveBatchScheduler, AdaptiveScheduler, DefaultScheduler, SchedulerBase, SpeculativeScheduler

public interface SchedulerNG
extends GlobalFailureHandler, AutoCloseableAsync

Interface for scheduling Flink jobs.

Instances are created via SchedulerNGFactory, and receive a JobGraph when instantiated.

Implementations can expect that methods will not be invoked concurrently. In fact, all invocations will originate from a thread in the ComponentMainThreadExecutor.

Method Summary

Modifier and Type	Method and Description
void	acknowledgeCheckpoint(JobID jobID, ExecutionAttemptID executionAttemptID, long checkpointId, CheckpointMetrics checkpointMetrics, TaskStateSnapshot checkpointState)
void	cancel()
void	declineCheckpoint(DeclineCheckpoint decline)
CompletableFuture<CoordinationResponse>	deliverCoordinationRequestToCoordinator(OperatorID operator, CoordinationRequest request) Delivers a coordination request to the OperatorCoordinator with the given OperatorID and returns the coordinator's response.
void	deliverOperatorEventToCoordinator(ExecutionAttemptID taskExecution, OperatorID operator, OperatorEvent evt) Delivers the given OperatorEvent to the OperatorCoordinator with the given OperatorID.
CompletableFuture<JobStatus>	getJobTerminationFuture()
void	notifyEndOfData(ExecutionAttemptID executionAttemptID) Notifies that the task has reached the end of data.
void	notifyKvStateRegistered(JobID jobId, JobVertexID jobVertexId, KeyGroupRange keyGroupRange, String registrationName, KvStateID kvStateId, InetSocketAddress kvStateServerAddress)
void	notifyKvStateUnregistered(JobID jobId, JobVertexID jobVertexId, KeyGroupRange keyGroupRange, String registrationName)
void	reportCheckpointMetrics(JobID jobID, ExecutionAttemptID executionAttemptID, long checkpointId, CheckpointMetrics checkpointMetrics)
CheckpointStatsSnapshot	requestCheckpointStats() Returns the checkpoint statistics for a given job.
ExecutionGraphInfo	requestJob()
JobDetails	requestJobDetails()
default JobResourceRequirements	requestJobResourceRequirements() Read current job resource requirements.
JobStatus	requestJobStatus()
KvStateLocation	requestKvStateLocation(JobID jobId, String registrationName)
SerializedInputSplit	requestNextInputSplit(JobVertexID vertexID, ExecutionAttemptID executionAttempt)
ExecutionState	requestPartitionState(IntermediateDataSetID intermediateResultId, ResultPartitionID resultPartitionId)
void	startScheduling()
CompletableFuture<String>	stopWithSavepoint(String targetDirectory, boolean terminate, SavepointFormatType formatType)
CompletableFuture<CompletedCheckpoint>	triggerCheckpoint(CheckpointType checkpointType)
CompletableFuture<String>	triggerSavepoint(String targetDirectory, boolean cancelJob, SavepointFormatType formatType)
void	updateAccumulators(AccumulatorSnapshot accumulatorSnapshot)
default void	updateJobResourceRequirements(JobResourceRequirements jobResourceRequirements) Update job resource requirements.
default boolean	updateTaskExecutionState(TaskExecutionState taskExecutionState)
boolean	updateTaskExecutionState(TaskExecutionStateTransition taskExecutionState)

Methods inherited from interface org.apache.flink.runtime.scheduler.GlobalFailureHandler

handleGlobalFailure

Methods inherited from interface org.apache.flink.util.AutoCloseableAsync

close, closeAsync

Method Detail

startScheduling

void startScheduling()

cancel

void cancel()

getJobTerminationFuture

CompletableFuture<JobStatus> getJobTerminationFuture()

updateTaskExecutionState

default boolean updateTaskExecutionState(TaskExecutionState taskExecutionState)

updateTaskExecutionState

boolean updateTaskExecutionState(TaskExecutionStateTransition taskExecutionState)

requestNextInputSplit

SerializedInputSplit requestNextInputSplit(JobVertexID vertexID,
                                           ExecutionAttemptID executionAttempt)
                                    throws IOException

Throws:

IOException

requestPartitionState

ExecutionState requestPartitionState(IntermediateDataSetID intermediateResultId,
                                     ResultPartitionID resultPartitionId)
                              throws PartitionProducerDisposedException

Throws:

PartitionProducerDisposedException

requestJob

ExecutionGraphInfo requestJob()

requestCheckpointStats

CheckpointStatsSnapshot requestCheckpointStats()

Returns the checkpoint statistics for a given job. Although the CheckpointStatsSnapshot is included in the ExecutionGraphInfo, this method is preferred to requestJob() because it is less expensive.

Returns:

checkpoint statistics snapshot for job graph

requestJobStatus

JobStatus requestJobStatus()

requestJobDetails

JobDetails requestJobDetails()

requestKvStateLocation

KvStateLocation requestKvStateLocation(JobID jobId,
                                       String registrationName)
                                throws UnknownKvStateLocation,
                                       FlinkJobNotFoundException

Throws:

UnknownKvStateLocation

FlinkJobNotFoundException

notifyKvStateRegistered

void notifyKvStateRegistered(JobID jobId,
                             JobVertexID jobVertexId,
                             KeyGroupRange keyGroupRange,
                             String registrationName,
                             KvStateID kvStateId,
                             InetSocketAddress kvStateServerAddress)
                      throws FlinkJobNotFoundException

Throws:

FlinkJobNotFoundException

notifyKvStateUnregistered

void notifyKvStateUnregistered(JobID jobId,
                               JobVertexID jobVertexId,
                               KeyGroupRange keyGroupRange,
                               String registrationName)
                        throws FlinkJobNotFoundException

Throws:

FlinkJobNotFoundException

updateAccumulators

void updateAccumulators(AccumulatorSnapshot accumulatorSnapshot)

triggerSavepoint

CompletableFuture<String> triggerSavepoint(@Nullable
                                           String targetDirectory,
                                           boolean cancelJob,
                                           SavepointFormatType formatType)

triggerCheckpoint

CompletableFuture<CompletedCheckpoint> triggerCheckpoint(CheckpointType checkpointType)

acknowledgeCheckpoint

void acknowledgeCheckpoint(JobID jobID,
                           ExecutionAttemptID executionAttemptID,
                           long checkpointId,
                           CheckpointMetrics checkpointMetrics,
                           TaskStateSnapshot checkpointState)

reportCheckpointMetrics

void reportCheckpointMetrics(JobID jobID,
                             ExecutionAttemptID executionAttemptID,
                             long checkpointId,
                             CheckpointMetrics checkpointMetrics)

declineCheckpoint

void declineCheckpoint(DeclineCheckpoint decline)

stopWithSavepoint

CompletableFuture<String> stopWithSavepoint(String targetDirectory,
                                            boolean terminate,
                                            SavepointFormatType formatType)

deliverOperatorEventToCoordinator

void deliverOperatorEventToCoordinator(ExecutionAttemptID taskExecution,
                                       OperatorID operator,
                                       OperatorEvent evt)
                                throws FlinkException

Delivers the given OperatorEvent to the OperatorCoordinator with the given OperatorID.

Failure semantics: If the task manager sends an event for a non-running task or a non-existing operator coordinator, then respond with an exception to the call. If task and coordinator exist, then we assume that the call from the TaskManager was valid, and any bubbling exception needs to cause a job failure

Throws:

FlinkException - Thrown, if the task is not running or no operator/coordinator exists for the given ID.

deliverCoordinationRequestToCoordinator

CompletableFuture<CoordinationResponse> deliverCoordinationRequestToCoordinator(OperatorID operator,
                                                                                CoordinationRequest request)
                                                                         throws FlinkException

Delivers a coordination request to the OperatorCoordinator with the given OperatorID and returns the coordinator's response.

Returns:

A future containing the response.

Throws:

FlinkException - Thrown, if the task is not running, or no operator/coordinator exists for the given ID, or the coordinator cannot handle client events.

notifyEndOfData

void notifyEndOfData(ExecutionAttemptID executionAttemptID)

Notifies that the task has reached the end of data.

Parameters:

executionAttemptID - The execution attempt id.

requestJobResourceRequirements

default JobResourceRequirements requestJobResourceRequirements()

Read current job resource requirements.

Returns:

Current resource requirements.

updateJobResourceRequirements

default void updateJobResourceRequirements(JobResourceRequirements jobResourceRequirements)

Update job resource requirements.

Parameters:

jobResourceRequirements - new resource requirements