org.apache.flink.runtime.scheduler

Interface SchedulerNG

All Known Implementing Classes:

DefaultScheduler, SchedulerBase

public interface SchedulerNG

Interface for scheduling Flink jobs.

Instances are created via SchedulerNGFactory, and receive a JobGraph when instantiated.

Implementations can expect that methods will not be invoked concurrently. In fact, all invocations will originate from a thread in the ComponentMainThreadExecutor, which will be passed via setMainThreadExecutor(ComponentMainThreadExecutor).

Method Summary

Modifier and Type	Method and Description
void	acknowledgeCheckpoint(JobID jobID, ExecutionAttemptID executionAttemptID, long checkpointId, CheckpointMetrics checkpointMetrics, TaskStateSnapshot checkpointState)
void	cancel()
void	declineCheckpoint(DeclineCheckpoint decline)
CompletableFuture<CoordinationResponse>	deliverCoordinationRequestToCoordinator(OperatorID operator, CoordinationRequest request) Delivers a coordination request to the OperatorCoordinator with the given OperatorID and returns the coordinator's response.
void	deliverOperatorEventToCoordinator(ExecutionAttemptID taskExecution, OperatorID operator, OperatorEvent evt) Delivers the given OperatorEvent to the OperatorCoordinator with the given OperatorID.
CompletableFuture<Void>	getTerminationFuture()
void	handleGlobalFailure(Throwable cause)
void	notifyKvStateRegistered(JobID jobId, JobVertexID jobVertexId, KeyGroupRange keyGroupRange, String registrationName, KvStateID kvStateId, InetSocketAddress kvStateServerAddress)
void	notifyKvStateUnregistered(JobID jobId, JobVertexID jobVertexId, KeyGroupRange keyGroupRange, String registrationName)
void	registerJobStatusListener(JobStatusListener jobStatusListener)
ArchivedExecutionGraph	requestJob()
JobDetails	requestJobDetails()
JobStatus	requestJobStatus()
KvStateLocation	requestKvStateLocation(JobID jobId, String registrationName)
SerializedInputSplit	requestNextInputSplit(JobVertexID vertexID, ExecutionAttemptID executionAttempt)
Optional<OperatorBackPressureStats>	requestOperatorBackPressureStats(JobVertexID jobVertexId)
ExecutionState	requestPartitionState(IntermediateDataSetID intermediateResultId, ResultPartitionID resultPartitionId)
void	scheduleOrUpdateConsumers(ResultPartitionID partitionID)
void	setMainThreadExecutor(ComponentMainThreadExecutor mainThreadExecutor)
void	startScheduling()
CompletableFuture<String>	stopWithSavepoint(String targetDirectory, boolean terminate)
void	suspend(Throwable cause)
CompletableFuture<String>	triggerSavepoint(String targetDirectory, boolean cancelJob)
void	updateAccumulators(AccumulatorSnapshot accumulatorSnapshot)
default boolean	updateTaskExecutionState(TaskExecutionState taskExecutionState)
boolean	updateTaskExecutionState(TaskExecutionStateTransition taskExecutionState)

Method Detail

setMainThreadExecutor

void setMainThreadExecutor(ComponentMainThreadExecutor mainThreadExecutor)

registerJobStatusListener

void registerJobStatusListener(JobStatusListener jobStatusListener)

startScheduling

void startScheduling()

suspend

void suspend(Throwable cause)

cancel

void cancel()

getTerminationFuture

CompletableFuture<Void> getTerminationFuture()

handleGlobalFailure

void handleGlobalFailure(Throwable cause)

updateTaskExecutionState

default boolean updateTaskExecutionState(TaskExecutionState taskExecutionState)

updateTaskExecutionState

boolean updateTaskExecutionState(TaskExecutionStateTransition taskExecutionState)

requestNextInputSplit

SerializedInputSplit requestNextInputSplit(JobVertexID vertexID,
                                           ExecutionAttemptID executionAttempt)
                                    throws IOException

Throws:

IOException

requestPartitionState

ExecutionState requestPartitionState(IntermediateDataSetID intermediateResultId,
                                     ResultPartitionID resultPartitionId)
                              throws PartitionProducerDisposedException

Throws:

PartitionProducerDisposedException

scheduleOrUpdateConsumers

void scheduleOrUpdateConsumers(ResultPartitionID partitionID)

requestJob

ArchivedExecutionGraph requestJob()

requestJobStatus

JobStatus requestJobStatus()

requestJobDetails

JobDetails requestJobDetails()

requestKvStateLocation

KvStateLocation requestKvStateLocation(JobID jobId,
                                       String registrationName)
                                throws UnknownKvStateLocation,
                                       FlinkJobNotFoundException

Throws:

UnknownKvStateLocation

FlinkJobNotFoundException

notifyKvStateRegistered

void notifyKvStateRegistered(JobID jobId,
                             JobVertexID jobVertexId,
                             KeyGroupRange keyGroupRange,
                             String registrationName,
                             KvStateID kvStateId,
                             InetSocketAddress kvStateServerAddress)
                      throws FlinkJobNotFoundException

Throws:

FlinkJobNotFoundException

notifyKvStateUnregistered

void notifyKvStateUnregistered(JobID jobId,
                               JobVertexID jobVertexId,
                               KeyGroupRange keyGroupRange,
                               String registrationName)
                        throws FlinkJobNotFoundException

Throws:

FlinkJobNotFoundException

updateAccumulators

void updateAccumulators(AccumulatorSnapshot accumulatorSnapshot)

requestOperatorBackPressureStats

Optional<OperatorBackPressureStats> requestOperatorBackPressureStats(JobVertexID jobVertexId)
                                                              throws FlinkException

Throws:

FlinkException

triggerSavepoint

CompletableFuture<String> triggerSavepoint(@Nullable
                                           String targetDirectory,
                                           boolean cancelJob)

acknowledgeCheckpoint

void acknowledgeCheckpoint(JobID jobID,
                           ExecutionAttemptID executionAttemptID,
                           long checkpointId,
                           CheckpointMetrics checkpointMetrics,
                           TaskStateSnapshot checkpointState)

declineCheckpoint

void declineCheckpoint(DeclineCheckpoint decline)

stopWithSavepoint

CompletableFuture<String> stopWithSavepoint(String targetDirectory,
                                            boolean terminate)

deliverOperatorEventToCoordinator

void deliverOperatorEventToCoordinator(ExecutionAttemptID taskExecution,
                                       OperatorID operator,
                                       OperatorEvent evt)
                                throws FlinkException

Delivers the given OperatorEvent to the OperatorCoordinator with the given OperatorID.

Failure semantics: If the task manager sends an event for a non-running task or a non-existing operator coordinator, then respond with an exception to the call. If task and coordinator exist, then we assume that the call from the TaskManager was valid, and any bubbling exception needs to cause a job failure

Throws:

FlinkException - Thrown, if the task is not running or no operator/coordinator exists for the given ID.

deliverCoordinationRequestToCoordinator

CompletableFuture<CoordinationResponse> deliverCoordinationRequestToCoordinator(OperatorID operator,
                                                                                CoordinationRequest request)
                                                                         throws FlinkException

Delivers a coordination request to the OperatorCoordinator with the given OperatorID and returns the coordinator's response.

Returns:

A future containing the response.

Throws:

FlinkException - Thrown, if the task is not running, or no operator/coordinator exists for the given ID, or the coordinator cannot handle client events.