org.apache.flink.runtime.jobmaster

Interface JobMasterGateway

All Superinterfaces:

CheckpointCoordinatorGateway, FencedRpcGateway<JobMasterId>, KvStateLocationOracle, KvStateRegistryGateway, RpcGateway

All Known Implementing Classes:

JobMaster

public interface JobMasterGateway
extends CheckpointCoordinatorGateway, FencedRpcGateway<JobMasterId>, KvStateLocationOracle, KvStateRegistryGateway

JobMaster rpc gateway interface.

Method Summary

Modifier and Type	Method and Description
CompletableFuture<Acknowledge>	cancel(Time timeout) Cancels the currently executed job.
void	disconnectResourceManager(ResourceManagerId resourceManagerId, Exception cause) Disconnects the resource manager from the job manager because of the given cause.
CompletableFuture<Acknowledge>	disconnectTaskManager(ResourceID resourceID, Exception cause) Disconnects the given TaskExecutor from the JobMaster.
void	failSlot(ResourceID taskManagerId, AllocationID allocationId, Exception cause) Fails the slot with the given allocation id and cause.
void	heartbeatFromResourceManager(ResourceID resourceID) Sends heartbeat request from the resource manager.
void	heartbeatFromTaskManager(ResourceID resourceID, AccumulatorReport accumulatorReport) Sends the heartbeat to job manager from task manager.
void	notifyAllocationFailure(AllocationID allocationID, Exception cause) Notifies that the allocation has failed.
CompletableFuture<Collection<SlotOffer>>	offerSlots(ResourceID taskManagerId, Collection<SlotOffer> slots, Time timeout) Offers the given slots to the job manager.
CompletableFuture<RegistrationResponse>	registerTaskManager(String taskManagerRpcAddress, TaskManagerLocation taskManagerLocation, Time timeout) Registers the task manager at the job manager.
CompletableFuture<ClassloadingProps>	requestClassloadingProps() Request the classloading props of this job.
CompletableFuture<ArchivedExecutionGraph>	requestJob(Time timeout) Requests the ArchivedExecutionGraph of the executed job.
CompletableFuture<JobDetails>	requestJobDetails(Time timeout) Request the details of the executed job.
CompletableFuture<JobStatus>	requestJobStatus(Time timeout) Requests the current job status.
CompletableFuture<SerializedInputSplit>	requestNextInputSplit(JobVertexID vertexID, ExecutionAttemptID executionAttempt) Requests the next input split for the ExecutionJobVertex.
CompletableFuture<OperatorBackPressureStatsResponse>	requestOperatorBackPressureStats(JobVertexID jobVertexId) Requests the statistics on operator back pressure.
CompletableFuture<ExecutionState>	requestPartitionState(IntermediateDataSetID intermediateResultId, ResultPartitionID partitionId) Requests the current state of the partition.
CompletableFuture<Acknowledge>	rescaleJob(int newParallelism, RescalingBehaviour rescalingBehaviour, Time timeout) Triggers rescaling of the executed job.
CompletableFuture<Acknowledge>	rescaleOperators(Collection<JobVertexID> operators, int newParallelism, RescalingBehaviour rescalingBehaviour, Time timeout) Triggers rescaling of the given set of operators.
CompletableFuture<Acknowledge>	scheduleOrUpdateConsumers(ResultPartitionID partitionID, Time timeout) Notifies the JobManager about available data for a produced partition.
CompletableFuture<Acknowledge>	stop(Time timeout) Cancel the currently executed job.
CompletableFuture<String>	triggerSavepoint(String targetDirectory, boolean cancelJob, Time timeout) Triggers taking a savepoint of the executed job.
CompletableFuture<Acknowledge>	updateTaskExecutionState(TaskExecutionState taskExecutionState) Updates the task execution state for a given task.

Methods inherited from interface org.apache.flink.runtime.checkpoint.CheckpointCoordinatorGateway

acknowledgeCheckpoint, declineCheckpoint

Methods inherited from interface org.apache.flink.runtime.rpc.FencedRpcGateway

getFencingToken

Methods inherited from interface org.apache.flink.runtime.rpc.RpcGateway

getAddress, getHostname

Methods inherited from interface org.apache.flink.runtime.jobmaster.KvStateLocationOracle

requestKvStateLocation

Methods inherited from interface org.apache.flink.runtime.jobmaster.KvStateRegistryGateway

notifyKvStateRegistered, notifyKvStateUnregistered

Method Detail

cancel

CompletableFuture<Acknowledge> cancel(Time timeout)

Cancels the currently executed job.

Parameters:

timeout - of this operation

Returns:

Future acknowledge of the operation

stop

CompletableFuture<Acknowledge> stop(Time timeout)

Cancel the currently executed job.

Parameters:

timeout - of this operation

Returns:

Future acknowledge if the cancellation was successful

rescaleJob

CompletableFuture<Acknowledge> rescaleJob(int newParallelism,
                                          RescalingBehaviour rescalingBehaviour,
                                          Time timeout)

Triggers rescaling of the executed job.

Parameters:

newParallelism - new parallelism of the job

rescalingBehaviour - defining how strict the rescaling has to be executed

timeout - of this operation

Returns:

Future which is completed with Acknowledge once the rescaling was successful

rescaleOperators

CompletableFuture<Acknowledge> rescaleOperators(Collection<JobVertexID> operators,
                                                int newParallelism,
                                                RescalingBehaviour rescalingBehaviour,
                                                Time timeout)

Triggers rescaling of the given set of operators.

Parameters:

operators - set of operators which shall be rescaled

newParallelism - new parallelism of the given set of operators

rescalingBehaviour - defining how strict the rescaling has to be executed

timeout - of this operation

Returns:

Future which is completed with Acknowledge once the rescaling was successful

updateTaskExecutionState

CompletableFuture<Acknowledge> updateTaskExecutionState(TaskExecutionState taskExecutionState)

Updates the task execution state for a given task.

Parameters:

taskExecutionState - New task execution state for a given task

Returns:

Future flag of the task execution state update result

requestNextInputSplit

CompletableFuture<SerializedInputSplit> requestNextInputSplit(JobVertexID vertexID,
                                                              ExecutionAttemptID executionAttempt)

Requests the next input split for the ExecutionJobVertex. The next input split is sent back to the sender as a SerializedInputSplit message.

Parameters:

vertexID - The job vertex id

executionAttempt - The execution attempt id

Returns:

The future of the input split. If there is no further input split, will return an empty object.

requestPartitionState

CompletableFuture<ExecutionState> requestPartitionState(IntermediateDataSetID intermediateResultId,
                                                        ResultPartitionID partitionId)

Requests the current state of the partition. The state of a partition is currently bound to the state of the producing execution.

Parameters:

intermediateResultId - The execution attempt ID of the task requesting the partition state.

partitionId - The partition ID of the partition to request the state of.

Returns:

The future of the partition state

scheduleOrUpdateConsumers

CompletableFuture<Acknowledge> scheduleOrUpdateConsumers(ResultPartitionID partitionID,
                                                         Time timeout)

Notifies the JobManager about available data for a produced partition.

There is a call to this method for each ExecutionVertex instance once per produced ResultPartition instance, either when first producing data (for pipelined executions) or when all data has been produced (for staged executions).

The JobManager then can decide when to schedule the partition consumers of the given session.

Parameters:

partitionID - The partition which has already produced data

timeout - before the rpc call fails

Returns:

Future acknowledge of the schedule or update operation

disconnectTaskManager

CompletableFuture<Acknowledge> disconnectTaskManager(ResourceID resourceID,
                                                     Exception cause)

Disconnects the given TaskExecutor from the JobMaster.

Parameters:

resourceID - identifying the TaskManager to disconnect

cause - for the disconnection of the TaskManager

Returns:

Future acknowledge once the JobMaster has been disconnected from the TaskManager

disconnectResourceManager

void disconnectResourceManager(ResourceManagerId resourceManagerId,
                               Exception cause)

Disconnects the resource manager from the job manager because of the given cause.

Parameters:

resourceManagerId - identifying the resource manager leader id

cause - of the disconnect

requestClassloadingProps

CompletableFuture<ClassloadingProps> requestClassloadingProps()

Request the classloading props of this job.

offerSlots

CompletableFuture<Collection<SlotOffer>> offerSlots(ResourceID taskManagerId,
                                                    Collection<SlotOffer> slots,
                                                    Time timeout)

Offers the given slots to the job manager. The response contains the set of accepted slots.

Parameters:

taskManagerId - identifying the task manager

slots - to offer to the job manager

timeout - for the rpc call

Returns:

Future set of accepted slots.

failSlot

void failSlot(ResourceID taskManagerId,
              AllocationID allocationId,
              Exception cause)

Fails the slot with the given allocation id and cause.

Parameters:

taskManagerId - identifying the task manager

allocationId - identifying the slot to fail

cause - of the failing

registerTaskManager

CompletableFuture<RegistrationResponse> registerTaskManager(String taskManagerRpcAddress,
                                                            TaskManagerLocation taskManagerLocation,
                                                            Time timeout)

Registers the task manager at the job manager.

Parameters:

taskManagerRpcAddress - the rpc address of the task manager

taskManagerLocation - location of the task manager

timeout - for the rpc call

Returns:

Future registration response indicating whether the registration was successful or not

heartbeatFromTaskManager

void heartbeatFromTaskManager(ResourceID resourceID,
                              AccumulatorReport accumulatorReport)

Sends the heartbeat to job manager from task manager.

Parameters:

resourceID - unique id of the task manager

accumulatorReport - report containing accumulator updates

heartbeatFromResourceManager

void heartbeatFromResourceManager(ResourceID resourceID)

Sends heartbeat request from the resource manager.

Parameters:

resourceID - unique id of the resource manager

requestJobDetails

CompletableFuture<JobDetails> requestJobDetails(Time timeout)

Request the details of the executed job.

Parameters:

timeout - for the rpc call

Returns:

Future details of the executed job

requestJobStatus

CompletableFuture<JobStatus> requestJobStatus(Time timeout)

Requests the current job status.

Parameters:

timeout - for the rpc call

Returns:

Future containing the current job status

requestJob

CompletableFuture<ArchivedExecutionGraph> requestJob(Time timeout)

Requests the ArchivedExecutionGraph of the executed job.

Parameters:

timeout - for the rpc call

Returns:

Future which is completed with the ArchivedExecutionGraph of the executed job

triggerSavepoint

CompletableFuture<String> triggerSavepoint(@Nullable
                                           String targetDirectory,
                                           boolean cancelJob,
                                           Time timeout)

Triggers taking a savepoint of the executed job.

Parameters:

targetDirectory - to which to write the savepoint data or null if the default savepoint directory should be used

timeout - for the rpc call

Returns:

Future which is completed with the savepoint path once completed

requestOperatorBackPressureStats

CompletableFuture<OperatorBackPressureStatsResponse> requestOperatorBackPressureStats(JobVertexID jobVertexId)

Requests the statistics on operator back pressure.

Parameters:

jobVertexId - JobVertex for which the stats are requested.

Returns:

A Future to the OperatorBackPressureStatsResponse or null if the stats are not available (yet).

notifyAllocationFailure

void notifyAllocationFailure(AllocationID allocationID,
                             Exception cause)

Notifies that the allocation has failed.

Parameters:

allocationID - the failed allocation id.

cause - the reason that the allocation failed