org.apache.flink.runtime.jobmaster

Interface JobMasterGateway

All Superinterfaces:

CheckpointCoordinatorGateway, FencedRpcGateway<JobMasterId>, JobMasterOperatorEventGateway, KvStateLocationOracle, KvStateRegistryGateway, RpcGateway

All Known Implementing Classes:

JobMaster

public interface JobMasterGateway
extends CheckpointCoordinatorGateway, FencedRpcGateway<JobMasterId>, KvStateLocationOracle, KvStateRegistryGateway, JobMasterOperatorEventGateway

JobMaster rpc gateway interface.

Method Summary

Modifier and Type	Method and Description
CompletableFuture<Acknowledge>	cancel(Time timeout) Cancels the currently executed job.
CompletableFuture<CoordinationResponse>	deliverCoordinationRequestToCoordinator(OperatorID operatorId, SerializedValue<CoordinationRequest> serializedRequest, Time timeout) Deliver a coordination request to a specified coordinator and return the response.
void	disconnectResourceManager(ResourceManagerId resourceManagerId, Exception cause) Disconnects the resource manager from the job manager because of the given cause.
CompletableFuture<Acknowledge>	disconnectTaskManager(ResourceID resourceID, Exception cause) Disconnects the given TaskExecutor from the JobMaster.
void	failSlot(ResourceID taskManagerId, AllocationID allocationId, Exception cause) Fails the slot with the given allocation id and cause.
void	heartbeatFromResourceManager(ResourceID resourceID) Sends heartbeat request from the resource manager.
void	heartbeatFromTaskManager(ResourceID resourceID, AccumulatorReport accumulatorReport) Sends the heartbeat to job manager from task manager.
void	notifyAllocationFailure(AllocationID allocationID, Exception cause) Notifies that the allocation has failed.
CompletableFuture<Collection<SlotOffer>>	offerSlots(ResourceID taskManagerId, Collection<SlotOffer> slots, Time timeout) Offers the given slots to the job manager.
CompletableFuture<RegistrationResponse>	registerTaskManager(String taskManagerRpcAddress, UnresolvedTaskManagerLocation unresolvedTaskManagerLocation, JobID jobId, Time timeout) Registers the task manager at the job manager.
CompletableFuture<ArchivedExecutionGraph>	requestJob(Time timeout) Requests the ArchivedExecutionGraph of the executed job.
CompletableFuture<JobDetails>	requestJobDetails(Time timeout) Request the details of the executed job.
CompletableFuture<JobStatus>	requestJobStatus(Time timeout) Requests the current job status.
CompletableFuture<SerializedInputSplit>	requestNextInputSplit(JobVertexID vertexID, ExecutionAttemptID executionAttempt) Requests the next input split for the ExecutionJobVertex.
CompletableFuture<OperatorBackPressureStatsResponse>	requestOperatorBackPressureStats(JobVertexID jobVertexId) Requests the statistics on operator back pressure.
CompletableFuture<ExecutionState>	requestPartitionState(IntermediateDataSetID intermediateResultId, ResultPartitionID partitionId) Requests the current state of the partition.
CompletableFuture<Acknowledge>	scheduleOrUpdateConsumers(ResultPartitionID partitionID, Time timeout) Notifies the JobManager about available data for a produced partition.
CompletableFuture<String>	stopWithSavepoint(String targetDirectory, boolean terminate, Time timeout) Stops the job with a savepoint.
CompletableFuture<String>	triggerSavepoint(String targetDirectory, boolean cancelJob, Time timeout) Triggers taking a savepoint of the executed job.
CompletableFuture<Object>	updateGlobalAggregate(String aggregateName, Object aggregand, byte[] serializedAggregationFunction) Update the aggregate and return the new value.
CompletableFuture<Acknowledge>	updateTaskExecutionState(TaskExecutionState taskExecutionState) Updates the task execution state for a given task.

Methods inherited from interface org.apache.flink.runtime.checkpoint.CheckpointCoordinatorGateway

acknowledgeCheckpoint, declineCheckpoint

Methods inherited from interface org.apache.flink.runtime.rpc.FencedRpcGateway

getFencingToken

Methods inherited from interface org.apache.flink.runtime.rpc.RpcGateway

getAddress, getHostname

Methods inherited from interface org.apache.flink.runtime.jobmaster.KvStateLocationOracle

requestKvStateLocation

Methods inherited from interface org.apache.flink.runtime.jobmaster.KvStateRegistryGateway

notifyKvStateRegistered, notifyKvStateUnregistered

Methods inherited from interface org.apache.flink.runtime.jobmaster.JobMasterOperatorEventGateway

sendOperatorEventToCoordinator

Method Detail

cancel

CompletableFuture<Acknowledge> cancel(Time timeout)

Cancels the currently executed job.

Parameters:

timeout - of this operation

Returns:

Future acknowledge of the operation

updateTaskExecutionState

CompletableFuture<Acknowledge> updateTaskExecutionState(TaskExecutionState taskExecutionState)

Updates the task execution state for a given task.

Parameters:

taskExecutionState - New task execution state for a given task

Returns:

Future flag of the task execution state update result

requestNextInputSplit

CompletableFuture<SerializedInputSplit> requestNextInputSplit(JobVertexID vertexID,
                                                              ExecutionAttemptID executionAttempt)

Requests the next input split for the ExecutionJobVertex. The next input split is sent back to the sender as a SerializedInputSplit message.

Parameters:

vertexID - The job vertex id

executionAttempt - The execution attempt id

Returns:

The future of the input split. If there is no further input split, will return an empty object.

requestPartitionState

CompletableFuture<ExecutionState> requestPartitionState(IntermediateDataSetID intermediateResultId,
                                                        ResultPartitionID partitionId)

Requests the current state of the partition. The state of a partition is currently bound to the state of the producing execution.

Parameters:

intermediateResultId - The execution attempt ID of the task requesting the partition state.

partitionId - The partition ID of the partition to request the state of.

Returns:

The future of the partition state

scheduleOrUpdateConsumers

CompletableFuture<Acknowledge> scheduleOrUpdateConsumers(ResultPartitionID partitionID,
                                                         Time timeout)

Notifies the JobManager about available data for a produced partition.

There is a call to this method for each ExecutionVertex instance once per produced ResultPartition instance, either when first producing data (for pipelined executions) or when all data has been produced (for staged executions).

The JobManager then can decide when to schedule the partition consumers of the given session.

Parameters:

partitionID - The partition which has already produced data

timeout - before the rpc call fails

Returns:

Future acknowledge of the schedule or update operation

disconnectTaskManager

CompletableFuture<Acknowledge> disconnectTaskManager(ResourceID resourceID,
                                                     Exception cause)

Disconnects the given TaskExecutor from the JobMaster.

Parameters:

resourceID - identifying the TaskManager to disconnect

cause - for the disconnection of the TaskManager

Returns:

Future acknowledge once the JobMaster has been disconnected from the TaskManager

disconnectResourceManager

void disconnectResourceManager(ResourceManagerId resourceManagerId,
                               Exception cause)

Disconnects the resource manager from the job manager because of the given cause.

Parameters:

resourceManagerId - identifying the resource manager leader id

cause - of the disconnect

offerSlots

CompletableFuture<Collection<SlotOffer>> offerSlots(ResourceID taskManagerId,
                                                    Collection<SlotOffer> slots,
                                                    Time timeout)

Offers the given slots to the job manager. The response contains the set of accepted slots.

Parameters:

taskManagerId - identifying the task manager

slots - to offer to the job manager

timeout - for the rpc call

Returns:

Future set of accepted slots.

failSlot

void failSlot(ResourceID taskManagerId,
              AllocationID allocationId,
              Exception cause)

Fails the slot with the given allocation id and cause.

Parameters:

taskManagerId - identifying the task manager

allocationId - identifying the slot to fail

cause - of the failing

registerTaskManager

CompletableFuture<RegistrationResponse> registerTaskManager(String taskManagerRpcAddress,
                                                            UnresolvedTaskManagerLocation unresolvedTaskManagerLocation,
                                                            JobID jobId,
                                                            Time timeout)

Registers the task manager at the job manager.

Parameters:

taskManagerRpcAddress - the rpc address of the task manager

unresolvedTaskManagerLocation - unresolved location of the task manager

jobId - jobId specifying the job for which the JobMaster should be responsible

timeout - for the rpc call

Returns:

Future registration response indicating whether the registration was successful or not

heartbeatFromTaskManager

void heartbeatFromTaskManager(ResourceID resourceID,
                              AccumulatorReport accumulatorReport)

Sends the heartbeat to job manager from task manager.

Parameters:

resourceID - unique id of the task manager

accumulatorReport - report containing accumulator updates

heartbeatFromResourceManager

void heartbeatFromResourceManager(ResourceID resourceID)

Sends heartbeat request from the resource manager.

Parameters:

resourceID - unique id of the resource manager

requestJobDetails

CompletableFuture<JobDetails> requestJobDetails(Time timeout)

Request the details of the executed job.

Parameters:

timeout - for the rpc call

Returns:

Future details of the executed job

requestJobStatus

CompletableFuture<JobStatus> requestJobStatus(Time timeout)

Requests the current job status.

Parameters:

timeout - for the rpc call

Returns:

Future containing the current job status

requestJob

CompletableFuture<ArchivedExecutionGraph> requestJob(Time timeout)

Requests the ArchivedExecutionGraph of the executed job.

Parameters:

timeout - for the rpc call

Returns:

Future which is completed with the ArchivedExecutionGraph of the executed job

triggerSavepoint

CompletableFuture<String> triggerSavepoint(@Nullable
                                           String targetDirectory,
                                           boolean cancelJob,
                                           Time timeout)

Triggers taking a savepoint of the executed job.

Parameters:

targetDirectory - to which to write the savepoint data or null if the default savepoint directory should be used

timeout - for the rpc call

Returns:

Future which is completed with the savepoint path once completed

stopWithSavepoint

CompletableFuture<String> stopWithSavepoint(@Nullable
                                            String targetDirectory,
                                            boolean terminate,
                                            Time timeout)

Stops the job with a savepoint.

Parameters:

targetDirectory - to which to write the savepoint data or null if the default savepoint directory should be used

terminate - flag indicating if the job should terminate or just suspend

timeout - for the rpc call

Returns:

Future which is completed with the savepoint path once completed

requestOperatorBackPressureStats

CompletableFuture<OperatorBackPressureStatsResponse> requestOperatorBackPressureStats(JobVertexID jobVertexId)

Requests the statistics on operator back pressure.

Parameters:

jobVertexId - JobVertex for which the stats are requested.

Returns:

A Future to the OperatorBackPressureStatsResponse.

notifyAllocationFailure

void notifyAllocationFailure(AllocationID allocationID,
                             Exception cause)

Notifies that the allocation has failed.

Parameters:

allocationID - the failed allocation id.

cause - the reason that the allocation failed

updateGlobalAggregate

CompletableFuture<Object> updateGlobalAggregate(String aggregateName,
                                                Object aggregand,
                                                byte[] serializedAggregationFunction)

Update the aggregate and return the new value.

Parameters:

aggregateName - The name of the aggregate to update

aggregand - The value to add to the aggregate

serializedAggregationFunction - The function to apply to the current aggregate and aggregand to obtain the new aggregate value, this should be of type AggregateFunction

Returns:

The updated aggregate

deliverCoordinationRequestToCoordinator

CompletableFuture<CoordinationResponse> deliverCoordinationRequestToCoordinator(OperatorID operatorId,
                                                                                SerializedValue<CoordinationRequest> serializedRequest,
                                                                                Time timeout)

Deliver a coordination request to a specified coordinator and return the response.

Parameters:

operatorId - identifying the coordinator to receive the request

serializedRequest - serialized request to deliver

Returns:

A future containing the response. The response will fail with a FlinkException if the task is not running, or no operator/coordinator exists for the given ID, or the coordinator cannot handle client events.