org.apache.flink.runtime.resourcemanager

Interface ResourceManagerGateway

All Superinterfaces:

BlocklistListener, ClusterPartitionManager, FencedRpcGateway<ResourceManagerId>, RpcGateway

All Known Implementing Classes:

ActiveResourceManager, ResourceManager, StandaloneResourceManager

public interface ResourceManagerGateway
extends FencedRpcGateway<ResourceManagerId>, ClusterPartitionManager, BlocklistListener

The ResourceManager's RPC gateway interface.

Method Summary

Modifier and Type	Method and Description
CompletableFuture<Acknowledge>	declareRequiredResources(JobMasterId jobMasterId, ResourceRequirements resourceRequirements, Time timeout) Declares the absolute resource requirements for a job.
CompletableFuture<Acknowledge>	deregisterApplication(ApplicationStatus finalStatus, String diagnostics) Deregister Flink from the underlying resource management system.
void	disconnectJobManager(JobID jobId, JobStatus jobStatus, Exception cause) Disconnects a JobManager specified by the given resourceID from the ResourceManager.
void	disconnectTaskManager(ResourceID resourceID, Exception cause) Disconnects a TaskManager specified by the given resourceID from the ResourceManager.
CompletableFuture<Integer>	getNumberOfRegisteredTaskManagers() Gets the currently registered number of TaskManagers.
CompletableFuture<Void>	heartbeatFromJobManager(ResourceID heartbeatOrigin) Sends the heartbeat to resource manager from job manager.
CompletableFuture<Void>	heartbeatFromTaskManager(ResourceID heartbeatOrigin, TaskExecutorHeartbeatPayload heartbeatPayload) Sends the heartbeat to resource manager from task manager.
void	notifySlotAvailable(InstanceID instanceId, SlotID slotID, AllocationID oldAllocationId) Sent by the TaskExecutor to notify the ResourceManager that a slot has become available.
CompletableFuture<RegistrationResponse>	registerJobMaster(JobMasterId jobMasterId, ResourceID jobMasterResourceId, String jobMasterAddress, JobID jobId, Time timeout) Register a JobMaster at the resource manager.
CompletableFuture<RegistrationResponse>	registerTaskExecutor(TaskExecutorRegistration taskExecutorRegistration, Time timeout) Register a TaskExecutor at the resource manager.
CompletableFuture<ProfilingInfo>	requestProfiling(ResourceID taskManagerId, int duration, ProfilingInfo.ProfilingMode mode, Duration timeout) Requests the profiling instance from the given TaskExecutor.
CompletableFuture<ResourceOverview>	requestResourceOverview(Time timeout) Requests the resource overview.
CompletableFuture<TaskExecutorThreadInfoGateway>	requestTaskExecutorThreadInfoGateway(ResourceID taskManagerId, Time timeout) Requests the TaskExecutorGateway.
CompletableFuture<TaskManagerInfoWithSlots>	requestTaskManagerDetailsInfo(ResourceID taskManagerId, Time timeout) Requests detail information about the given TaskExecutor.
CompletableFuture<TransientBlobKey>	requestTaskManagerFileUploadByName(ResourceID taskManagerId, String fileName, Time timeout) Deprecated. use requestTaskManagerFileUploadByNameAndType(ResourceID, String, FileType, Duration) as instead.
CompletableFuture<TransientBlobKey>	requestTaskManagerFileUploadByNameAndType(ResourceID taskManagerId, String fileName, FileType fileType, Duration timeout) Request the file upload from the given TaskExecutor to the cluster's BlobServer.
CompletableFuture<TransientBlobKey>	requestTaskManagerFileUploadByType(ResourceID taskManagerId, FileType fileType, Time timeout) Request the file upload from the given TaskExecutor to the cluster's BlobServer.
CompletableFuture<Collection<TaskManagerInfo>>	requestTaskManagerInfo(Time timeout) Requests information about the registered TaskExecutor.
CompletableFuture<Collection<LogInfo>>	requestTaskManagerLogList(ResourceID taskManagerId, Time timeout) Request log list from the given TaskExecutor.
CompletableFuture<Collection<Tuple2<ResourceID,String>>>	requestTaskManagerMetricQueryServiceAddresses(Time timeout) Requests the paths for the TaskManager's MetricQueryService to query.
CompletableFuture<Collection<ProfilingInfo>>	requestTaskManagerProfilingList(ResourceID taskManagerId, Duration timeout) Request profiling list from the given TaskExecutor.
CompletableFuture<ThreadDumpInfo>	requestThreadDump(ResourceID taskManagerId, Time timeout) Requests the thread dump from the given TaskExecutor.
CompletableFuture<Acknowledge>	sendSlotReport(ResourceID taskManagerResourceId, InstanceID taskManagerRegistrationId, SlotReport slotReport, Time timeout) Sends the given SlotReport to the ResourceManager.

Methods inherited from interface org.apache.flink.runtime.rpc.FencedRpcGateway

getFencingToken

Methods inherited from interface org.apache.flink.runtime.rpc.RpcGateway

getAddress, getHostname

Methods inherited from interface org.apache.flink.runtime.io.network.partition.ClusterPartitionManager

getClusterPartitionsShuffleDescriptors, listDataSets, releaseClusterPartitions, reportClusterPartitions

Methods inherited from interface org.apache.flink.runtime.blocklist.BlocklistListener

notifyNewBlockedNodes

Method Detail

registerJobMaster

CompletableFuture<RegistrationResponse> registerJobMaster(JobMasterId jobMasterId,
                                                          ResourceID jobMasterResourceId,
                                                          String jobMasterAddress,
                                                          JobID jobId,
                                                          Time timeout)

Register a JobMaster at the resource manager.

Parameters:

jobMasterId - The fencing token for the JobMaster leader

jobMasterResourceId - The resource ID of the JobMaster that registers

jobMasterAddress - The address of the JobMaster that registers

jobId - The Job ID of the JobMaster that registers

timeout - Timeout for the future to complete

Returns:

Future registration response

declareRequiredResources

CompletableFuture<Acknowledge> declareRequiredResources(JobMasterId jobMasterId,
                                                        ResourceRequirements resourceRequirements,
                                                        Time timeout)

Declares the absolute resource requirements for a job.

Parameters:

jobMasterId - id of the JobMaster

resourceRequirements - resource requirements

Returns:

The confirmation that the requirements have been processed

registerTaskExecutor

CompletableFuture<RegistrationResponse> registerTaskExecutor(TaskExecutorRegistration taskExecutorRegistration,
                                                             Time timeout)

Register a TaskExecutor at the resource manager.

Parameters:

taskExecutorRegistration - the task executor registration.

timeout - The timeout for the response.

Returns:

The future to the response by the ResourceManager.

sendSlotReport

CompletableFuture<Acknowledge> sendSlotReport(ResourceID taskManagerResourceId,
                                              InstanceID taskManagerRegistrationId,
                                              SlotReport slotReport,
                                              Time timeout)

Sends the given SlotReport to the ResourceManager.

Parameters:

taskManagerRegistrationId - id identifying the sending TaskManager

slotReport - which is sent to the ResourceManager

timeout - for the operation

Returns:

Future which is completed with Acknowledge once the slot report has been received.

notifySlotAvailable

void notifySlotAvailable(InstanceID instanceId,
                         SlotID slotID,
                         AllocationID oldAllocationId)

Sent by the TaskExecutor to notify the ResourceManager that a slot has become available.

Parameters:

instanceId - TaskExecutor's instance id

slotID - The SlotID of the freed slot

oldAllocationId - to which the slot has been allocated

deregisterApplication

CompletableFuture<Acknowledge> deregisterApplication(ApplicationStatus finalStatus,
                                                     @Nullable
                                                     String diagnostics)

Deregister Flink from the underlying resource management system.

Parameters:

finalStatus - final status with which to deregister the Flink application

diagnostics - additional information for the resource management system, can be null

getNumberOfRegisteredTaskManagers

CompletableFuture<Integer> getNumberOfRegisteredTaskManagers()

Gets the currently registered number of TaskManagers.

Returns:

The future to the number of registered TaskManagers.

heartbeatFromTaskManager

CompletableFuture<Void> heartbeatFromTaskManager(ResourceID heartbeatOrigin,
                                                 TaskExecutorHeartbeatPayload heartbeatPayload)

Sends the heartbeat to resource manager from task manager.

Parameters:

heartbeatOrigin - unique id of the task manager

heartbeatPayload - payload from the originating TaskManager

Returns:

future which is completed exceptionally if the operation fails

heartbeatFromJobManager

CompletableFuture<Void> heartbeatFromJobManager(ResourceID heartbeatOrigin)

Sends the heartbeat to resource manager from job manager.

Parameters:

heartbeatOrigin - unique id of the job manager

Returns:

future which is completed exceptionally if the operation fails

disconnectTaskManager

void disconnectTaskManager(ResourceID resourceID,
                           Exception cause)

Disconnects a TaskManager specified by the given resourceID from the ResourceManager.

Parameters:

resourceID - identifying the TaskManager to disconnect

cause - for the disconnection of the TaskManager

disconnectJobManager

void disconnectJobManager(JobID jobId,
                          JobStatus jobStatus,
                          Exception cause)

Disconnects a JobManager specified by the given resourceID from the ResourceManager.

Parameters:

jobId - JobID for which the JobManager was the leader

jobStatus - status of the job at the time of disconnection

cause - for the disconnection of the JobManager

requestTaskManagerInfo

CompletableFuture<Collection<TaskManagerInfo>> requestTaskManagerInfo(Time timeout)

Requests information about the registered TaskExecutor.

Parameters:

timeout - of the request

Returns:

Future collection of TaskManager information

requestTaskManagerDetailsInfo

CompletableFuture<TaskManagerInfoWithSlots> requestTaskManagerDetailsInfo(ResourceID taskManagerId,
                                                                          Time timeout)

Requests detail information about the given TaskExecutor.

Parameters:

taskManagerId - identifying the TaskExecutor for which to return information

timeout - of the request

Returns:

Future TaskManager information and its allocated slots

requestResourceOverview

CompletableFuture<ResourceOverview> requestResourceOverview(Time timeout)

Requests the resource overview. The resource overview provides information about the connected TaskManagers, the total number of slots and the number of available slots.

Parameters:

timeout - of the request

Returns:

Future containing the resource overview

requestTaskManagerMetricQueryServiceAddresses

CompletableFuture<Collection<Tuple2<ResourceID,String>>> requestTaskManagerMetricQueryServiceAddresses(Time timeout)

Requests the paths for the TaskManager's MetricQueryService to query.

Parameters:

timeout - for the asynchronous operation

Returns:

Future containing the collection of resource ids and the corresponding metric query service path

requestTaskManagerFileUploadByType

CompletableFuture<TransientBlobKey> requestTaskManagerFileUploadByType(ResourceID taskManagerId,
                                                                       FileType fileType,
                                                                       Time timeout)

Request the file upload from the given TaskExecutor to the cluster's BlobServer. The corresponding TransientBlobKey is returned.

Parameters:

taskManagerId - identifying the TaskExecutor to upload the specified file

fileType - type of the file to upload

timeout - for the asynchronous operation

Returns:

Future which is completed with the TransientBlobKey after uploading the file to the BlobServer.

requestTaskManagerFileUploadByName

@Deprecated
CompletableFuture<TransientBlobKey> requestTaskManagerFileUploadByName(ResourceID taskManagerId,
                                                                                   String fileName,
                                                                                   Time timeout)

Deprecated. use requestTaskManagerFileUploadByNameAndType(ResourceID, String, FileType, Duration) as instead.

Request the file upload from the given TaskExecutor to the cluster's BlobServer. The corresponding TransientBlobKey is returned. To support different type file upload with name, using ResourceManager.requestTaskManagerFileUploadByNameAndType(org.apache.flink.runtime.clusterframework.types.ResourceID, java.lang.String, org.apache.flink.runtime.taskexecutor.FileType, java.time.Duration) as instead.

Parameters:

taskManagerId - identifying the TaskExecutor to upload the specified file

fileName - name of the file to upload

timeout - for the asynchronous operation

Returns:

Future which is completed with the TransientBlobKey after uploading the file to the BlobServer.

requestTaskManagerFileUploadByNameAndType

CompletableFuture<TransientBlobKey> requestTaskManagerFileUploadByNameAndType(ResourceID taskManagerId,
                                                                              String fileName,
                                                                              FileType fileType,
                                                                              Duration timeout)

Request the file upload from the given TaskExecutor to the cluster's BlobServer. The corresponding TransientBlobKey is returned.

Parameters:

taskManagerId - identifying the TaskExecutor to upload the specified file

fileName - name of the file to upload

fileType - type of the file to upload

timeout - for the asynchronous operation

Returns:

Future which is completed with the TransientBlobKey after uploading the file to the BlobServer.

requestTaskManagerLogList

CompletableFuture<Collection<LogInfo>> requestTaskManagerLogList(ResourceID taskManagerId,
                                                                 Time timeout)

Request log list from the given TaskExecutor.

Parameters:

taskManagerId - identifying the TaskExecutor to get log list from

timeout - for the asynchronous operation

Returns:

Future which is completed with the historical log list

requestThreadDump

CompletableFuture<ThreadDumpInfo> requestThreadDump(ResourceID taskManagerId,
                                                    Time timeout)

Requests the thread dump from the given TaskExecutor.

Parameters:

taskManagerId - taskManagerId identifying the TaskExecutor to get the thread dump from

timeout - timeout of the asynchronous operation

Returns:

Future containing the thread dump information

requestTaskExecutorThreadInfoGateway

CompletableFuture<TaskExecutorThreadInfoGateway> requestTaskExecutorThreadInfoGateway(ResourceID taskManagerId,
                                                                                      Time timeout)

Requests the TaskExecutorGateway.

Parameters:

taskManagerId - identifying the TaskExecutor.

Returns:

Future containing the task executor gateway.

requestTaskManagerProfilingList

CompletableFuture<Collection<ProfilingInfo>> requestTaskManagerProfilingList(ResourceID taskManagerId,
                                                                             Duration timeout)

Request profiling list from the given TaskExecutor.

Parameters:

taskManagerId - identifying the TaskExecutor to get profiling list from

timeout - for the asynchronous operation

Returns:

Future which is completed with the historical profiling list

requestProfiling

CompletableFuture<ProfilingInfo> requestProfiling(ResourceID taskManagerId,
                                                  int duration,
                                                  ProfilingInfo.ProfilingMode mode,
                                                  Duration timeout)

Requests the profiling instance from the given TaskExecutor.

Parameters:

taskManagerId - taskManagerId identifying the TaskExecutor to get the profiling from

duration - profiling duration

mode - profiling mode ProfilingInfo.ProfilingMode

timeout - timeout of the asynchronous operation

Returns:

Future containing the created profiling information