Package org.apache.flink.yarn

Class YarnResourceManagerDriver

java.lang.Object

org.apache.flink.runtime.resourcemanager.active.AbstractResourceManagerDriver<YarnWorkerNode>

org.apache.flink.yarn.YarnResourceManagerDriver

All Implemented Interfaces:

ResourceManagerDriver<YarnWorkerNode>

public class YarnResourceManagerDriver
extends AbstractResourceManagerDriver<YarnWorkerNode>

Implementation of ResourceManagerDriver for Yarn deployment.

Field Summary

Fields inherited from class org.apache.flink.runtime.resourcemanager.active.AbstractResourceManagerDriver

flinkClientConfig, flinkConfig, log

Constructor Summary

Constructors

Constructor	Description
YarnResourceManagerDriver(Configuration flinkConfig, YarnResourceManagerDriverConfiguration configuration, YarnResourceManagerClientFactory yarnResourceManagerClientFactory, YarnNodeManagerClientFactory yarnNodeManagerClientFactory)

Method Summary

Modifier and Type	Method	Description
void	deregisterApplication(ApplicationStatus finalStatus, String optionalDiagnostics)	The deployment specific code to deregister the application.
static String	getContainerCompletedCause(org.apache.hadoop.yarn.api.records.ContainerStatus containerStatus)
protected void	initializeInternal()	Initialize the deployment specific components.
void	releaseResource(YarnWorkerNode workerNode)	Release resource to the external resource manager.
CompletableFuture<YarnWorkerNode>	requestResource(TaskExecutorProcessSpec taskExecutorProcessSpec)	Request resource from the external resource manager.
void	terminate()	Terminate the deployment specific components.

Methods inherited from class org.apache.flink.runtime.resourcemanager.active.AbstractResourceManagerDriver

getBlockedNodeRetriever, getIoExecutor, getMainThreadExecutor, getResourceEventHandler, initialize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

YarnResourceManagerDriver

public YarnResourceManagerDriver(Configuration flinkConfig,
                                 YarnResourceManagerDriverConfiguration configuration,
                                 YarnResourceManagerClientFactory yarnResourceManagerClientFactory,
                                 YarnNodeManagerClientFactory yarnNodeManagerClientFactory)

Method Detail

initializeInternal

protected void initializeInternal()
                           throws Exception

Description copied from class: AbstractResourceManagerDriver

Initialize the deployment specific components.

Specified by:

initializeInternal in class AbstractResourceManagerDriver<YarnWorkerNode>

Throws:

Exception

terminate

public void terminate()
               throws Exception

Description copied from interface: ResourceManagerDriver

Terminate the deployment specific components.

Throws:

Exception

deregisterApplication

public void deregisterApplication(ApplicationStatus finalStatus,
                                  @Nullable
                                  String optionalDiagnostics)

Description copied from interface: ResourceManagerDriver

The deployment specific code to deregister the application. This should report the application's final status.

This method also needs to make sure all pending containers that are not registered yet are returned.

Parameters:

finalStatus - The application status to report.

optionalDiagnostics - A diagnostics message or null.

requestResource

public CompletableFuture<YarnWorkerNode> requestResource(TaskExecutorProcessSpec taskExecutorProcessSpec)

Description copied from interface: ResourceManagerDriver

Request resource from the external resource manager.

This method request a new resource from the external resource manager, and tries to launch a task manager inside the allocated resource, with respect to the provided taskExecutorProcessSpec. The returned future will be completed with a worker node in the deployment specific type, or exceptionally if the allocation has failed.

Note: The returned future could be cancelled by ResourceManager. This means ResourceManager don't need this resource anymore, Driver should try to cancel this request from the external resource manager.

Note: Completion of the returned future does not necessarily mean the success of resource allocation and task manager launching. Allocation and launching failures can still happen after the future completion. In such cases, ResourceEventHandler.onWorkerTerminated(org.apache.flink.runtime.clusterframework.types.ResourceID, java.lang.String) will be called.

The future is guaranteed to be completed in the rpc main thread, before trying to launch the task manager, thus before the task manager registration. It is also guaranteed that ResourceEventHandler.onWorkerTerminated(org.apache.flink.runtime.clusterframework.types.ResourceID, java.lang.String) will not be called on the requested worker, until the returned future is completed successfully.

Parameters:

taskExecutorProcessSpec - Resource specification of the requested worker.

Returns:

Future that wraps worker node of the requested resource, in the deployment specific type.

releaseResource

public void releaseResource(YarnWorkerNode workerNode)

Description copied from interface: ResourceManagerDriver

Release resource to the external resource manager.

Parameters:

workerNode - Worker node to be released, in the deployment specific type.

getContainerCompletedCause

public static String getContainerCompletedCause(org.apache.hadoop.yarn.api.records.ContainerStatus containerStatus)