public class YarnApplicationMasterRunner extends Object
JobManager
and YarnFlinkResourceManager
.
The JobManager handles Flink job execution, while the YarnFlinkResourceManager handles container
allocation and failure detection.Modifier and Type | Field and Description |
---|---|
protected static org.slf4j.Logger |
LOG
Logger
|
Constructor and Description |
---|
YarnApplicationMasterRunner() |
Modifier and Type | Method and Description |
---|---|
static org.apache.hadoop.yarn.api.records.ContainerLaunchContext |
createTaskManagerContext(Configuration flinkConfig,
org.apache.hadoop.yarn.conf.YarnConfiguration yarnConfig,
Map<String,String> env,
ContaineredTaskManagerParameters tmParams,
Configuration taskManagerConfig,
String workingDirectory,
Class<?> taskManagerMainClass,
org.slf4j.Logger log)
Creates the launch context, which describes how to bring up a TaskManager process in
an allocated YARN container.
|
protected Class<? extends MemoryArchivist> |
getArchivistClass() |
protected Class<? extends JobManager> |
getJobManagerClass() |
protected Class<? extends YarnFlinkResourceManager> |
getResourceManagerClass() |
protected Class<? extends TaskManager> |
getTaskManagerClass() |
static void |
main(String[] args)
The entry point for the YARN application master.
|
protected int |
run(String[] args)
The instance entry point for the YARN application master.
|
protected int |
runApplicationMaster(Configuration config)
The main work method, must run as a privileged action.
|
public static void main(String[] args)
args
- The command line arguments.protected int run(String[] args)
runApplicationMaster(Configuration)
as a
privileged action.args
- The command line arguments.protected int runApplicationMaster(Configuration config)
protected Class<? extends YarnFlinkResourceManager> getResourceManagerClass()
protected Class<? extends JobManager> getJobManagerClass()
protected Class<? extends MemoryArchivist> getArchivistClass()
protected Class<? extends TaskManager> getTaskManagerClass()
public static org.apache.hadoop.yarn.api.records.ContainerLaunchContext createTaskManagerContext(Configuration flinkConfig, org.apache.hadoop.yarn.conf.YarnConfiguration yarnConfig, Map<String,String> env, ContaineredTaskManagerParameters tmParams, Configuration taskManagerConfig, String workingDirectory, Class<?> taskManagerMainClass, org.slf4j.Logger log) throws Exception
This code is extremely YARN specific and registers all the resources that the TaskManager needs (such as JAR file, config file, ...) and all environment variables in a YARN container launch context. The launch context then ensures that those resources will be copied into the containers transient working directory.
We do this work before we start the ResourceManager actor in order to fail early if any of the operations here fail.
flinkConfig
- The Flink configuration object.yarnConfig
- The YARN configuration object.env
- The environment variables.tmParams
- The TaskManager container memory parameters.taskManagerConfig
- The configuration for the TaskManagers.workingDirectory
- The current application master container's working directory.taskManagerMainClass
- The class with the main method.log
- The logger.Exception
- Thrown if teh launch context could not be created, for example if
the resources could not be copied.Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.