public class TaskManager$ extends Object
Modifier and Type | Field and Description |
---|---|
static TaskManager$ |
MODULE$
Static reference to the singleton instance of this Scala object.
|
Constructor and Description |
---|
TaskManager$() |
Modifier and Type | Method and Description |
---|---|
scala.Tuple7<TaskManagerConfiguration,TaskManagerLocation,MemoryManager,IOManager,NetworkEnvironment,LeaderRetrievalService,MetricRegistry> |
createTaskManagerComponents(Configuration configuration,
ResourceID resourceID,
String taskManagerHostname,
boolean localTaskManagerCommunication,
scala.Option<LeaderRetrievalService> leaderRetrievalServiceOption) |
scala.Tuple3<String,String,Object> |
getAndCheckJobManagerAddress(Configuration configuration)
Gets the protocol, hostname and port of the JobManager from the configuration.
|
akka.actor.Props |
getTaskManagerProps(Class<? extends TaskManager> taskManagerClass,
TaskManagerConfiguration taskManagerConfig,
ResourceID resourceID,
TaskManagerLocation taskManagerLocation,
MemoryManager memoryManager,
IOManager ioManager,
NetworkEnvironment networkEnvironment,
LeaderRetrievalService leaderRetrievalService,
MetricRegistry metricsRegistry) |
akka.actor.ActorRef |
getTaskManagerRemoteReference(String taskManagerUrl,
akka.actor.ActorSystem system,
scala.concurrent.duration.FiniteDuration timeout)
Resolves the TaskManager actor reference in a blocking fashion.
|
scala.concurrent.duration.FiniteDuration |
HEARTBEAT_INTERVAL() |
grizzled.slf4j.Logger |
LOG()
TaskManager logger for synchronous logging (not through the logging actor)
|
void |
main(String[] args)
Entry point (main method) to run the TaskManager in a standalone fashion.
|
long |
MAX_STARTUP_CONNECT_TIME()
Maximum time (milli seconds) that the TaskManager will spend searching for a
suitable network interface to use for communication
|
Configuration |
parseArgsAndLoadConfig(String[] args)
Parse the command line arguments of the TaskManager and loads the configuration.
|
scala.Tuple4<TaskManagerConfiguration,NetworkEnvironmentConfiguration,InetSocketAddress,MemoryType> |
parseTaskManagerConfiguration(Configuration configuration,
String taskManagerHostname,
boolean localTaskManagerCommunication)
Utility method to extract TaskManager config parameters from the configuration and to
sanity check them.
|
void |
runTaskManager(String taskManagerHostname,
ResourceID resourceID,
int actorSystemPort,
Configuration configuration)
Starts and runs the TaskManager.
|
void |
runTaskManager(String taskManagerHostname,
ResourceID resourceID,
int actorSystemPort,
Configuration configuration,
Class<? extends TaskManager> taskManagerClass)
Starts and runs the TaskManager.
|
int |
RUNTIME_FAILURE_RETURN_CODE()
Return code for critical errors during the runtime
|
scala.Tuple2<String,Object> |
selectNetworkInterfaceAndPort(Configuration configuration) |
void |
selectNetworkInterfaceAndRunTaskManager(Configuration configuration,
ResourceID resourceID,
Class<? extends TaskManager> taskManagerClass)
Starts and runs the TaskManager.
|
akka.actor.ActorRef |
startTaskManagerComponentsAndActor(Configuration configuration,
ResourceID resourceID,
akka.actor.ActorSystem actorSystem,
String taskManagerHostname,
scala.Option<String> taskManagerActorName,
scala.Option<LeaderRetrievalService> leaderRetrievalServiceOption,
boolean localTaskManagerCommunication,
Class<? extends TaskManager> taskManagerClass)
Starts the task manager actor.
|
long |
STARTUP_CONNECT_LOG_SUPPRESS()
Time (milli seconds) after which the TaskManager will start logging failed
connection attempts
|
int |
STARTUP_FAILURE_RETURN_CODE()
Return code for unsuccessful TaskManager startup
|
String |
TASK_MANAGER_NAME()
The name of the TaskManager actor
|
public static final TaskManager$ MODULE$
public grizzled.slf4j.Logger LOG()
public int STARTUP_FAILURE_RETURN_CODE()
public int RUNTIME_FAILURE_RETURN_CODE()
public String TASK_MANAGER_NAME()
public long MAX_STARTUP_CONNECT_TIME()
public long STARTUP_CONNECT_LOG_SUPPRESS()
public scala.concurrent.duration.FiniteDuration HEARTBEAT_INTERVAL()
public void main(String[] args)
args
- The command line arguments.public Configuration parseArgsAndLoadConfig(String[] args)
args
- Command line argumentspublic void selectNetworkInterfaceAndRunTaskManager(Configuration configuration, ResourceID resourceID, Class<? extends TaskManager> taskManagerClass)
This method first tries to select the network interface to use for the TaskManager communication. The network interface is used both for the actor communication (coordination) as well as for the data exchange between task managers. Unless the hostname/interface is explicitly configured in the configuration, this method will try out various interfaces and methods to connect to the JobManager and select the one where the connection attempt is successful.
After selecting the network interface, this method brings up an actor system for the TaskManager and its actors, starts the TaskManager's services (library cache, shuffle network stack, ...), and starts the TaskManager itself.
configuration
- The configuration for the TaskManager.taskManagerClass
- The actor class to instantiate.
Allows to use TaskManager subclasses for example for YARN.public scala.Tuple2<String,Object> selectNetworkInterfaceAndPort(Configuration configuration)
public void runTaskManager(String taskManagerHostname, ResourceID resourceID, int actorSystemPort, Configuration configuration)
This method will also spawn a process reaper for the TaskManager (kill the process if the actor fails) and optionally start the JVM memory logging thread.
taskManagerHostname
- The hostname/address of the interface where the actor system
will communicate.resourceID
- The id of the resource which the task manager will run on.actorSystemPort
- The port at which the actor system will communicate.configuration
- The configuration for the TaskManager.public void runTaskManager(String taskManagerHostname, ResourceID resourceID, int actorSystemPort, Configuration configuration, Class<? extends TaskManager> taskManagerClass)
This method will also spawn a process reaper for the TaskManager (kill the process if the actor fails) and optionally start the JVM memory logging thread.
taskManagerHostname
- The hostname/address of the interface where the actor system
will communicate.resourceID
- The id of the resource which the task manager will run on.actorSystemPort
- The port at which the actor system will communicate.configuration
- The configuration for the TaskManager.taskManagerClass
- The actor class to instantiate. Allows the use of TaskManager
subclasses for example for YARN.public akka.actor.ActorRef startTaskManagerComponentsAndActor(Configuration configuration, ResourceID resourceID, akka.actor.ActorSystem actorSystem, String taskManagerHostname, scala.Option<String> taskManagerActorName, scala.Option<LeaderRetrievalService> leaderRetrievalServiceOption, boolean localTaskManagerCommunication, Class<? extends TaskManager> taskManagerClass)
configuration
- The configuration for the TaskManager.resourceID
- The id of the resource which the task manager will run on.actorSystem
- The actor system that should run the TaskManager actor.taskManagerHostname
- The hostname/address that describes the TaskManager's data location.taskManagerActorName
- Optionally the name of the TaskManager actor. If none is given,
the actor will use a random name.leaderRetrievalServiceOption
- Optionally, a leader retrieval service can be provided. If
none is given, then a LeaderRetrievalService is
constructed from the configuration.localTaskManagerCommunication
- If true, the TaskManager will not initiate the
TCP network stack.taskManagerClass
- The class of the TaskManager actor. May be used to give
subclasses that understand additional actor messages.IllegalConfigurationException
- Thrown, if the given config contains illegal values.IOException
- Thrown, if any of the I/O components (such as buffer pools,
I/O manager, ...) cannot be properly started.Exception
- Thrown is some other error occurs while parsing the configuration
or starting the TaskManager components.public akka.actor.Props getTaskManagerProps(Class<? extends TaskManager> taskManagerClass, TaskManagerConfiguration taskManagerConfig, ResourceID resourceID, TaskManagerLocation taskManagerLocation, MemoryManager memoryManager, IOManager ioManager, NetworkEnvironment networkEnvironment, LeaderRetrievalService leaderRetrievalService, MetricRegistry metricsRegistry)
public scala.Tuple7<TaskManagerConfiguration,TaskManagerLocation,MemoryManager,IOManager,NetworkEnvironment,LeaderRetrievalService,MetricRegistry> createTaskManagerComponents(Configuration configuration, ResourceID resourceID, String taskManagerHostname, boolean localTaskManagerCommunication, scala.Option<LeaderRetrievalService> leaderRetrievalServiceOption)
public akka.actor.ActorRef getTaskManagerRemoteReference(String taskManagerUrl, akka.actor.ActorSystem system, scala.concurrent.duration.FiniteDuration timeout)
taskManagerUrl
- The akka URL of the JobManager.system
- The local actor system that should perform the lookup.timeout
- The maximum time to wait until the lookup fails.IOException
- Thrown, if the lookup fails.public scala.Tuple4<TaskManagerConfiguration,NetworkEnvironmentConfiguration,InetSocketAddress,MemoryType> parseTaskManagerConfiguration(Configuration configuration, String taskManagerHostname, boolean localTaskManagerCommunication)
configuration
- The configuration.taskManagerHostname
- The host name under which the TaskManager communicates.localTaskManagerCommunication
- True, to skip initializing the network stack.
Use only in cases where only one task manager runs.public scala.Tuple3<String,String,Object> getAndCheckJobManagerAddress(Configuration configuration)
configuration
- The configuration to read the config values from.Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.