org.apache.flink.mesos.runtime.clusterframework

Class MesosFlinkResourceManager

java.lang.Object

akka.actor.UntypedActor

org.apache.flink.runtime.akka.FlinkUntypedActor

org.apache.flink.runtime.clusterframework.FlinkResourceManager<RegisteredMesosWorkerNode>

org.apache.flink.mesos.runtime.clusterframework.MesosFlinkResourceManager

All Implemented Interfaces:

akka.actor.Actor

public class MesosFlinkResourceManager
extends FlinkResourceManager<RegisteredMesosWorkerNode>

Flink Resource Manager for Apache Mesos.

Nested Class Summary

Nested classes/interfaces inherited from interface akka.actor.Actor

akka.actor.Actor.emptyBehavior$, akka.actor.Actor.ignoringBehavior$

Field Summary

Fields inherited from class org.apache.flink.runtime.clusterframework.FlinkResourceManager

config, EXIT_CODE_FATAL_ERROR, RESOURCE_MANAGER_NAME

Fields inherited from class org.apache.flink.runtime.akka.FlinkUntypedActor

LOG

Constructor Summary

Constructors

Constructor and Description
MesosFlinkResourceManager(Configuration flinkConfig, MesosConfiguration mesosConfig, MesosWorkerStore workerStore, LeaderRetrievalService leaderRetrievalService, MesosTaskManagerParameters taskManagerParameters, ContainerSpecification taskManagerContainerSpec, MesosArtifactResolver artifactResolver, int maxFailedTasks, int numInitialTaskManagers)

Method Summary

Modifier and Type	Method and Description
static akka.actor.Props	createActorProps(Class<? extends MesosFlinkResourceManager> actorClass, Configuration flinkConfig, MesosConfiguration mesosConfig, MesosWorkerStore workerStore, LeaderRetrievalService leaderRetrievalService, MesosTaskManagerParameters taskManagerParameters, ContainerSpecification taskManagerContainerSpec, MesosArtifactResolver artifactResolver, org.slf4j.Logger log) Creates the props needed to instantiate this actor.
protected akka.actor.ActorRef	createConnectionMonitor()
protected akka.actor.ActorRef	createLaunchCoordinator()
protected akka.actor.ActorRef	createReconciliationCoordinator()
protected akka.actor.ActorRef	createTaskRouter()
protected void	fatalError(String message, Throwable error) Notifies the resource master of a fatal error.
protected int	getNumWorkerRequestsPending() Gets the number of requested workers that have not yet been granted.
protected int	getNumWorkersPendingRegistration() Gets the number of containers that have been started, but where the TaskManager has not yet registered at the job manager.
protected void	handleMessage(Object message) This method receives the actor messages after they have been filtered for a match with the leader session.
protected void	initialize() Initializes the framework specific components.
void	postStop()
protected Collection<RegisteredMesosWorkerNode>	reacceptRegisteredWorkers(Collection<ResourceID> toConsolidate) Accept the given registered workers into the internal state.
protected void	releasePendingWorker(ResourceID id) Release the given pending worker.
protected void	releaseStartedWorker(RegisteredMesosWorkerNode worker) Release the given started worker.
protected void	requestNewWorkers(int numWorkers) Plan for some additional workers to be launched.
protected void	shutdownApplication(ApplicationStatus finalStatus, String optionalDiagnostics) Called to shut down the cluster (not a failover situation).
protected RegisteredMesosWorkerNode	workerStarted(ResourceID resourceID) Accept the given started worker into the internal state.

Methods inherited from class org.apache.flink.runtime.clusterframework.FlinkResourceManager

allStartedWorkers, getDesignatedWorkerPoolSize, getLeaderSessionID, getNumberOfStartedTaskManagers, getResourceManagerProps, getStartedTaskManagers, isStarted, notifyWorkerFailed, preStart, sendInfoMessage, startResourceManagerActors, startResourceManagerActors, triggerCheckWorkers, triggerConnectingToJobManager

Methods inherited from class org.apache.flink.runtime.akka.FlinkUntypedActor

decorateMessage, onReceive

Methods inherited from class akka.actor.UntypedActor

akka$actor$Actor$_setter_$context_$eq, akka$actor$Actor$_setter_$self_$eq, aroundPostRestart, aroundPostStop, aroundPreRestart, aroundPreStart, aroundReceive, context, getContext, getSelf, getSender, postRestart, preRestart, receive, self, sender, supervisorStrategy, unhandled

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MesosFlinkResourceManager

public MesosFlinkResourceManager(Configuration flinkConfig,
                                 MesosConfiguration mesosConfig,
                                 MesosWorkerStore workerStore,
                                 LeaderRetrievalService leaderRetrievalService,
                                 MesosTaskManagerParameters taskManagerParameters,
                                 ContainerSpecification taskManagerContainerSpec,
                                 MesosArtifactResolver artifactResolver,
                                 int maxFailedTasks,
                                 int numInitialTaskManagers)

Method Detail

initialize

protected void initialize()
                   throws Exception

Description copied from class: FlinkResourceManager

Initializes the framework specific components.

Specified by:

initialize in class FlinkResourceManager<RegisteredMesosWorkerNode>

Throws:

Exception - Exceptions during initialization cause the resource manager to fail. If the framework is able to recover this resource manager, it will be restarted.

createConnectionMonitor

protected akka.actor.ActorRef createConnectionMonitor()

createTaskRouter

protected akka.actor.ActorRef createTaskRouter()

createLaunchCoordinator

protected akka.actor.ActorRef createLaunchCoordinator()

createReconciliationCoordinator

protected akka.actor.ActorRef createReconciliationCoordinator()

postStop

public void postStop()

Specified by:

postStop in interface akka.actor.Actor

Overrides:

postStop in class FlinkResourceManager<RegisteredMesosWorkerNode>

handleMessage

protected void handleMessage(Object message)

Description copied from class: FlinkResourceManager

This method receives the actor messages after they have been filtered for a match with the leader session.

Overrides:

handleMessage in class FlinkResourceManager<RegisteredMesosWorkerNode>

Parameters:

message - The incoming actor message.

shutdownApplication

protected void shutdownApplication(ApplicationStatus finalStatus,
                                   String optionalDiagnostics)

Called to shut down the cluster (not a failover situation).

Specified by:

shutdownApplication in class FlinkResourceManager<RegisteredMesosWorkerNode>

Parameters:

finalStatus - The application status to report.

optionalDiagnostics - An optional diagnostics message.

fatalError

protected void fatalError(String message,
                          Throwable error)

Description copied from class: FlinkResourceManager

Notifies the resource master of a fatal error.

IMPORTANT: This should not cleanly shut down this master, but exit it in such a way that a high-availability setting would restart this or fail over to another master.

Specified by:

fatalError in class FlinkResourceManager<RegisteredMesosWorkerNode>

requestNewWorkers

protected void requestNewWorkers(int numWorkers)

Plan for some additional workers to be launched.

Specified by:

requestNewWorkers in class FlinkResourceManager<RegisteredMesosWorkerNode>

Parameters:

numWorkers - The number of workers to allocate.

workerStarted

protected RegisteredMesosWorkerNode workerStarted(ResourceID resourceID)

Accept the given started worker into the internal state.

Specified by:

workerStarted in class FlinkResourceManager<RegisteredMesosWorkerNode>

Parameters:

resourceID - The worker resource id

Returns:

A registered worker node record.

reacceptRegisteredWorkers

protected Collection<RegisteredMesosWorkerNode> reacceptRegisteredWorkers(Collection<ResourceID> toConsolidate)

Accept the given registered workers into the internal state.

Specified by:

reacceptRegisteredWorkers in class FlinkResourceManager<RegisteredMesosWorkerNode>

Parameters:

toConsolidate - The worker IDs known previously to the JobManager.

Returns:

A collection of registered worker node records.

releasePendingWorker

protected void releasePendingWorker(ResourceID id)

Release the given pending worker.

Specified by:

releasePendingWorker in class FlinkResourceManager<RegisteredMesosWorkerNode>

Parameters:

id - The worker resource id

releaseStartedWorker

protected void releaseStartedWorker(RegisteredMesosWorkerNode worker)

Release the given started worker.

Specified by:

releaseStartedWorker in class FlinkResourceManager<RegisteredMesosWorkerNode>

Parameters:

worker - The worker resource id

getNumWorkerRequestsPending

protected int getNumWorkerRequestsPending()

Description copied from class: FlinkResourceManager

Gets the number of requested workers that have not yet been granted.

Specified by:

getNumWorkerRequestsPending in class FlinkResourceManager<RegisteredMesosWorkerNode>

Returns:

The number pending worker requests. Must never be smaller than 0.

getNumWorkersPendingRegistration

protected int getNumWorkersPendingRegistration()

Description copied from class: FlinkResourceManager

Gets the number of containers that have been started, but where the TaskManager has not yet registered at the job manager.

Specified by:

getNumWorkersPendingRegistration in class FlinkResourceManager<RegisteredMesosWorkerNode>

Returns:

The number of started containers pending TaskManager registration. Must never be smaller than 0.

createActorProps

public static akka.actor.Props createActorProps(Class<? extends MesosFlinkResourceManager> actorClass,
                                                Configuration flinkConfig,
                                                MesosConfiguration mesosConfig,
                                                MesosWorkerStore workerStore,
                                                LeaderRetrievalService leaderRetrievalService,
                                                MesosTaskManagerParameters taskManagerParameters,
                                                ContainerSpecification taskManagerContainerSpec,
                                                MesosArtifactResolver artifactResolver,
                                                org.slf4j.Logger log)

Creates the props needed to instantiate this actor.

Rather than extracting and validating parameters in the constructor, this factory method takes care of that. That way, errors occur synchronously, and are not swallowed simply in a failed asynchronous attempt to start the actor.

Parameters:

actorClass - The actor class, to allow overriding this actor with subclasses for testing.

flinkConfig - The Flink configuration object.

taskManagerParameters - The parameters for launching TaskManager containers.

taskManagerContainerSpec - The container specification.

artifactResolver - The artifact resolver to locate artifacts

log - The logger to log to.

Returns:

The Props object to instantiate the MesosFlinkResourceManager actor.