org.apache.flink.runtime.minicluster

Class FlinkMiniCluster

java.lang.Object

org.apache.flink.runtime.minicluster.FlinkMiniCluster

All Implemented Interfaces:

LeaderRetrievalListener

Direct Known Subclasses:

LocalFlinkMiniCluster

public abstract class FlinkMiniCluster
extends Object
implements LeaderRetrievalListener

Abstract base class for Flink's mini cluster. The mini cluster starts a JobManager and one or multiple TaskManager. Depending on the settings, the different actors can all be run in the same ActorSystem or each one in its own.

Constructor Summary

Constructors

Constructor and Description
FlinkMiniCluster(Configuration userConfiguration, boolean useSingleActorSystem)

Method Summary

Modifier and Type	Method and Description
void	awaitTermination()
protected void	clearLeader()
Configuration	configuration()
protected LeaderRetrievalService	createLeaderRetrievalService()
scala.concurrent.ExecutionContextExecutor	executionContext()
Object	futureLock() Future lock
abstract Configuration	generateConfiguration(Configuration userConfiguration)
com.typesafe.config.Config	getJobManagerAkkaConfig(int index)
List<akka.actor.ActorRef>	getJobManagersAsJava()
ActorGateway	getLeaderGateway(scala.concurrent.duration.FiniteDuration timeout)
scala.concurrent.Future<ActorGateway>	getLeaderGatewayFuture()
int	getLeaderIndex(scala.concurrent.duration.FiniteDuration timeout)
scala.concurrent.Future<Object>	getLeaderIndexFuture()
int	getNumberOfJobManagers()
com.typesafe.config.Config	getTaskManagerAkkaConfig(int index)
scala.collection.Seq<akka.actor.ActorRef>	getTaskManagers()
List<akka.actor.ActorRef>	getTaskManagersAsJava()
void	handleError(Exception exception) This method is called by the LeaderRetrievalService in case of an exception.
String	hostname()
scala.Option<scala.collection.Seq<akka.actor.ActorRef>>	jobManagerActors()
scala.Option<scala.collection.Seq<akka.actor.ActorSystem>>	jobManagerActorSystems()
scala.concurrent.Promise<ActorGateway>	leaderGateway() Future to the ActorGateway of the current leader
scala.concurrent.Promise<Object>	leaderIndex() Future to the index of the current leader
protected scala.Option<LeaderRetrievalService>	leaderRetrievalService()
protected org.slf4j.Logger	LOG()
void	notifyLeaderAddress(String address, UUID leaderSessionID) This method is called by the LeaderRetrievalService when a new leader is elected.
int	numJobManagers()
int	numTaskManagers()
RecoveryMode	recoveryMode()
boolean	running()
void	setDefaultCiConfig(Configuration config) Sets CI environment (Travis) specific config defaults.
protected void	shutdown()
void	shutdownJobClientActorSystem(akka.actor.ActorSystem actorSystem)
void	start()
void	start(boolean waitForTaskManagerRegistration)
akka.actor.ActorSystem	startJobClientActorSystem(JobID jobID)
abstract akka.actor.ActorRef	startJobManager(int index, akka.actor.ActorSystem system)
akka.actor.ActorSystem	startJobManagerActorSystem(int index)
abstract akka.actor.ActorRef	startTaskManager(int index, akka.actor.ActorSystem system)
akka.actor.ActorSystem	startTaskManagerActorSystem(int index)
scala.Option<WebMonitor>	startWebServer(Configuration config, akka.actor.ActorSystem actorSystem, String jobManagerAkkaURL)
void	stop()
JobExecutionResult	submitJobAndWait(JobGraph jobGraph, boolean printUpdates)
JobExecutionResult	submitJobAndWait(JobGraph jobGraph, boolean printUpdates, scala.concurrent.duration.FiniteDuration timeout)
JobExecutionResult	submitJobAndWait(JobGraph jobGraph, boolean printUpdates, scala.concurrent.duration.FiniteDuration timeout, LeaderRetrievalService leaderRetrievalService)
JobSubmissionResult	submitJobDetached(JobGraph jobGraph)
scala.Option<scala.collection.Seq<akka.actor.ActorRef>>	taskManagerActors()
scala.Option<scala.collection.Seq<akka.actor.ActorSystem>>	taskManagerActorSystems()
scala.concurrent.duration.FiniteDuration	timeout()
Configuration	userConfiguration()
boolean	useSingleActorSystem()
void	waitForTaskManagersToBeRegistered() Waits with the default timeout until all task managers have registered at the job manager
void	waitForTaskManagersToBeRegistered(scala.concurrent.duration.FiniteDuration timeout) Waits until all task managers have registered at the job manager until the timeout is reached.
scala.Option<WebMonitor>	webMonitor()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FlinkMiniCluster

public FlinkMiniCluster(Configuration userConfiguration,
                        boolean useSingleActorSystem)

Method Detail

userConfiguration

public Configuration userConfiguration()

useSingleActorSystem

public boolean useSingleActorSystem()

LOG

protected org.slf4j.Logger LOG()

hostname

public String hostname()

configuration

public Configuration configuration()

leaderGateway

public scala.concurrent.Promise<ActorGateway> leaderGateway()

Future to the ActorGateway of the current leader

leaderIndex

public scala.concurrent.Promise<Object> leaderIndex()

Future to the index of the current leader

futureLock

public Object futureLock()

Future lock

executionContext

public scala.concurrent.ExecutionContextExecutor executionContext()

timeout

public scala.concurrent.duration.FiniteDuration timeout()

recoveryMode

public RecoveryMode recoveryMode()

numJobManagers

public int numJobManagers()

numTaskManagers

public int numTaskManagers()

jobManagerActorSystems

public scala.Option<scala.collection.Seq<akka.actor.ActorSystem>> jobManagerActorSystems()

jobManagerActors

public scala.Option<scala.collection.Seq<akka.actor.ActorRef>> jobManagerActors()

webMonitor

public scala.Option<WebMonitor> webMonitor()

taskManagerActorSystems

public scala.Option<scala.collection.Seq<akka.actor.ActorSystem>> taskManagerActorSystems()

taskManagerActors

public scala.Option<scala.collection.Seq<akka.actor.ActorRef>> taskManagerActors()

leaderRetrievalService

protected scala.Option<LeaderRetrievalService> leaderRetrievalService()

generateConfiguration

public abstract Configuration generateConfiguration(Configuration userConfiguration)

startJobManager

public abstract akka.actor.ActorRef startJobManager(int index,
                                                    akka.actor.ActorSystem system)

startTaskManager

public abstract akka.actor.ActorRef startTaskManager(int index,
                                                     akka.actor.ActorSystem system)

getNumberOfJobManagers

public int getNumberOfJobManagers()

getJobManagersAsJava

public List<akka.actor.ActorRef> getJobManagersAsJava()

getTaskManagers

public scala.collection.Seq<akka.actor.ActorRef> getTaskManagers()

getTaskManagersAsJava

public List<akka.actor.ActorRef> getTaskManagersAsJava()

getLeaderGatewayFuture

public scala.concurrent.Future<ActorGateway> getLeaderGatewayFuture()

getLeaderGateway

public ActorGateway getLeaderGateway(scala.concurrent.duration.FiniteDuration timeout)

getLeaderIndexFuture

public scala.concurrent.Future<Object> getLeaderIndexFuture()

getLeaderIndex

public int getLeaderIndex(scala.concurrent.duration.FiniteDuration timeout)

getJobManagerAkkaConfig

public com.typesafe.config.Config getJobManagerAkkaConfig(int index)

getTaskManagerAkkaConfig

public com.typesafe.config.Config getTaskManagerAkkaConfig(int index)

setDefaultCiConfig

public void setDefaultCiConfig(Configuration config)

Sets CI environment (Travis) specific config defaults.

startJobManagerActorSystem

public akka.actor.ActorSystem startJobManagerActorSystem(int index)

startTaskManagerActorSystem

public akka.actor.ActorSystem startTaskManagerActorSystem(int index)

startJobClientActorSystem

public akka.actor.ActorSystem startJobClientActorSystem(JobID jobID)

start

public void start()

start

public void start(boolean waitForTaskManagerRegistration)

startWebServer

public scala.Option<WebMonitor> startWebServer(Configuration config,
                                               akka.actor.ActorSystem actorSystem,
                                               String jobManagerAkkaURL)

stop

public void stop()

shutdown

protected void shutdown()

awaitTermination

public void awaitTermination()

running

public boolean running()

waitForTaskManagersToBeRegistered

public void waitForTaskManagersToBeRegistered()

Waits with the default timeout until all task managers have registered at the job manager

Throws:

TimeoutException

InterruptedException

waitForTaskManagersToBeRegistered

public void waitForTaskManagersToBeRegistered(scala.concurrent.duration.FiniteDuration timeout)

Waits until all task managers have registered at the job manager until the timeout is reached.

Parameters:

timeout -

Throws:

TimeoutException

InterruptedException

submitJobAndWait

public JobExecutionResult submitJobAndWait(JobGraph jobGraph,
                                           boolean printUpdates)

submitJobAndWait

public JobExecutionResult submitJobAndWait(JobGraph jobGraph,
                                           boolean printUpdates,
                                           scala.concurrent.duration.FiniteDuration timeout)

submitJobAndWait

public JobExecutionResult submitJobAndWait(JobGraph jobGraph,
                                           boolean printUpdates,
                                           scala.concurrent.duration.FiniteDuration timeout,
                                           LeaderRetrievalService leaderRetrievalService)

submitJobDetached

public JobSubmissionResult submitJobDetached(JobGraph jobGraph)

shutdownJobClientActorSystem

public void shutdownJobClientActorSystem(akka.actor.ActorSystem actorSystem)

createLeaderRetrievalService

protected LeaderRetrievalService createLeaderRetrievalService()

clearLeader

protected void clearLeader()

notifyLeaderAddress

public void notifyLeaderAddress(String address,
                                UUID leaderSessionID)

Description copied from interface: LeaderRetrievalListener

This method is called by the LeaderRetrievalService when a new leader is elected.

Specified by:

notifyLeaderAddress in interface LeaderRetrievalListener

Parameters:

address - The address of the new leader

leaderSessionID - The new leader session ID

handleError

public void handleError(Exception exception)

Description copied from interface: LeaderRetrievalListener

This method is called by the LeaderRetrievalService in case of an exception. This assures that the LeaderRetrievalListener is aware of any problems occurring in the LeaderRetrievalService thread.

Specified by:

handleError in interface LeaderRetrievalListener