org.apache.flink.api.common

Class PlanExecutor

java.lang.Object

org.apache.flink.api.common.PlanExecutor

Direct Known Subclasses:

LocalExecutor, RemoteExecutor

@Internal
public abstract class PlanExecutor
extends Object

A PlanExecutor executes a Flink program's dataflow plan. All Flink programs are translated to dataflow plans prior to execution.

The specific implementation (such as the org.apache.flink.client.LocalExecutor and org.apache.flink.client.RemoteExecutor) determines where and how to run the dataflow. The concrete implementations of the executors are loaded dynamically, because they depend on the full set of all runtime classes.

PlanExecutors can be started explicitly, in which case they keep running until stopped. If a program is submitted to a plan executor that is not running, it will start up for that program, and shut down afterwards.

Constructor Summary

Constructors

Constructor and Description
PlanExecutor()

Method Summary

Modifier and Type	Method and Description
static PlanExecutor	createLocalExecutor(Configuration configuration) Creates an executor that runs the plan locally in a multi-threaded environment.
static PlanExecutor	createRemoteExecutor(String hostname, int port, Configuration clientConfiguration, List<URL> jarFiles, List<URL> globalClasspaths) Creates an executor that runs the plan on a remote environment.
abstract void	endSession(JobID jobID) Ends the job session, identified by the given JobID.
abstract JobExecutionResult	executePlan(Plan plan) Execute the given program.
abstract String	getOptimizerPlanAsJSON(Plan plan) Gets the programs execution plan in a JSON format.
boolean	isPrintingStatusDuringExecution() Gets whether the executor prints progress results to "standard out" (System.out).
abstract boolean	isRunning() Checks if this executor is currently running.
void	setPrintStatusDuringExecution(boolean printStatus) Sets whether the executor should print progress results to "standard out" (System.out).
abstract void	start() Starts the program executor.
abstract void	stop() Shuts down the plan executor and releases all local resources.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PlanExecutor

public PlanExecutor()

Method Detail

setPrintStatusDuringExecution

public void setPrintStatusDuringExecution(boolean printStatus)

Sets whether the executor should print progress results to "standard out" (System.out). All progress messages are logged using the configured logging framework independent of the value set here.

Parameters:

printStatus - True, to print progress updates to standard out, false to not do that.

isPrintingStatusDuringExecution

public boolean isPrintingStatusDuringExecution()

Gets whether the executor prints progress results to "standard out" (System.out).

Returns:

True, if the executor prints progress messages to standard out, false if not.

start

public abstract void start()
                    throws Exception

Starts the program executor. After the executor has been started, it will keep running until stop() is called.

Throws:

Exception - Thrown, if the executor startup failed.

stop

public abstract void stop()
                   throws Exception

Shuts down the plan executor and releases all local resources.

This method also ends all sessions created by this executor. Remote job executions may complete, but the session is not kept alive after that.

Throws:

Exception - Thrown, if the proper shutdown failed.

isRunning

public abstract boolean isRunning()

Checks if this executor is currently running.

Returns:

True is the executor is running, false otherwise.

executePlan

public abstract JobExecutionResult executePlan(Plan plan)
                                        throws Exception

Execute the given program.

If the executor has not been started before, then this method will start the executor and stop it after the execution has completed. This implies that one needs to explicitly start the executor for all programs where multiple dataflow parts depend on each other. Otherwise, the previous parts will no longer be available, because the executor immediately shut down after the execution.

Parameters:

plan - The plan of the program to execute.

Returns:

The execution result, containing for example the net runtime of the program, and the accumulators.

Throws:

Exception - Thrown, if job submission caused an exception.

getOptimizerPlanAsJSON

public abstract String getOptimizerPlanAsJSON(Plan plan)
                                       throws Exception

Gets the programs execution plan in a JSON format.

Parameters:

plan - The program to get the execution plan for.

Returns:

The execution plan, as a JSON string.

Throws:

Exception - Thrown, if the executor could not connect to the compiler.

endSession

public abstract void endSession(JobID jobID)
                         throws Exception

Ends the job session, identified by the given JobID. Jobs can be kept around as sessions, if a session timeout is specified. Keeping Jobs as sessions allows users to incrementally add new operations to their dataflow, that refer to previous intermediate results of the dataflow.

Parameters:

jobID - The JobID identifying the job session.

Throws:

Exception - Thrown, if the message to finish the session cannot be delivered.

createLocalExecutor

public static PlanExecutor createLocalExecutor(Configuration configuration)

Creates an executor that runs the plan locally in a multi-threaded environment.

Returns:

A local executor.

createRemoteExecutor

public static PlanExecutor createRemoteExecutor(String hostname,
                                                int port,
                                                Configuration clientConfiguration,
                                                List<URL> jarFiles,
                                                List<URL> globalClasspaths)

Creates an executor that runs the plan on a remote environment. The remote executor is typically used to send the program to a cluster for execution.

Parameters:

hostname - The address of the JobManager to send the program to.

port - The port of the JobManager to send the program to.

clientConfiguration - The configuration for the client (Akka, default.parallelism).

jarFiles - A list of jar files that contain the user-defined function (UDF) classes and all classes used from within the UDFs.

globalClasspaths - A list of URLs that are added to the classpath of each user code classloader of the program. Paths must specify a protocol (e.g. file://) and be accessible on all nodes.

Returns:

A remote executor.