org.apache.flink.client

Class RemoteExecutor

java.lang.Object

org.apache.flink.api.common.PlanExecutor

org.apache.flink.client.RemoteExecutor

public class RemoteExecutor
extends PlanExecutor

The RemoteExecutor is a PlanExecutor that takes the program and ships it to a remote Flink cluster for execution.

The RemoteExecutor is pointed at the JobManager and gets the program and (if necessary) the set of libraries that need to be shipped together with the program.

The RemoteExecutor is used in the RemoteEnvironment to remotely execute program parts.

Constructor Summary

Constructors

Constructor and Description
RemoteExecutor(InetSocketAddress inet, Configuration clientConfiguration, List<URL> jarFiles, List<URL> globalClasspaths)
RemoteExecutor(String hostport, Configuration clientConfiguration, URL jarFile)
RemoteExecutor(String hostname, int port)
RemoteExecutor(String hostname, int port, Configuration clientConfiguration)
RemoteExecutor(String hostname, int port, Configuration clientConfiguration, List<URL> jarFiles, List<URL> globalClasspaths)
RemoteExecutor(String hostname, int port, Configuration clientConfiguration, URL jarFile)
RemoteExecutor(String hostname, int port, List<URL> jarFiles)
RemoteExecutor(String hostname, int port, URL jarFile)
RemoteExecutor(String hostport, URL jarFile)

Method Summary

Modifier and Type	Method and Description
void	endSession(JobID jobID) Ends the job session, identified by the given JobID.
JobExecutionResult	executePlan(Plan plan) Execute the given program.
JobExecutionResult	executePlanWithJars(JobWithJars program)
int	getDefaultParallelism() Gets the parallelism that will be used when neither the program does not define any parallelism at all.
String	getOptimizerPlanAsJSON(Plan plan) Gets the programs execution plan in a JSON format.
boolean	isRunning() Checks if this executor is currently running.
void	setDefaultParallelism(int defaultParallelism) Sets the parallelism that will be used when neither the program does not define any parallelism at all.
void	start() Starts the program executor.
void	stop() Shuts down the plan executor and releases all local resources.

Methods inherited from class org.apache.flink.api.common.PlanExecutor

createLocalExecutor, createRemoteExecutor, isPrintingStatusDuringExecution, setPrintStatusDuringExecution

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RemoteExecutor

public RemoteExecutor(String hostname,
                      int port)

RemoteExecutor

public RemoteExecutor(String hostname,
                      int port,
                      URL jarFile)

RemoteExecutor

public RemoteExecutor(String hostport,
                      URL jarFile)

RemoteExecutor

public RemoteExecutor(String hostname,
                      int port,
                      List<URL> jarFiles)

RemoteExecutor

public RemoteExecutor(String hostname,
                      int port,
                      Configuration clientConfiguration)

RemoteExecutor

public RemoteExecutor(String hostname,
                      int port,
                      Configuration clientConfiguration,
                      URL jarFile)

RemoteExecutor

public RemoteExecutor(String hostport,
                      Configuration clientConfiguration,
                      URL jarFile)

RemoteExecutor

public RemoteExecutor(String hostname,
                      int port,
                      Configuration clientConfiguration,
                      List<URL> jarFiles,
                      List<URL> globalClasspaths)

RemoteExecutor

public RemoteExecutor(InetSocketAddress inet,
                      Configuration clientConfiguration,
                      List<URL> jarFiles,
                      List<URL> globalClasspaths)

Method Detail

setDefaultParallelism

public void setDefaultParallelism(int defaultParallelism)

Sets the parallelism that will be used when neither the program does not define any parallelism at all.

Parameters:

defaultParallelism - The default parallelism for the executor.

getDefaultParallelism

public int getDefaultParallelism()

Gets the parallelism that will be used when neither the program does not define any parallelism at all.

Returns:

The default parallelism for the executor.

start

public void start()
           throws Exception

Description copied from class: PlanExecutor

Starts the program executor. After the executor has been started, it will keep running until PlanExecutor.stop() is called.

Specified by:

start in class PlanExecutor

Throws:

Exception - Thrown, if the executor startup failed.

stop

public void stop()
          throws Exception

Description copied from class: PlanExecutor

Shuts down the plan executor and releases all local resources.

This method also ends all sessions created by this executor. Remote job executions may complete, but the session is not kept alive after that.

Specified by:

stop in class PlanExecutor

Throws:

Exception - Thrown, if the proper shutdown failed.

isRunning

public boolean isRunning()

Description copied from class: PlanExecutor

Checks if this executor is currently running.

Specified by:

isRunning in class PlanExecutor

Returns:

True is the executor is running, false otherwise.

executePlan

public JobExecutionResult executePlan(Plan plan)
                               throws Exception

Description copied from class: PlanExecutor

Execute the given program.

If the executor has not been started before, then this method will start the executor and stop it after the execution has completed. This implies that one needs to explicitly start the executor for all programs where multiple dataflow parts depend on each other. Otherwise, the previous parts will no longer be available, because the executor immediately shut down after the execution.

Specified by:

executePlan in class PlanExecutor

Parameters:

plan - The plan of the program to execute.

Returns:

The execution result, containing for example the net runtime of the program, and the accumulators.

Throws:

Exception - Thrown, if job submission caused an exception.

executePlanWithJars

public JobExecutionResult executePlanWithJars(JobWithJars program)
                                       throws Exception

Throws:

Exception

getOptimizerPlanAsJSON

public String getOptimizerPlanAsJSON(Plan plan)
                              throws Exception

Description copied from class: PlanExecutor

Gets the programs execution plan in a JSON format.

Specified by:

getOptimizerPlanAsJSON in class PlanExecutor

Parameters:

plan - The program to get the execution plan for.

Returns:

The execution plan, as a JSON string.

Throws:

Exception - Thrown, if the executor could not connect to the compiler.

endSession

public void endSession(JobID jobID)
                throws Exception

Description copied from class: PlanExecutor

Ends the job session, identified by the given JobID. Jobs can be kept around as sessions, if a session timeout is specified. Keeping Jobs as sessions allows users to incrementally add new operations to their dataflow, that refer to previous intermediate results of the dataflow.

Specified by:

endSession in class PlanExecutor

Parameters:

jobID - The JobID identifying the job session.

Throws:

Exception - Thrown, if the message to finish the session cannot be delivered.