org.apache.flink.streaming.api.environment

Class RemoteStreamEnvironment

java.lang.Object

org.apache.flink.streaming.api.environment.StreamExecutionEnvironment

org.apache.flink.streaming.api.environment.RemoteStreamEnvironment

All Implemented Interfaces:

AutoCloseable

@Public
public class RemoteStreamEnvironment
extends StreamExecutionEnvironment

A StreamExecutionEnvironment for executing on a cluster.

Field Summary

Fields inherited from class org.apache.flink.streaming.api.environment.StreamExecutionEnvironment

cacheFile, checkpointCfg, config, configuration, DEFAULT_JOB_NAME, isChainingEnabled, transformations

Constructor Summary

Constructors

Constructor and Description
RemoteStreamEnvironment(PipelineExecutorServiceLoader executorServiceLoader, String host, int port, Configuration clientConfiguration, String[] jarFiles, URL[] globalClasspaths, SavepointRestoreSettings savepointRestoreSettings)
RemoteStreamEnvironment(String host, int port, Configuration clientConfiguration, String... jarFiles) Creates a new RemoteStreamEnvironment that points to the master (JobManager) described by the given host name and port.
RemoteStreamEnvironment(String host, int port, Configuration clientConfiguration, String[] jarFiles, URL[] globalClasspaths) Creates a new RemoteStreamEnvironment that points to the master (JobManager) described by the given host name and port.
RemoteStreamEnvironment(String host, int port, Configuration clientConfiguration, String[] jarFiles, URL[] globalClasspaths, SavepointRestoreSettings savepointRestoreSettings) Creates a new RemoteStreamEnvironment that points to the master (JobManager) described by the given host name and port.
RemoteStreamEnvironment(String host, int port, String... jarFiles) Creates a new RemoteStreamEnvironment that points to the master (JobManager) described by the given host name and port.

Method Summary

Modifier and Type	Method and Description
JobExecutionResult	execute(StreamGraph streamGraph) Triggers the program execution.
Configuration	getClientConfiguration() Deprecated. This method is going to be removed in the next releases.
String	getHost() Gets the hostname of the master (JobManager), where the program will be executed.
int	getPort() Gets the port of the master (JobManager), where the program will be executed.
String	toString()

Methods inherited from class org.apache.flink.streaming.api.environment.StreamExecutionEnvironment

addDefaultKryoSerializer, addDefaultKryoSerializer, addOperator, addSource, addSource, addSource, addSource, areExplicitEnvironmentsAllowed, clean, clearJobListeners, close, configure, configure, createInput, createInput, createLocalEnvironment, createLocalEnvironment, createLocalEnvironment, createLocalEnvironment, createLocalEnvironmentWithWebUI, createRemoteEnvironment, createRemoteEnvironment, createRemoteEnvironment, disableOperatorChaining, enableChangelogStateBackend, enableCheckpointing, enableCheckpointing, enableCheckpointing, enableCheckpointing, execute, execute, executeAsync, executeAsync, executeAsync, fromCollection, fromCollection, fromCollection, fromCollection, fromElements, fromElements, fromParallelCollection, fromParallelCollection, fromSequence, fromSource, fromSource, generateSequence, generateStreamGraph, getBufferTimeout, getCachedFiles, getCheckpointConfig, getCheckpointingMode, getCheckpointInterval, getConfig, getConfiguration, getDefaultLocalParallelism, getDefaultSavepointDirectory, getExecutionEnvironment, getExecutionEnvironment, getExecutionPlan, getJobListeners, getMaxParallelism, getNumberOfExecutionRetries, getParallelism, getRestartStrategy, getStateBackend, getStreamGraph, getStreamGraph, getStreamTimeCharacteristic, getTransformations, getUserClassloader, initializeContextEnvironment, invalidateClusterDataset, isChainingEnabled, isChangelogStateBackendEnabled, isForceCheckpointing, isForceUnalignedCheckpoints, isUnalignedCheckpointsEnabled, listCompletedClusterDatasets, readFile, readFile, readFile, readFile, readFileStream, readTextFile, readTextFile, registerCachedFile, registerCachedFile, registerCacheTransformation, registerCollectIterator, registerJobListener, registerSlotSharingGroup, registerType, registerTypeWithKryoSerializer, registerTypeWithKryoSerializer, resetContextEnvironment, setBufferTimeout, setDefaultLocalParallelism, setDefaultSavepointDirectory, setDefaultSavepointDirectory, setDefaultSavepointDirectory, setMaxParallelism, setNumberOfExecutionRetries, setParallelism, setRestartStrategy, setRuntimeMode, setStateBackend, setStreamTimeCharacteristic, socketTextStream, socketTextStream, socketTextStream, socketTextStream, socketTextStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

RemoteStreamEnvironment

public RemoteStreamEnvironment(String host,
                               int port,
                               String... jarFiles)

Creates a new RemoteStreamEnvironment that points to the master (JobManager) described by the given host name and port.

Parameters:

host - The host name or address of the master (JobManager), where the program should be executed.

port - The port of the master (JobManager), where the program should be executed.

jarFiles - The JAR files with code that needs to be shipped to the cluster. If the program uses user-defined functions, user-defined input formats, or any libraries, those must be provided in the JAR files.

RemoteStreamEnvironment

public RemoteStreamEnvironment(String host,
                               int port,
                               Configuration clientConfiguration,
                               String... jarFiles)

Creates a new RemoteStreamEnvironment that points to the master (JobManager) described by the given host name and port.

Parameters:

host - The host name or address of the master (JobManager), where the program should be executed.

port - The port of the master (JobManager), where the program should be executed.

clientConfiguration - The configuration used to parametrize the client that connects to the remote cluster.

jarFiles - The JAR files with code that needs to be shipped to the cluster. If the program uses user-defined functions, user-defined input formats, or any libraries, those must be provided in the JAR files.

RemoteStreamEnvironment

public RemoteStreamEnvironment(String host,
                               int port,
                               Configuration clientConfiguration,
                               String[] jarFiles,
                               URL[] globalClasspaths)

Creates a new RemoteStreamEnvironment that points to the master (JobManager) described by the given host name and port.

Parameters:

host - The host name or address of the master (JobManager), where the program should be executed.

port - The port of the master (JobManager), where the program should be executed.

clientConfiguration - The configuration used to parametrize the client that connects to the remote cluster.

jarFiles - The JAR files with code that needs to be shipped to the cluster. If the program uses user-defined functions, user-defined input formats, or any libraries, those must be provided in the JAR files.

globalClasspaths - The paths of directories and JAR files that are added to each user code classloader on all nodes in the cluster. Note that the paths must specify a protocol (e.g. file://) and be accessible on all nodes (e.g. by means of a NFS share). The protocol must be supported by the URLClassLoader.

RemoteStreamEnvironment

@PublicEvolving
public RemoteStreamEnvironment(String host,
                                               int port,
                                               Configuration clientConfiguration,
                                               String[] jarFiles,
                                               URL[] globalClasspaths,
                                               SavepointRestoreSettings savepointRestoreSettings)

Creates a new RemoteStreamEnvironment that points to the master (JobManager) described by the given host name and port.

Parameters:

host - The host name or address of the master (JobManager), where the program should be executed.

port - The port of the master (JobManager), where the program should be executed.

clientConfiguration - The configuration used to parametrize the client that connects to the remote cluster.

jarFiles - The JAR files with code that needs to be shipped to the cluster. If the program uses user-defined functions, user-defined input formats, or any libraries, those must be provided in the JAR files.

globalClasspaths - The paths of directories and JAR files that are added to each user code classloader on all nodes in the cluster. Note that the paths must specify a protocol (e.g. file://) and be accessible on all nodes (e.g. by means of a NFS share). The protocol must be supported by the URLClassLoader.

savepointRestoreSettings - Optional savepoint restore settings for job execution.

RemoteStreamEnvironment

@PublicEvolving
public RemoteStreamEnvironment(PipelineExecutorServiceLoader executorServiceLoader,
                                               String host,
                                               int port,
                                               Configuration clientConfiguration,
                                               String[] jarFiles,
                                               URL[] globalClasspaths,
                                               SavepointRestoreSettings savepointRestoreSettings)

Method Detail

execute

public JobExecutionResult execute(StreamGraph streamGraph)
                           throws Exception

Description copied from class: StreamExecutionEnvironment

Triggers the program execution. The environment will execute all parts of the program that have resulted in a "sink" operation. Sink operations are for example printing results or forwarding them to a message queue.

Overrides:

execute in class StreamExecutionEnvironment

Parameters:

streamGraph - the stream graph representing the transformations

Returns:

The result of the job execution, containing elapsed time and accumulators.

Throws:

Exception - which occurs during job execution.

toString

public String toString()

Overrides:

toString in class Object

getHost

public String getHost()

Gets the hostname of the master (JobManager), where the program will be executed.

Returns:

The hostname of the master

getPort

public int getPort()

Gets the port of the master (JobManager), where the program will be executed.

Returns:

The port of the master

getClientConfiguration

@Deprecated
public Configuration getClientConfiguration()

Deprecated. This method is going to be removed in the next releases.