@PublicEvolving public class PythonStreamExecutionEnvironment extends Object
StreamExecutionEnvironment
.
The PythonStreamExecutionEnvironment is the context in which a streaming program is executed.
The environment provides methods to control the job execution (such as setting the parallelism or the fault tolerance/checkpointing parameters) and to interact with the outside world (data access).
Modifier and Type | Class and Description |
---|---|
static class |
PythonStreamExecutionEnvironment.PythonJobParameters
Utility class for storing/retrieving parameters in/from a
ExecutionConfig.GlobalJobParameters . |
public PythonDataStream create_python_source(SourceFunction<Object> src) throws Exception
Exception
public PythonDataStream add_java_source(SourceFunction<Object> src)
src
- A native java source (e.g. PythonFlinkKafkaConsumer09)public PythonDataStream from_elements(org.python.core.PyObject... elements)
StreamExecutionEnvironment.fromElements(java.lang.Object[])
.elements
- The array of PyObject elements to create the data stream from.public PythonDataStream from_collection(Collection<Object> collection)
StreamExecutionEnvironment.fromCollection(java.util.Collection)
The input Collection
is of type Object
, because it is a collection
of Python elements. * There type is determined in runtime, by the Jython framework.
collection
- The collection of python elements to create the data stream from.public PythonDataStream from_collection(Iterator<Object> iter) throws Exception
Note that this operation will result in a non-parallel data stream source, i.e., a data stream source with a parallelism of one.
iter
- The iterator of elements to create the data stream fromException
StreamExecutionEnvironment.fromCollection(java.util.Iterator, org.apache.flink.api.common.typeinfo.TypeInformation)
public PythonDataStream generate_sequence(long from, long to)
StreamExecutionEnvironment.generateSequence(long, long)
.from
- The number to start at (inclusive)to
- The number to stop at (inclusive)public PythonDataStream read_text_file(String path) throws IOException
StreamExecutionEnvironment.readTextFile(java.lang.String)
.path
- The path of the file, as a URI (e.g., "file:///some/local/file" or "hdfs://host:port/file/path").IOException
public PythonDataStream socket_text_stream(String host, int port)
StreamExecutionEnvironment.socketTextStream(java.lang.String, int)
.host
- The host name which a server socket bindsport
- The port number which a server socket binds. A port number of 0 means that the port number is automatically
allocated.public PythonStreamExecutionEnvironment enable_checkpointing(long interval)
StreamExecutionEnvironment.enableCheckpointing(long)
.interval
- Time interval between state checkpoints in milliseconds.PythonStreamExecutionEnvironment
instance of the callerpublic PythonStreamExecutionEnvironment enable_checkpointing(long interval, CheckpointingMode mode)
StreamExecutionEnvironment.enableCheckpointing(long, CheckpointingMode)
.interval
- Time interval between state checkpoints in milliseconds.mode
- The checkpointing mode, selecting between "exactly once" and "at least once" guaranteed.PythonStreamExecutionEnvironment
instance of the callerpublic PythonStreamExecutionEnvironment set_parallelism(int parallelism)
StreamExecutionEnvironment.setParallelism(int)
.parallelism
- The parallelismPythonStreamExecutionEnvironment
instance of the callerpublic JobExecutionResult execute() throws Exception
StreamExecutionEnvironment.execute()
.Exception
public JobExecutionResult execute(String job_name) throws Exception
StreamExecutionEnvironment.execute(java.lang.String)
.Exception
- which occurs during job execution.Copyright © 2014–2019 The Apache Software Foundation. All rights reserved.