public class PythonFlatMapFunction extends AbstractPythonUDF<FlatMapFunction<org.python.core.PyObject,Object>> implements FlatMapFunction<org.python.core.PyObject,org.python.core.PyObject>
PythonFlatMapFunction
is a thin wrapper layer over a Python UDF FlatMapFunction
.
It receives a FlatMapFunction
as an input and keeps it internally in a serialized form.
It is then delivered, as part of the job graph, up to the TaskManager, then it is opened and becomes
a sort of mediator to the Python UDF FlatMapFunction
.
This function is used internally by the Python thin wrapper layer over the streaming data functionality
fun, log
Constructor and Description |
---|
PythonFlatMapFunction(FlatMapFunction<org.python.core.PyObject,Object> fun) |
Modifier and Type | Method and Description |
---|---|
void |
flatMap(org.python.core.PyObject value,
Collector<org.python.core.PyObject> out)
The core method of the FlatMapFunction.
|
void |
open(Configuration config)
Initialization method for the function.
|
close
getIterationRuntimeContext, getRuntimeContext, setRuntimeContext
public PythonFlatMapFunction(FlatMapFunction<org.python.core.PyObject,Object> fun) throws IOException
IOException
public void open(Configuration config) throws Exception
RichFunction
The configuration object passed to the function can be used for configuration and initialization. The configuration contains all parameters that were configured on the function in the program composition.
public class MyMapper extends FilterFunction<String> {
private String searchString;
public void open(Configuration parameters) {
this.searchString = parameters.getString("foo");
}
public boolean filter(String value) {
return value.equals(searchString);
}
}
By default, this method does nothing.
open
in interface RichFunction
open
in class AbstractPythonUDF<FlatMapFunction<org.python.core.PyObject,Object>>
config
- The configuration containing the parameters attached to the contract.Exception
- Implementations may forward exceptions, which are caught by the runtime. When the
runtime catches an exception, it aborts the task and lets the fail-over logic
decide whether to retry the task execution.Configuration
public void flatMap(org.python.core.PyObject value, Collector<org.python.core.PyObject> out) throws Exception
FlatMapFunction
flatMap
in interface FlatMapFunction<org.python.core.PyObject,org.python.core.PyObject>
value
- The input value.out
- The collector for returning result values.Exception
- This method may throw exceptions. Throwing an exception will cause the operation
to fail and may trigger recovery.Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.