IN
- Type of the input elements.@Internal public abstract class AbstractArrowPythonScalarFunctionRunner<IN> extends AbstractPythonScalarFunctionRunner<IN>
PythonFunctionRunner
used to execute Arrow Python ScalarFunction
s.Modifier and Type | Field and Description |
---|---|
protected org.apache.arrow.vector.VectorSchemaRoot |
root
Container that holds a set of vectors for the input elements to be sent to the Python worker.
|
baos, baosWrapper, flinkMetricContainer, mainInputReceiver, resultReceiver
Constructor and Description |
---|
AbstractArrowPythonScalarFunctionRunner(String taskName,
org.apache.beam.sdk.fn.data.FnDataReceiver<byte[]> resultReceiver,
PythonFunctionInfo[] scalarFunctions,
PythonEnvironmentManager environmentManager,
RowType inputType,
RowType outputType,
int maxArrowBatchSize,
Map<String,String> jobOptions,
FlinkMetricContainer flinkMetricContainer) |
Modifier and Type | Method and Description |
---|---|
void |
close()
Tear-down the Python function runner.
|
abstract ArrowWriter<IN> |
createArrowWriter()
Creates an
ArrowWriter . |
org.apache.beam.runners.fnexecution.control.OutputReceiverFactory |
createOutputReceiverFactory() |
void |
finishBundle()
Forces to finish the processing of the current bundle of elements.
|
String |
getInputOutputCoderUrn() |
void |
open()
Prepares the Python function runner, such as preparing the Python execution environment, etc.
|
void |
processElement(IN element)
Executes the Python function with the input element.
|
getUserDefinedFunctionsProto
createExecutableStage, getInputType, getOutputType, getUserDefinedFunctionProto
createJobBundleFactory, createPythonExecutionEnvironment, startBundle
protected transient org.apache.arrow.vector.VectorSchemaRoot root
public AbstractArrowPythonScalarFunctionRunner(String taskName, org.apache.beam.sdk.fn.data.FnDataReceiver<byte[]> resultReceiver, PythonFunctionInfo[] scalarFunctions, PythonEnvironmentManager environmentManager, RowType inputType, RowType outputType, int maxArrowBatchSize, Map<String,String> jobOptions, FlinkMetricContainer flinkMetricContainer)
public void open() throws Exception
PythonFunctionRunner
open
in interface PythonFunctionRunner<IN>
open
in class AbstractPythonFunctionRunner<IN>
Exception
public void close() throws Exception
PythonFunctionRunner
close
in interface PythonFunctionRunner<IN>
close
in class AbstractPythonFunctionRunner<IN>
Exception
public void processElement(IN element)
PythonFunctionRunner
public void finishBundle() throws Exception
PythonFunctionRunner
finishBundle
in interface PythonFunctionRunner<IN>
finishBundle
in class AbstractPythonFunctionRunner<IN>
Exception
public org.apache.beam.runners.fnexecution.control.OutputReceiverFactory createOutputReceiverFactory()
createOutputReceiverFactory
in class AbstractPythonFunctionRunner<IN>
public String getInputOutputCoderUrn()
getInputOutputCoderUrn
in class AbstractPythonStatelessFunctionRunner<IN>
public abstract ArrowWriter<IN> createArrowWriter()
ArrowWriter
.Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.