@Internal public class ArrowPythonScalarFunctionOperator extends AbstractRowPythonScalarFunctionOperator
ScalarFunction
operator for the old planner.AbstractStatelessFunctionOperator.StreamRecordCRowWrappingCollector, AbstractStatelessFunctionOperator.StreamRecordRowDataWrappingCollector
cRowWrapper
forwardedFields, scalarFunctions
bais, baisWrapper, forwardedInputQueue, inputType, outputType, userDefinedFunctionInputOffsets, userDefinedFunctionInputType, userDefinedFunctionOutputType, userDefinedFunctionResultQueue
chainingStrategy, latencyStats, LOG, metrics, output, processingTimeService
Constructor and Description |
---|
ArrowPythonScalarFunctionOperator(Configuration config,
PythonFunctionInfo[] scalarFunctions,
RowType inputType,
RowType outputType,
int[] udfInputOffsets,
int[] forwardedFields) |
Modifier and Type | Method and Description |
---|---|
void |
close()
This method is called after all records have been added to the operators via the methods
OneInputStreamOperator.processElement(StreamRecord) , or TwoInputStreamOperator.processElement1(StreamRecord) and TwoInputStreamOperator.processElement2(StreamRecord) . |
PythonFunctionRunner<Row> |
createPythonFunctionRunner(org.apache.beam.sdk.fn.data.FnDataReceiver<byte[]> resultReceiver,
PythonEnvironmentManager pythonEnvironmentManager,
Map<String,String> jobOptions) |
void |
emitResults()
Sends the execution results to the downstream operator.
|
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g.
|
bufferInput, getFunctionInput
getPythonEnv
createPythonFunctionRunner, processElement
createPythonEnvironmentManager, dispose, endInput, getFlinkMetricContainer, getPythonConfig, prepareSnapshotPreBarrier, processWatermark
getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, numEventTimeTimers, numProcessingTimeTimers, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
processLatencyMarker
getMetricGroup, getOperatorID, initializeState, setKeyContextElement1, setKeyContextElement2, snapshotState
notifyCheckpointAborted, notifyCheckpointComplete
getCurrentKey, setCurrentKey
public ArrowPythonScalarFunctionOperator(Configuration config, PythonFunctionInfo[] scalarFunctions, RowType inputType, RowType outputType, int[] udfInputOffsets, int[] forwardedFields)
public void open() throws Exception
AbstractStreamOperator
The default implementation does nothing.
open
in interface StreamOperator<org.apache.flink.table.runtime.types.CRow>
open
in class AbstractRowPythonScalarFunctionOperator
Exception
- An exception in this method causes the operator to fail.public void close() throws Exception
AbstractStreamOperator
OneInputStreamOperator.processElement(StreamRecord)
, or TwoInputStreamOperator.processElement1(StreamRecord)
and TwoInputStreamOperator.processElement2(StreamRecord)
.
The method is expected to flush all remaining buffered data. Exceptions during this flushing of buffered should be propagated, in order to cause the operation to be recognized asa failed, because the last data items are not processed properly.
close
in interface StreamOperator<org.apache.flink.table.runtime.types.CRow>
close
in class AbstractPythonFunctionOperator<org.apache.flink.table.runtime.types.CRow,org.apache.flink.table.runtime.types.CRow>
Exception
- An exception in this method causes the operator to fail.public PythonFunctionRunner<Row> createPythonFunctionRunner(org.apache.beam.sdk.fn.data.FnDataReceiver<byte[]> resultReceiver, PythonEnvironmentManager pythonEnvironmentManager, Map<String,String> jobOptions)
createPythonFunctionRunner
in class AbstractStatelessFunctionOperator<org.apache.flink.table.runtime.types.CRow,org.apache.flink.table.runtime.types.CRow,Row>
public void emitResults() throws IOException
AbstractPythonFunctionOperator
emitResults
in class AbstractPythonFunctionOperator<org.apache.flink.table.runtime.types.CRow,org.apache.flink.table.runtime.types.CRow>
IOException
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.