IN
- Type of the input elements.OUT
- Type of the output elements.UDFIN
- Type of the UDF input type.@Internal public abstract class AbstractPythonScalarFunctionOperator<IN,OUT,UDFIN> extends AbstractStatelessFunctionOperator<IN,OUT,UDFIN>
ScalarFunction
s. It executes the
Python ScalarFunction
s in separate Python execution environment.
The inputs are assumed as the following format: {{{ +------------------+--------------+ | forwarded fields | extra fields | +------------------+--------------+ }}}.
The Python UDFs may take input columns directly from the input row or the execution result of Java UDFs: 1) The input columns from the input row can be referred from the 'forwarded fields'; 2) The Java UDFs will be computed and the execution results can be referred from the 'extra fields'.
The outputs will be as the following format: {{{ +------------------+-------------------------+ | forwarded fields | scalar function results | +------------------+-------------------------+ }}}.
AbstractStatelessFunctionOperator.StreamRecordCRowWrappingCollector, AbstractStatelessFunctionOperator.StreamRecordRowDataWrappingCollector
Modifier and Type | Field and Description |
---|---|
protected int[] |
forwardedFields
The offset of the fields which should be forwarded.
|
protected PythonFunctionInfo[] |
scalarFunctions
The Python
ScalarFunction s to be executed. |
bais, baisWrapper, forwardedInputQueue, inputType, outputType, userDefinedFunctionInputOffsets, userDefinedFunctionInputType, userDefinedFunctionOutputType, userDefinedFunctionResultQueue
chainingStrategy, latencyStats, LOG, metrics, output, processingTimeService
Modifier and Type | Method and Description |
---|---|
PythonEnv |
getPythonEnv()
Returns the
PythonEnv used to create PythonEnvironmentManager.. |
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g.
|
bufferInput, createPythonFunctionRunner, createPythonFunctionRunner, getFunctionInput, processElement
close, createPythonEnvironmentManager, dispose, emitResults, endInput, getFlinkMetricContainer, getPythonConfig, prepareSnapshotPreBarrier, processWatermark
getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, numEventTimeTimers, numProcessingTimeTimers, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
processLatencyMarker
getMetricGroup, getOperatorID, initializeState, setKeyContextElement1, setKeyContextElement2, snapshotState
notifyCheckpointAborted, notifyCheckpointComplete
getCurrentKey, setCurrentKey
protected final PythonFunctionInfo[] scalarFunctions
ScalarFunction
s to be executed.protected final int[] forwardedFields
public void open() throws Exception
AbstractStreamOperator
The default implementation does nothing.
open
in interface StreamOperator<OUT>
open
in class AbstractStatelessFunctionOperator<IN,OUT,UDFIN>
Exception
- An exception in this method causes the operator to fail.public PythonEnv getPythonEnv()
AbstractPythonFunctionOperator
PythonEnv
used to create PythonEnvironmentManager..getPythonEnv
in class AbstractPythonFunctionOperator<IN,OUT>
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.