@Internal public abstract class AbstractExternalTwoInputPythonFunctionOperator<IN1,IN2,OUT> extends AbstractExternalDataStreamPythonFunctionOperator<OUT> implements TwoInputStreamOperator<IN1,IN2,OUT>, BoundedMultiInput
AbstractExternalTwoInputPythonFunctionOperator
is responsible for launching beam runner
which will start a python harness to execute two-input user defined python function.Modifier and Type | Field and Description |
---|---|
protected ByteArrayOutputStreamWithPos |
baos |
protected DataOutputViewStreamWrapper |
baosWrapper |
pythonFunctionRunner
bundleFinishedCallback, config, elementCount, lastFinishBundleTime, maxBundleSize, systemEnvEnabled
chainingStrategy, latencyStats, LOG, metrics, output, processingTimeService
Constructor and Description |
---|
AbstractExternalTwoInputPythonFunctionOperator(Configuration config,
DataStreamPythonFunctionInfo pythonFunctionInfo,
TypeInformation<IN1> inputTypeInfo1,
TypeInformation<IN2> inputTypeInfo2,
TypeInformation<OUT> outputTypeInfo) |
Modifier and Type | Method and Description |
---|---|
FlinkFnApi.CoderInfoDescriptor |
createInputCoderInfoDescriptor() |
FlinkFnApi.CoderInfoDescriptor |
createOutputCoderInfoDescriptor() |
void |
emitResult(Tuple3<String,byte[],Integer> resultTuple)
Sends the execution result to the downstream operator.
|
void |
endInput(int inputId)
It is notified that no more data will arrive from the input identified by the
inputId . |
protected TypeInformation<IN1> |
getLeftInputType() |
protected TypeInformation<IN2> |
getRightInputType() |
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g. state initialization.
|
void |
processElement(boolean isLeft,
long timestamp,
long watermark,
Object element) |
addSideOutputTags, createSideOutputCoderDescriptors, getInternalParameters, getOutputTagById, getProducedType, getPythonEnv, getPythonFunctionInfo, getSideOutputTags, getSideOutputTypeSerializerById, setNumPartitions
close, createPythonEnvironmentManager, createPythonFunctionRunner, emitResults, invokeFinishBundle
checkInvokeFinishBundleByCount, finish, getConfiguration, getFlinkMetricContainer, isBundleFinished, prepareSnapshotPreBarrier, processWatermark, setCurrentKey
getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
processElement1, processElement2, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus1, processWatermarkStatus2
close, finish, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState
notifyCheckpointAborted, notifyCheckpointComplete
getCurrentKey, setCurrentKey
copy
hasKeyContext
protected transient ByteArrayOutputStreamWithPos baos
protected transient DataOutputViewStreamWrapper baosWrapper
public AbstractExternalTwoInputPythonFunctionOperator(Configuration config, DataStreamPythonFunctionInfo pythonFunctionInfo, TypeInformation<IN1> inputTypeInfo1, TypeInformation<IN2> inputTypeInfo2, TypeInformation<OUT> outputTypeInfo)
public void open() throws Exception
AbstractStreamOperator
The default implementation does nothing.
open
in interface StreamOperator<OUT>
open
in class AbstractExternalDataStreamPythonFunctionOperator<OUT>
Exception
- An exception in this method causes the operator to fail.public void endInput(int inputId) throws Exception
BoundedMultiInput
inputId
. The inputId
is numbered starting from 1, and `1` indicates the first input.
WARNING: It is not safe to use this method to commit any transactions or other side
effects! You can use this method to e.g. flush data buffered for the given input or implement
an ordered reading from multiple inputs via InputSelectable
.
endInput
in interface BoundedMultiInput
Exception
public void emitResult(Tuple3<String,byte[],Integer> resultTuple) throws Exception
AbstractExternalPythonFunctionOperator
emitResult
in class AbstractExternalPythonFunctionOperator<OUT>
Exception
public void processElement(boolean isLeft, long timestamp, long watermark, Object element) throws Exception
Exception
public FlinkFnApi.CoderInfoDescriptor createInputCoderInfoDescriptor()
public FlinkFnApi.CoderInfoDescriptor createOutputCoderInfoDescriptor()
protected TypeInformation<IN1> getLeftInputType()
protected TypeInformation<IN2> getRightInputType()
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.