IN
- The type of the input elements@Internal public class PythonTimestampsAndWatermarksOperator<IN> extends OneInputPythonFunctionOperator<IN,IN,Row,Long> implements ProcessingTimeCallback
These two responsibilities run in the same operator rather than in two different ones, because the implementation of the timestamp assigner and the watermark generator is frequently in the same class (and should be run in the same instance), even though the separate interfaces support the use of different classes.
Modifier and Type | Field and Description |
---|---|
static String |
STREAM_TIMESTAMP_AND_WATERMARK_OPERATOR_NAME |
bais, baisWrapper, baos, baosWrapper, collector, DATA_STREAM_STATELESS_FUNCTION_URN, jobOptions
elementCount, maxBundleSize, pythonFunctionRunner
chainingStrategy, latencyStats, LOG, metrics, output, processingTimeService
Constructor and Description |
---|
PythonTimestampsAndWatermarksOperator(Configuration config,
TypeInformation<IN> inputTypeInfo,
DataStreamPythonFunctionInfo pythonFunctionInfo,
WatermarkStrategy<IN> watermarkStrategy) |
Modifier and Type | Method and Description |
---|---|
void |
close()
This method is called after all records have been added to the operators via the methods
Input.processElement(StreamRecord) , or TwoInputStreamOperator.processElement1(StreamRecord) and TwoInputStreamOperator.processElement2(StreamRecord) . |
void |
configureEmitProgressiveWatermarks(boolean emitProgressiveWatermarks) |
void |
emitResult(Tuple2<byte[],Integer> resultTuple)
Sends the execution result to the downstream operator.
|
String |
getCoderUrn() |
void |
onProcessingTime(long timestamp)
This method is invoked with the timestamp for which the trigger was scheduled.
|
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g.
|
void |
processElement(StreamRecord<IN> element)
Processes one element that arrived on this input of the
MultipleInputStreamOperator . |
void |
processWatermark(Watermark mark)
Processes a
Watermark that arrived on the first input of this two-input operator. |
createPythonFunctionRunner, getFunctionUrn, getInternalParameters, getPythonEnv
endInput
checkInvokeFinishBundleByCount, createPythonEnvironmentManager, dispose, emitResults, getConfig, getFlinkMetricContainer, getPythonConfig, invokeFinishBundle, isBundleFinished, prepareSnapshotPreBarrier, setCurrentKey, setPythonConfig
getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, setup, snapshotState, snapshotState
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
setKeyContextElement
dispose, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState
notifyCheckpointAborted, notifyCheckpointComplete
getCurrentKey, setCurrentKey
processLatencyMarker
public static final String STREAM_TIMESTAMP_AND_WATERMARK_OPERATOR_NAME
public PythonTimestampsAndWatermarksOperator(Configuration config, TypeInformation<IN> inputTypeInfo, DataStreamPythonFunctionInfo pythonFunctionInfo, WatermarkStrategy<IN> watermarkStrategy)
public void open() throws Exception
AbstractStreamOperator
The default implementation does nothing.
open
in interface StreamOperator<IN>
open
in class OneInputPythonFunctionOperator<IN,IN,Row,Long>
Exception
- An exception in this method causes the operator to fail.public void processElement(StreamRecord<IN> element) throws Exception
Input
MultipleInputStreamOperator
.
This method is guaranteed to not be called concurrently with other methods of the operator.processElement
in interface Input<IN>
processElement
in class OneInputPythonFunctionOperator<IN,IN,Row,Long>
Exception
public void emitResult(Tuple2<byte[],Integer> resultTuple) throws Exception
AbstractPythonFunctionOperator
emitResult
in class AbstractPythonFunctionOperator<IN>
Exception
public String getCoderUrn()
getCoderUrn
in class OneInputPythonFunctionOperator<IN,IN,Row,Long>
public void configureEmitProgressiveWatermarks(boolean emitProgressiveWatermarks)
public void onProcessingTime(long timestamp)
ProcessingTimeCallback
If the triggering is delayed for whatever reason (trigger timer was blocked, JVM stalled due to a garbage collection), the timestamp supplied to this function will still be the original timestamp for which the trigger was scheduled.
onProcessingTime
in interface ProcessingTimeCallback
timestamp
- The timestamp for which the trigger event was scheduled.public void processWatermark(Watermark mark) throws Exception
Input
Watermark
that arrived on the first input of this two-input operator.
This method is guaranteed to not be called concurrently with other methods of the operator.processWatermark
in interface Input<IN>
processWatermark
in class AbstractPythonFunctionOperator<IN>
Exception
Watermark
public void close() throws Exception
AbstractStreamOperator
Input.processElement(StreamRecord)
, or TwoInputStreamOperator.processElement1(StreamRecord)
and TwoInputStreamOperator.processElement2(StreamRecord)
.
The method is expected to flush all remaining buffered data. Exceptions during this flushing of buffered should be propagated, in order to cause the operation to be recognized asa failed, because the last data items are not processed properly.
close
in interface StreamOperator<IN>
close
in class AbstractPythonFunctionOperator<IN>
Exception
- An exception in this method causes the operator to fail.Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.