IN- Input type for the operator.
OUT- Output type for the operator.
@Internal public class AsyncWaitOperator<IN,OUT> extends AbstractUdfStreamOperator<OUT,AsyncFunction<IN,OUT>> implements OneInputStreamOperator<IN,OUT>, BoundedOneInput
AsyncWaitOperatorallows to asynchronously process incoming stream records. For that the operator creates an
ResultFuturewhich is passed to an
AsyncFunction. Within the async function, the user can complete the async collector arbitrarily. Once the async collector has been completed, the result is emitted by the operator's emitter to downstream operators.
The operator offers different output modes depending on the chosen
order to give exactly once processing guarantees, the operator stores all currently in-flight
StreamElement in it's operator state. Upon recovery the recorded set of stream elements
The operator also support retry on predefined condition and strategy
AsyncRetryStrategy. If given strategy is NO_RETRY_STRATEGY or a custom implemented
AsyncRetryPredicate which defines empty retry
predicate, then no retry will happen, everything keeps the same as the
without retry support.
In case of chaining of this operator, it has to be made sure that the operators in the chain
are opened tail to head. The reason for this is that an opened
already emitting recovered
StreamElement to downstream operators.
|Constructor and Description|
|Modifier and Type||Method and Description|
It is notified that no more data will arrive from the input.
Stream operators with state which can be restored need to override this hook method.
This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g.
Processes one element that arrived on this input of the
Initializes the operator.
Stream operators with state, which want to participate in a snapshot need to override this hook method.
close, finish, getUserFunction, getUserFunctionParameters, notifyCheckpointAborted, notifyCheckpointComplete, setOutputType
getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, isUsingCustomRawKeyedState, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, registerCounterOnOutput, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, snapshotState
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
close, finish, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState
public AsyncWaitOperator(@Nonnull AsyncFunction<IN,OUT> asyncFunction, long timeout, int capacity, @Nonnull AsyncDataStream.OutputMode outputMode, @Nonnull AsyncRetryStrategy<OUT> asyncRetryStrategy, @Nonnull ProcessingTimeService processingTimeService, @Nonnull MailboxExecutor mailboxExecutor)
public void setup(StreamTask<?,?> containingTask, StreamConfig config, Output<StreamRecord<OUT>> output)
public void open() throws Exception
The default implementation does nothing.
public void processElement(StreamRecord<IN> record) throws Exception
MultipleInputStreamOperator. This method is guaranteed to not be called concurrently with other methods of the operator.
Watermarkthat arrived on the first input of this two-input operator. This method is guaranteed to not be called concurrently with other methods of the operator.
public void snapshotState(StateSnapshotContext context) throws Exception
public void initializeState(StateInitializationContext context) throws Exception
public void endInput() throws Exception
WARNING: It is not safe to use this method to commit any transactions or other side
effects! You can use this method to flush any buffered data that can later on be committed
e.g. in a
NOTE: Given it is semantically very similar to the
method. It might be dropped in favour of the other method at some point in time.
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.