IN - Input type for the operator.OUT - Output type for the operator.@Internal public class AsyncWaitOperator<IN,OUT> extends AbstractUdfStreamOperator<OUT,AsyncFunction<IN,OUT>> implements OneInputStreamOperator<IN,OUT>, BoundedOneInput
AsyncWaitOperator allows to asynchronously process incoming stream records. For that
the operator creates an ResultFuture which is passed to an AsyncFunction. Within
the async function, the user can complete the async collector arbitrarily. Once the async
collector has been completed, the result is emitted by the operator's emitter to downstream
operators.
The operator offers different output modes depending on the chosen AsyncDataStream.OutputMode. In
order to give exactly once processing guarantees, the operator stores all currently in-flight
StreamElement in it's operator state. Upon recovery the recorded set of stream elements
is replayed.
The operator also support retry on predefined condition and strategy AsyncRetryStrategy. If given strategy is NO_RETRY_STRATEGY or a custom implemented AsyncRetryPredicate which defines empty retry
predicate, then no retry will happen, everything keeps the same as the AsyncWaitOperator
without retry support.
In case of chaining of this operator, it has to be made sure that the operators in the chain
are opened tail to head. The reason for this is that an opened AsyncWaitOperator starts
already emitting recovered StreamElement to downstream operators.
userFunctionchainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService| Constructor and Description |
|---|
AsyncWaitOperator(AsyncFunction<IN,OUT> asyncFunction,
long timeout,
int capacity,
AsyncDataStream.OutputMode outputMode,
AsyncRetryStrategy<OUT> asyncRetryStrategy,
ProcessingTimeService processingTimeService,
MailboxExecutor mailboxExecutor) |
| Modifier and Type | Method and Description |
|---|---|
void |
endInput()
It is notified that no more data will arrive from the input.
|
void |
initializeState(StateInitializationContext context)
Stream operators with state which can be restored need to override this hook method.
|
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g.
|
void |
processElement(StreamRecord<IN> record)
Processes one element that arrived on this input of the
MultipleInputStreamOperator. |
void |
processWatermark(Watermark mark)
Processes a
Watermark that arrived on the first input of this two-input operator. |
void |
setup(StreamTask<?,?> containingTask,
StreamConfig config,
Output<StreamRecord<OUT>> output)
Initializes the operator.
|
void |
snapshotState(StateSnapshotContext context)
Stream operators with state, which want to participate in a snapshot need to override this
hook method.
|
close, finish, getUserFunction, getUserFunctionParameters, notifyCheckpointAborted, notifyCheckpointComplete, setOutputTypegetChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, isUsingCustomRawKeyedState, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, snapshotStateclone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitsetKeyContextElementclose, finish, getMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotStatenotifyCheckpointAborted, notifyCheckpointCompletegetCurrentKey, setCurrentKeyprocessLatencyMarker, processWatermarkStatushasKeyContextpublic AsyncWaitOperator(@Nonnull AsyncFunction<IN,OUT> asyncFunction, long timeout, int capacity, @Nonnull AsyncDataStream.OutputMode outputMode, @Nonnull AsyncRetryStrategy<OUT> asyncRetryStrategy, @Nonnull ProcessingTimeService processingTimeService, @Nonnull MailboxExecutor mailboxExecutor)
public void setup(StreamTask<?,?> containingTask, StreamConfig config, Output<StreamRecord<OUT>> output)
SetupableStreamOperatorsetup in interface SetupableStreamOperator<OUT>setup in class AbstractUdfStreamOperator<OUT,AsyncFunction<IN,OUT>>public void open()
throws Exception
AbstractStreamOperatorThe default implementation does nothing.
open in interface StreamOperator<OUT>open in class AbstractUdfStreamOperator<OUT,AsyncFunction<IN,OUT>>Exception - An exception in this method causes the operator to fail.public void processElement(StreamRecord<IN> record) throws Exception
InputMultipleInputStreamOperator.
This method is guaranteed to not be called concurrently with other methods of the operator.processElement in interface Input<IN>Exceptionpublic void processWatermark(Watermark mark) throws Exception
InputWatermark that arrived on the first input of this two-input operator.
This method is guaranteed to not be called concurrently with other methods of the operator.processWatermark in interface Input<IN>processWatermark in class AbstractStreamOperator<OUT>ExceptionWatermarkpublic void snapshotState(StateSnapshotContext context) throws Exception
AbstractStreamOperatorsnapshotState in interface StreamOperatorStateHandler.CheckpointedStreamOperatorsnapshotState in class AbstractUdfStreamOperator<OUT,AsyncFunction<IN,OUT>>context - context that provides information and means required for taking a snapshotExceptionpublic void initializeState(StateInitializationContext context) throws Exception
AbstractStreamOperatorinitializeState in interface StreamOperatorStateHandler.CheckpointedStreamOperatorinitializeState in class AbstractUdfStreamOperator<OUT,AsyncFunction<IN,OUT>>context - context that allows to register different states.Exceptionpublic void endInput()
throws Exception
BoundedOneInputWARNING: It is not safe to use this method to commit any transactions or other side
effects! You can use this method to flush any buffered data that can later on be committed
e.g. in a CheckpointListener.notifyCheckpointComplete(long).
NOTE: Given it is semantically very similar to the StreamOperator.finish()
method. It might be dropped in favour of the other method at some point in time.
endInput in interface BoundedOneInputExceptionCopyright © 2014–2024 The Apache Software Foundation. All rights reserved.