K - The type of key returned by the KeySelector.IN - The type of the incoming elements.OUT - The type of elements emitted by the InternalWindowFunction.W - The type of Window that the WindowAssigner assigns.@Internal public class EvictingWindowOperator<K,IN,OUT,W extends Window> extends WindowOperator<K,IN,Iterable<IN>,OUT,W>
WindowOperator that also allows an Evictor to be used.
The Evictor is used to remove elements from a pane before/after the evaluation of
InternalWindowFunction and after the window evaluation gets triggered by a Trigger.
WindowOperator.AbstractPerWindowStateStore, WindowOperator.Context, WindowOperator.MergingWindowStateStore, WindowOperator.PerWindowStateStore, WindowOperator.Timer<K,W extends Window>, WindowOperator.WindowContextallowedLateness, internalTimerService, keySerializer, lateDataOutputTag, numLateRecordsDropped, processContext, timestampedCollector, triggerContext, windowAssigner, windowAssignerContext, windowSerializeruserFunctionchainingStrategy, config, latencyStats, LOG, metrics, output, processingTimeService| Constructor and Description |
|---|
EvictingWindowOperator(WindowAssigner<? super IN,W> windowAssigner,
TypeSerializer<W> windowSerializer,
KeySelector<IN,K> keySelector,
TypeSerializer<K> keySerializer,
StateDescriptor<? extends ListState<StreamRecord<IN>>,?> windowStateDescriptor,
InternalWindowFunction<Iterable<IN>,OUT,K,W> windowFunction,
Trigger<? super IN,? super W> trigger,
Evictor<? super IN,? super W> evictor,
long allowedLateness,
OutputTag<IN> lateDataOutputTag) |
| Modifier and Type | Method and Description |
|---|---|
void |
close()
This method is called after all records have been added to the operators via the methods
OneInputStreamOperator.processElement(StreamRecord), or TwoInputStreamOperator.processElement1(StreamRecord) and TwoInputStreamOperator.processElement2(StreamRecord). |
void |
dispose()
This method is called at the very end of the operator's life, both in the case of a
successful completion of the operation, and in the case of a failure and canceling.
|
Evictor<? super IN,? super W> |
getEvictor() |
StateDescriptor<? extends AppendingState<IN,Iterable<IN>>,?> |
getStateDescriptor() |
void |
onEventTime(InternalTimer<K,W> timer)
Invoked when an event-time timer fires.
|
void |
onProcessingTime(InternalTimer<K,W> timer)
Invoked when a processing-time timer fires.
|
void |
open()
This method is called immediately before any elements are processed, it should contain the
operator's initialization logic, e.g.
|
void |
processElement(StreamRecord<IN> element)
Processes one element that arrived at this operator.
|
deleteCleanupTimer, getKeySelector, getMergingWindowSet, getTrigger, getWindowAssigner, isCleanupTime, isElementLate, isWindowLate, registerCleanupTimer, sideOutputgetUserFunction, getUserFunctionParameters, initializeState, notifyCheckpointAborted, notifyCheckpointComplete, setOutputType, setup, snapshotStategetChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, isUsingCustomRawKeyedState, numEventTimeTimers, numProcessingTimeTimers, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, snapshotStateclone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitprocessLatencyMarker, processWatermarkgetMetricGroup, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotStatenotifyCheckpointAborted, notifyCheckpointCompletegetCurrentKey, setCurrentKeypublic EvictingWindowOperator(WindowAssigner<? super IN,W> windowAssigner, TypeSerializer<W> windowSerializer, KeySelector<IN,K> keySelector, TypeSerializer<K> keySerializer, StateDescriptor<? extends ListState<StreamRecord<IN>>,?> windowStateDescriptor, InternalWindowFunction<Iterable<IN>,OUT,K,W> windowFunction, Trigger<? super IN,? super W> trigger, Evictor<? super IN,? super W> evictor, long allowedLateness, OutputTag<IN> lateDataOutputTag)
public void processElement(StreamRecord<IN> element) throws Exception
OneInputStreamOperatorprocessElement in interface OneInputStreamOperator<IN,OUT>processElement in class WindowOperator<K,IN,Iterable<IN>,OUT,W extends Window>Exceptionpublic void onEventTime(InternalTimer<K,W> timer) throws Exception
TriggerableonEventTime in interface Triggerable<K,W extends Window>onEventTime in class WindowOperator<K,IN,Iterable<IN>,OUT,W extends Window>Exceptionpublic void onProcessingTime(InternalTimer<K,W> timer) throws Exception
TriggerableonProcessingTime in interface Triggerable<K,W extends Window>onProcessingTime in class WindowOperator<K,IN,Iterable<IN>,OUT,W extends Window>Exceptionpublic void open()
throws Exception
AbstractStreamOperatorThe default implementation does nothing.
public void close()
throws Exception
AbstractStreamOperatorOneInputStreamOperator.processElement(StreamRecord), or TwoInputStreamOperator.processElement1(StreamRecord) and TwoInputStreamOperator.processElement2(StreamRecord).
The method is expected to flush all remaining buffered data. Exceptions during this flushing of buffered should be propagated, in order to cause the operation to be recognized asa failed, because the last data items are not processed properly.
public void dispose()
throws Exception
AbstractStreamOperatorThis method is expected to make a thorough effort to release all resources that the operator has acquired.
dispose in interface StreamOperator<OUT>dispose in interface Disposabledispose in class WindowOperator<K,IN,Iterable<IN>,OUT,W extends Window>Exception - if something goes wrong during disposal.@VisibleForTesting public Evictor<? super IN,? super W> getEvictor()
@VisibleForTesting public StateDescriptor<? extends AppendingState<IN,Iterable<IN>>,?> getStateDescriptor()
getStateDescriptor in class WindowOperator<K,IN,Iterable<IN>,OUT,W extends Window>Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.