Package org.apache.flink.table.runtime.operators.join.temporal

Class BaseTwoInputStreamOperatorWithStateRetention

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<RowData>

org.apache.flink.table.runtime.operators.join.temporal.BaseTwoInputStreamOperatorWithStateRetention

All Implemented Interfaces:

Serializable, CheckpointListener, KeyContext, KeyContextHandler, StreamOperator<RowData>, StreamOperatorStateHandler.CheckpointedStreamOperator, Triggerable<Object,VoidNamespace>, TwoInputStreamOperator<RowData,RowData,RowData>, YieldingOperator<RowData>

Direct Known Subclasses:

TemporalProcessTimeJoinOperator, TemporalRowTimeJoinOperator

@Internal
public abstract class BaseTwoInputStreamOperatorWithStateRetention
extends AbstractStreamOperator<RowData>
implements TwoInputStreamOperator<RowData,RowData,RowData>, Triggerable<Object,VoidNamespace>

An abstract TwoInputStreamOperator that allows its subclasses to clean up their state based on a TTL. This TTL should be specified in the provided minRetentionTime and maxRetentionTime.

For each known key, this operator registers a timer (in processing time) to fire after the TTL expires. When the timer fires, the subclass can decide which state to cleanup and what further action to take.

This class takes care of maintaining at most one timer per key.

IMPORTANT NOTE TO USERS: When extending this class, do not use processing time timers in your business logic. The reason is that:

1) if your timers collide with clean up timers and you delete them, then state clean-up will not be performed, and

2) (this one is the reason why this class does not allow to override the onProcessingTime()) the onProcessingTime with your logic would be also executed on each clean up timer.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
protected boolean	stateCleaningEnabled

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

combinedWatermark, config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	BaseTwoInputStreamOperatorWithStateRetention(long minRetentionTime, long maxRetentionTime)

Method Summary

Modifier and Type	Method	Description
protected void	cleanupLastTimer()
abstract void	cleanupState(long time)	The method to be called when a cleanup timer fires.
void	onProcessingTime(InternalTimer<Object,VoidNamespace> timer)	The users of this class are not allowed to use processing time timers.
void	open()	This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.
protected void	registerProcessingCleanupTimer()	If the user has specified a minRetentionTime and maxRetentionTime, this method registers a cleanup timer for currentProcessingTime + minRetentionTime.
boolean	useSplittableTimers()	Can be overridden to disable splittable timers for this particular operator even if config option is enabled.

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

close, finish, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isAsyncStateProcessingEnabled, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

close, finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.streaming.api.operators.Triggerable

onEventTime

Methods inherited from interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator

processElement1, processElement2, processLatencyMarker1, processLatencyMarker2, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark2, processWatermarkStatus1, processWatermarkStatus2

Field Detail

stateCleaningEnabled

protected final boolean stateCleaningEnabled

Constructor Detail

BaseTwoInputStreamOperatorWithStateRetention

protected BaseTwoInputStreamOperatorWithStateRetention(long minRetentionTime,
                                                       long maxRetentionTime)

Method Detail

useSplittableTimers

public boolean useSplittableTimers()

Description copied from class: AbstractStreamOperator

Can be overridden to disable splittable timers for this particular operator even if config option is enabled. By default, splittable timers are disabled.

Overrides:

useSplittableTimers in class AbstractStreamOperator<RowData>

Returns:

true if splittable timers should be used (subject to StreamConfig.isUnalignedCheckpointsEnabled() and StreamConfig.isUnalignedCheckpointsSplittableTimersEnabled(). false if splittable timers should never be used.

open

public void open()
          throws Exception

Description copied from class: AbstractStreamOperator

This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.

The default implementation does nothing.

Specified by:

open in interface StreamOperator<RowData>

Overrides:

open in class AbstractStreamOperator<RowData>

Throws:

Exception - An exception in this method causes the operator to fail.

registerProcessingCleanupTimer

protected void registerProcessingCleanupTimer()
                                       throws IOException

If the user has specified a minRetentionTime and maxRetentionTime, this method registers a cleanup timer for currentProcessingTime + minRetentionTime.

When this timer fires, the cleanupState(long) method is called.

Throws:

IOException

cleanupLastTimer

protected void cleanupLastTimer()
                         throws IOException

Throws:

IOException

onProcessingTime

public final void onProcessingTime(InternalTimer<Object,VoidNamespace> timer)
                            throws Exception

The users of this class are not allowed to use processing time timers. See class javadoc.

Specified by:

onProcessingTime in interface Triggerable<Object,VoidNamespace>

Throws:

Exception

cleanupState

public abstract void cleanupState(long time)

The method to be called when a cleanup timer fires.

Parameters:

time - The timestamp of the fired timer.