Class StreamingSemiAntiJoinOperator
- java.lang.Object
-
- org.apache.flink.streaming.api.operators.AbstractStreamOperator<RowData>
-
- org.apache.flink.table.runtime.operators.join.stream.AbstractStreamingJoinOperator
-
- org.apache.flink.table.runtime.operators.join.stream.StreamingSemiAntiJoinOperator
-
- All Implemented Interfaces:
Serializable
,CheckpointListener
,KeyContext
,KeyContextHandler
,StreamOperator<RowData>
,StreamOperatorStateHandler.CheckpointedStreamOperator
,TwoInputStreamOperator<RowData,RowData,RowData>
,YieldingOperator<RowData>
public class StreamingSemiAntiJoinOperator extends AbstractStreamingJoinOperator
Streaming unbounded Join operator which supports SEMI/ANTI JOIN.- See Also:
- Serialized Form
-
-
Field Summary
-
Fields inherited from class org.apache.flink.table.runtime.operators.join.stream.AbstractStreamingJoinOperator
collector, joinCondition, LEFT_RECORDS_STATE_NAME, leftInputSideSpec, leftStateRetentionTime, leftType, RIGHT_RECORDS_STATE_NAME, rightInputSideSpec, rightStateRetentionTime, rightType
-
Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator
config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager
-
-
Constructor Summary
Constructors Constructor Description StreamingSemiAntiJoinOperator(boolean isAntiJoin, InternalTypeInfo<RowData> leftType, InternalTypeInfo<RowData> rightType, GeneratedJoinCondition generatedJoinCondition, JoinInputSideSpec leftInputSideSpec, JoinInputSideSpec rightInputSideSpec, boolean[] filterNullKeys, long leftStateRetentionTime, long rightStateRetentionTIme)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
open()
This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.void
processElement1(StreamRecord<RowData> element)
Process an input element and output incremental joined records, retraction messages will be sent in some scenarios.void
processElement2(StreamRecord<RowData> element)
Process an input element and output incremental joined records, retraction messages will be sent in some scenarios.-
Methods inherited from class org.apache.flink.table.runtime.operators.join.stream.AbstractStreamingJoinOperator
close
-
Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator
finish, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState, useSplittableTimers
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener
notifyCheckpointAborted, notifyCheckpointComplete
-
Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext
getCurrentKey, setCurrentKey
-
Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler
hasKeyContext
-
Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator
finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState
-
Methods inherited from interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator
processLatencyMarker1, processLatencyMarker2, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark2, processWatermarkStatus1, processWatermarkStatus2
-
-
-
-
Constructor Detail
-
StreamingSemiAntiJoinOperator
public StreamingSemiAntiJoinOperator(boolean isAntiJoin, InternalTypeInfo<RowData> leftType, InternalTypeInfo<RowData> rightType, GeneratedJoinCondition generatedJoinCondition, JoinInputSideSpec leftInputSideSpec, JoinInputSideSpec rightInputSideSpec, boolean[] filterNullKeys, long leftStateRetentionTime, long rightStateRetentionTIme)
-
-
Method Detail
-
open
public void open() throws Exception
Description copied from class:AbstractStreamOperator
This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.The default implementation does nothing.
- Specified by:
open
in interfaceStreamOperator<RowData>
- Overrides:
open
in classAbstractStreamingJoinOperator
- Throws:
Exception
- An exception in this method causes the operator to fail.
-
processElement1
public void processElement1(StreamRecord<RowData> element) throws Exception
Process an input element and output incremental joined records, retraction messages will be sent in some scenarios.Following is the pseudo code to describe the core logic of this method.
if there is no matched rows on the other side if anti join, send input record if there are matched rows on the other side if semi join, send input record if the input record is accumulate, state.add(record, matched size) if the input record is retract, state.retract(record)
- Throws:
Exception
-
processElement2
public void processElement2(StreamRecord<RowData> element) throws Exception
Process an input element and output incremental joined records, retraction messages will be sent in some scenarios.Following is the pseudo code to describe the core logic of this method.
Note: "+I" represents "INSERT", "-D" represents "DELETE", "+U" represents "UPDATE_AFTER", "-U" represents "UPDATE_BEFORE".
if input record is accumulate | state.add(record) | if there is no matched rows on the other side, skip | if there are matched rows on the other side | | if the matched num in the matched rows == 0 | | if anti join, send -D[other]s | | if semi join, send +I/+U[other]s (using input RowKind) | | if the matched num in the matched rows > 0, skip | | otherState.update(other, old+1) | endif endif if input record is retract | state.retract(record) | if there is no matched rows on the other side, skip | if there are matched rows on the other side | | if the matched num in the matched rows == 0, this should never happen! | | if the matched num in the matched rows == 1 | | if semi join, send -D/-U[other] (using input RowKind) | | if anti join, send +I[other] | | if the matched num in the matched rows > 1, skip | | otherState.update(other, old-1) | endif endif
- Throws:
Exception
-
-