org.apache.flink.cep.operator

Class AbstractKeyedCEPPatternOperator<IN,KEY,OUT,F extends Function>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OUT>

org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator<OUT,F>

org.apache.flink.cep.operator.AbstractKeyedCEPPatternOperator<IN,KEY,OUT,F>

Type Parameters:

IN - Type of the input elements

KEY - Type of the key on which the input stream is keyed

OUT - Type of the output elements

All Implemented Interfaces:

Serializable, CheckpointListener, KeyContext, OneInputStreamOperator<IN,OUT>, OutputTypeConfigurable<OUT>, StreamOperator<OUT>, Triggerable<KEY,VoidNamespace>, Disposable

Direct Known Subclasses:

FlatSelectCepOperator, FlatSelectTimeoutCepOperator, SelectCepOperator, SelectTimeoutCepOperator

public abstract class AbstractKeyedCEPPatternOperator<IN,KEY,OUT,F extends Function>
extends AbstractUdfStreamOperator<OUT,F>
implements OneInputStreamOperator<IN,OUT>, Triggerable<KEY,VoidNamespace>

Abstract CEP pattern operator for a keyed input stream. For each key, the operator creates a NFA and a priority queue to buffer out of order elements. Both data structures are stored using the managed keyed state. Additionally, the set of all seen keys is kept as part of the operator state. This is necessary to trigger the execution for all keys upon receiving a new watermark.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

AbstractStreamOperator.CountingOutput

Field Summary

Fields

Modifier and Type	Field and Description
protected AfterMatchSkipStrategy	afterMatchSkipStrategy

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator

userFunction

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, latencyStats, LOG, metrics, output, timeServiceManager

Constructor Summary

Constructors

Constructor and Description
AbstractKeyedCEPPatternOperator(TypeSerializer<IN> inputSerializer, boolean isProcessingTime, NFACompiler.NFAFactory<IN> nfaFactory, EventComparator<IN> comparator, AfterMatchSkipStrategy afterMatchSkipStrategy, F function)

Method Summary

Modifier and Type	Method and Description
int	getPQSize(KEY key)
boolean	hasNonEmptyNFA(KEY key)
boolean	hasNonEmptyPQ(KEY key)
void	initializeState(StateInitializationContext context) Stream operators with state which can be restored need to override this hook method.
void	onEventTime(InternalTimer<KEY,VoidNamespace> timer) Invoked when an event-time timer fires.
void	onProcessingTime(InternalTimer<KEY,VoidNamespace> timer) Invoked when a processing-time timer fires.
void	open() This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g.
void	processElement(StreamRecord<IN> element) Processes one element that arrived at this operator.
protected abstract void	processMatchedSequences(Iterable<Map<String,List<IN>>> matchingSequences, long timestamp)
protected void	processTimedOutSequences(Iterable<Tuple2<Map<String,List<IN>>,Long>> timedOutSequences, long timestamp)

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator

close, dispose, getUserFunction, getUserFunctionParameters, notifyCheckpointComplete, setOutputType, setup, snapshotState

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getUserCodeClassloader, initializeState, numEventTimeTimers, numProcessingTimeTimers, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processWatermark, processWatermark1, processWatermark2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

processLatencyMarker, processWatermark

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

close, dispose, getChainingStrategy, getMetricGroup, getOperatorID, initializeState, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, setup, snapshotState

Methods inherited from interface org.apache.flink.runtime.state.CheckpointListener

notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Field Detail

afterMatchSkipStrategy

protected final AfterMatchSkipStrategy afterMatchSkipStrategy

Constructor Detail

AbstractKeyedCEPPatternOperator

public AbstractKeyedCEPPatternOperator(TypeSerializer<IN> inputSerializer,
                                       boolean isProcessingTime,
                                       NFACompiler.NFAFactory<IN> nfaFactory,
                                       EventComparator<IN> comparator,
                                       AfterMatchSkipStrategy afterMatchSkipStrategy,
                                       F function)

Method Detail

initializeState

public void initializeState(StateInitializationContext context)
                     throws Exception

Description copied from class: AbstractStreamOperator

Stream operators with state which can be restored need to override this hook method.

Overrides:

initializeState in class AbstractUdfStreamOperator<OUT,F extends Function>

Parameters:

context - context that allows to register different states.

Throws:

Exception

open

public void open()
          throws Exception

Description copied from class: AbstractStreamOperator

This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.

The default implementation does nothing.

Specified by:

open in interface StreamOperator<OUT>

Overrides:

open in class AbstractUdfStreamOperator<OUT,F extends Function>

Throws:

Exception - An exception in this method causes the operator to fail.

processElement

public void processElement(StreamRecord<IN> element)
                    throws Exception

Description copied from interface: OneInputStreamOperator

Processes one element that arrived at this operator. This method is guaranteed to not be called concurrently with other methods of the operator.

Specified by:

processElement in interface OneInputStreamOperator<IN,OUT>

Throws:

Exception

onEventTime

public void onEventTime(InternalTimer<KEY,VoidNamespace> timer)
                 throws Exception

Description copied from interface: Triggerable

Invoked when an event-time timer fires.

Specified by:

onEventTime in interface Triggerable<KEY,VoidNamespace>

Throws:

Exception

onProcessingTime

public void onProcessingTime(InternalTimer<KEY,VoidNamespace> timer)
                      throws Exception

Description copied from interface: Triggerable

Invoked when a processing-time timer fires.

Specified by:

onProcessingTime in interface Triggerable<KEY,VoidNamespace>

Throws:

Exception

processMatchedSequences

protected abstract void processMatchedSequences(Iterable<Map<String,List<IN>>> matchingSequences,
                                                long timestamp)
                                         throws Exception

Throws:

Exception

processTimedOutSequences

protected void processTimedOutSequences(Iterable<Tuple2<Map<String,List<IN>>,Long>> timedOutSequences,
                                        long timestamp)
                                 throws Exception

Throws:

Exception

hasNonEmptyNFA

@VisibleForTesting
public boolean hasNonEmptyNFA(KEY key)
                                          throws IOException

Throws:

IOException

hasNonEmptyPQ

@VisibleForTesting
public boolean hasNonEmptyPQ(KEY key)
                                         throws Exception

Throws:

Exception

getPQSize

@VisibleForTesting
public int getPQSize(KEY key)
                                 throws Exception

Throws:

Exception