Package org.apache.flink.table.runtime.operators.wmassigners

Class ProcTimeMiniBatchAssignerOperator

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<RowData>

org.apache.flink.table.runtime.operators.wmassigners.ProcTimeMiniBatchAssignerOperator

All Implemented Interfaces:

Serializable, ProcessingTimeService.ProcessingTimeCallback, CheckpointListener, Input<RowData>, KeyContext, KeyContextHandler, OneInputStreamOperator<RowData,RowData>, StreamOperator<RowData>, StreamOperatorStateHandler.CheckpointedStreamOperator, YieldingOperator<RowData>

public class ProcTimeMiniBatchAssignerOperator
extends AbstractStreamOperator<RowData>
implements OneInputStreamOperator<RowData,RowData>, ProcessingTimeService.ProcessingTimeCallback

A stream operator that emits mini-batch marker in a given period. This mini-batch assigner works in processing time, which means the mini-batch marker is generated in the given period using the processing time. The downstream operators will trigger mini-batch once the received mini-batch id advanced.

NOTE: currently, we use Watermark to represents the mini-batch marker.

The difference between this operator and RowTimeMiniBatchAssginerOperator is that, this operator generates watermarks by itself using processing time, but the other forwards watermarks from upstream.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

config, lastRecordAttributes1, lastRecordAttributes2, latencyStats, LOG, metrics, output, processingTimeService, stateHandler, stateKeySelector1, stateKeySelector2, timeServiceManager

Constructor Summary

Constructors

Constructor	Description
ProcTimeMiniBatchAssignerOperator(long intervalMs)

Method Summary

Modifier and Type	Method	Description
void	onProcessingTime(long timestamp)	This method is invoked with the time which the callback register for.
void	open()	This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.
void	processElement(StreamRecord<RowData> element)	Processes one element that arrived on this input of the MultipleInputStreamOperator.
void	processWatermark(Watermark mark)	Override the base implementation to completely ignore watermarks propagated from upstream (we rely only on the WatermarkStrategyWithPeriodicWatermarks to emit watermarks from here).

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

close, finish, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getStateKeySelector1, getStateKeySelector2, getTimeServiceManager, getUserCodeClassloader, hasKeyContext1, hasKeyContext2, initializeState, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, notifyCheckpointComplete, prepareSnapshotPreBarrier, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, processRecordAttributes, processRecordAttributes1, processRecordAttributes2, processWatermark1, processWatermark2, processWatermarkStatus, processWatermarkStatus1, processWatermarkStatus2, reportOrForwardLatencyMarker, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setMailboxExecutor, setProcessingTimeService, setup, snapshotState, snapshotState, useSplittableTimers

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.state.CheckpointListener

notifyCheckpointAborted, notifyCheckpointComplete

Methods inherited from interface org.apache.flink.streaming.api.operators.Input

processLatencyMarker, processRecordAttributes, processWatermarkStatus

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContextHandler

hasKeyContext

Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

setKeyContextElement

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

close, finish, getMetricGroup, getOperatorAttributes, getOperatorID, initializeState, prepareSnapshotPreBarrier, setKeyContextElement1, setKeyContextElement2, snapshotState

Constructor Detail

ProcTimeMiniBatchAssignerOperator

public ProcTimeMiniBatchAssignerOperator(long intervalMs)

Method Detail

open

public void open()
          throws Exception

Description copied from class: AbstractStreamOperator

This method is called immediately before any elements are processed, it should contain the operator's initialization logic, e.g. state initialization.

The default implementation does nothing.

Specified by:

open in interface StreamOperator<RowData>

Overrides:

open in class AbstractStreamOperator<RowData>

Throws:

Exception - An exception in this method causes the operator to fail.

processElement

public void processElement(StreamRecord<RowData> element)
                    throws Exception

Description copied from interface: Input

Processes one element that arrived on this input of the MultipleInputStreamOperator. This method is guaranteed to not be called concurrently with other methods of the operator.

Specified by:

processElement in interface Input<RowData>

Throws:

Exception

onProcessingTime

public void onProcessingTime(long timestamp)
                      throws Exception

Description copied from interface: ProcessingTimeService.ProcessingTimeCallback

This method is invoked with the time which the callback register for.

Specified by:

onProcessingTime in interface ProcessingTimeService.ProcessingTimeCallback

Parameters:

timestamp - The time this callback was registered for.

Throws:

Exception

processWatermark

public void processWatermark(Watermark mark)
                      throws Exception

Override the base implementation to completely ignore watermarks propagated from upstream (we rely only on the WatermarkStrategyWithPeriodicWatermarks to emit watermarks from here).

Specified by:

processWatermark in interface Input<RowData>

Overrides:

processWatermark in class AbstractStreamOperator<RowData>

Throws:

Exception

See Also:

Watermark