org.apache.flink.table.runtime.operators.aggregate.window.processors

Class AbstractWindowAggProcessor

java.lang.Object

org.apache.flink.table.runtime.operators.aggregate.window.processors.AbstractWindowAggProcessor

All Implemented Interfaces:

Serializable, SlicingWindowProcessor<Long>

Direct Known Subclasses:

SliceSharedWindowAggProcessor, SliceUnsharedWindowAggProcessor

public abstract class AbstractWindowAggProcessor
extends Object
implements SlicingWindowProcessor<Long>

A base implementation of SlicingWindowProcessor for window aggregate.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.table.runtime.operators.window.slicing.SlicingWindowProcessor

SlicingWindowProcessor.Context<W>

Field Summary

Fields

Modifier and Type	Field and Description
protected TypeSerializer<RowData>	accSerializer
protected NamespaceAggsHandleFunction<Long>	aggregator
protected ClockService	clockService
protected SlicingWindowProcessor.Context<Long>	ctx
protected long	currentProgress
protected GeneratedNamespaceAggsHandleFunction<Long>	genAggsHandler
protected boolean	isEventTime
protected JoinedRowData	reuseOutput
protected java.time.ZoneId	shiftTimeZone
protected SliceAssigner	sliceAssigner
protected boolean	useDayLightSaving The shift timezone is using DayLightSaving time or not.
protected WindowBuffer	windowBuffer
protected WindowBuffer.Factory	windowBufferFactory
protected long	windowInterval
protected WindowValueState<Long>	windowState state schema: [key, window_end, accumulator].
protected WindowTimerService<Long>	windowTimerService

Constructor Summary

Constructors

Constructor and Description
AbstractWindowAggProcessor(GeneratedNamespaceAggsHandleFunction<Long> genAggsHandler, WindowBuffer.Factory bufferFactory, SliceAssigner sliceAssigner, TypeSerializer<RowData> accSerializer, java.time.ZoneId shiftTimeZone)

Method Summary

Modifier and Type	Method and Description
void	advanceProgress(long progress) Advances the progress time, the progress time is watermark if working in event-time mode, or current processing time if working in processing-time mode.
void	clearWindow(Long windowEnd) Clear state and resources associated with the given window namespace.
void	close() The tear-down method of the function.
protected void	collect(RowData aggResult)
TypeSerializer<Long>	createWindowSerializer() Returns the serializer of the window type.
void	initializeWatermark(long watermark) Initializes the watermark which restores from state.
void	open(SlicingWindowProcessor.Context<Long> context) Initialization method for the function.
void	prepareCheckpoint() Performs a preparation before checkpoint.
boolean	processElement(RowData key, RowData element) Process an element with associated key from the input stream.
protected abstract long	sliceStateMergeTarget(long sliceToMerge) Returns the slice state target to merge the given slice into when firing windows.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.runtime.operators.window.slicing.SlicingWindowProcessor

fireWindow

Field Detail

genAggsHandler

protected final GeneratedNamespaceAggsHandleFunction<Long> genAggsHandler

windowBufferFactory

protected final WindowBuffer.Factory windowBufferFactory

sliceAssigner

protected final SliceAssigner sliceAssigner

accSerializer

protected final TypeSerializer<RowData> accSerializer

isEventTime

protected final boolean isEventTime

windowInterval

protected final long windowInterval

shiftTimeZone

protected final java.time.ZoneId shiftTimeZone

useDayLightSaving

protected final boolean useDayLightSaving

The shift timezone is using DayLightSaving time or not.

currentProgress

protected transient long currentProgress

ctx

protected transient SlicingWindowProcessor.Context<Long> ctx

clockService

protected transient ClockService clockService

windowTimerService

protected transient WindowTimerService<Long> windowTimerService

aggregator

protected transient NamespaceAggsHandleFunction<Long> aggregator

windowBuffer

protected transient WindowBuffer windowBuffer

windowState

protected transient WindowValueState<Long> windowState

state schema: [key, window_end, accumulator].

reuseOutput

protected transient JoinedRowData reuseOutput

Constructor Detail

AbstractWindowAggProcessor

public AbstractWindowAggProcessor(GeneratedNamespaceAggsHandleFunction<Long> genAggsHandler,
                                  WindowBuffer.Factory bufferFactory,
                                  SliceAssigner sliceAssigner,
                                  TypeSerializer<RowData> accSerializer,
                                  java.time.ZoneId shiftTimeZone)

Method Detail

open

public void open(SlicingWindowProcessor.Context<Long> context)
          throws Exception

Description copied from interface: SlicingWindowProcessor

Initialization method for the function. It is called before the actual working methods.

Specified by:

open in interface SlicingWindowProcessor<Long>

Throws:

Exception

initializeWatermark

public void initializeWatermark(long watermark)

Description copied from interface: SlicingWindowProcessor

Initializes the watermark which restores from state. The method is called after open method and before the actual working methods.

Specified by:

initializeWatermark in interface SlicingWindowProcessor<Long>

Parameters:

watermark - the initial watermark

processElement

public boolean processElement(RowData key,
                              RowData element)
                       throws Exception

Description copied from interface: SlicingWindowProcessor

Process an element with associated key from the input stream. Returns true if this element is dropped because of late arrival.

Specified by:

processElement in interface SlicingWindowProcessor<Long>

Parameters:

key - the key associated with the element

element - The element to process.

Throws:

Exception

sliceStateMergeTarget

protected abstract long sliceStateMergeTarget(long sliceToMerge)
                                       throws Exception

Returns the slice state target to merge the given slice into when firing windows. For unshared windows, there should no merging happens, so the merge target should be just the given sliceToMerge. For shared windows, the merge target should be the shared slice state.

Throws:

Exception

See Also:

SliceSharedAssigner.mergeSlices(long, SliceSharedAssigner.MergeCallback)

advanceProgress

public void advanceProgress(long progress)
                     throws Exception

Description copied from interface: SlicingWindowProcessor

Advances the progress time, the progress time is watermark if working in event-time mode, or current processing time if working in processing-time mode.

This will potentially flush buffered data into states, because the watermark advancement may be in a very small step, but we don't need to flush buffered data for every watermark advancement.

Specified by:

advanceProgress in interface SlicingWindowProcessor<Long>

Parameters:

progress - the current progress time

Throws:

Exception

prepareCheckpoint

public void prepareCheckpoint()
                       throws Exception

Description copied from interface: SlicingWindowProcessor

Performs a preparation before checkpoint. This usually flushes buffered data into state.

Specified by:

prepareCheckpoint in interface SlicingWindowProcessor<Long>

Throws:

Exception

clearWindow

public void clearWindow(Long windowEnd)
                 throws Exception

Description copied from interface: SlicingWindowProcessor

Clear state and resources associated with the given window namespace.

Note: the key context has been set.

Specified by:

clearWindow in interface SlicingWindowProcessor<Long>

Parameters:

windowEnd - the window to clear

Throws:

Exception

close

public void close()
           throws Exception

Description copied from interface: SlicingWindowProcessor

The tear-down method of the function. It is called after the last call to the main working methods.

Specified by:

close in interface SlicingWindowProcessor<Long>

Throws:

Exception

createWindowSerializer

public TypeSerializer<Long> createWindowSerializer()

Description copied from interface: SlicingWindowProcessor

Returns the serializer of the window type.

Specified by:

createWindowSerializer in interface SlicingWindowProcessor<Long>

collect

protected void collect(RowData aggResult)