Class UpdatableTopNFunction
- java.lang.Object
-
- org.apache.flink.api.common.functions.AbstractRichFunction
-
- org.apache.flink.streaming.api.functions.KeyedProcessFunction<RowData,RowData,RowData>
-
- org.apache.flink.table.runtime.operators.rank.AbstractTopNFunction
-
- org.apache.flink.table.runtime.operators.rank.UpdatableTopNFunction
-
- All Implemented Interfaces:
Serializable
,Function
,RichFunction
,CheckpointedFunction
public class UpdatableTopNFunction extends AbstractTopNFunction implements CheckpointedFunction
A TopN function could handle updating stream. It is a fast version ofRetractableTopNFunction
which only hold top n data in state, and keep sorted map in heap. However, the function only works in some special scenarios: 1. sort field collation is ascending and its mono is decreasing, or sort field collation is descending and its mono is increasing 2. input data has unique keys and unique key must contain partition key 3. input stream could not contain DELETE record or UPDATE_BEFORE record- See Also:
- Serialized Form
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.flink.streaming.api.functions.KeyedProcessFunction
KeyedProcessFunction.Context, KeyedProcessFunction.OnTimerContext
-
-
Field Summary
-
Fields inherited from class org.apache.flink.table.runtime.operators.rank.AbstractTopNFunction
hitCount, inputRowType, keyContext, outputRankNumber, rankEnd, requestCount, sortKeyComparator, sortKeySelector, ttlConfig
-
-
Constructor Summary
Constructors Constructor Description UpdatableTopNFunction(StateTtlConfig ttlConfig, InternalTypeInfo<RowData> inputRowType, RowDataKeySelector rowKeySelector, GeneratedRecordComparator generatedRecordComparator, RowDataKeySelector sortKeySelector, RankType rankType, RankRange rankRange, boolean generateUpdateBefore, boolean outputRankNumber, long cacheSize)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
initializeState(FunctionInitializationContext context)
This method is called when the parallel function instance is created during distributed execution.void
open(OpenContext openContext)
Initialization method for the function.void
processElement(RowData input, KeyedProcessFunction.Context context, Collector<RowData> out)
Process one element from the input stream.void
snapshotState(FunctionSnapshotContext context)
This method is called when a snapshot for a checkpoint is requested.-
Methods inherited from class org.apache.flink.table.runtime.operators.rank.AbstractTopNFunction
checkSortKeyInBufferRange, collectDelete, collectDelete, collectInsert, collectInsert, collectUpdateAfter, collectUpdateAfter, collectUpdateBefore, collectUpdateBefore, getDefaultTopNSize, hasOffset, initRankEnd, isInRankEnd, isInRankRange, registerMetric, setKeyContext
-
Methods inherited from class org.apache.flink.streaming.api.functions.KeyedProcessFunction
onTimer
-
Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction
close, getIterationRuntimeContext, getRuntimeContext, setRuntimeContext
-
-
-
-
Constructor Detail
-
UpdatableTopNFunction
public UpdatableTopNFunction(StateTtlConfig ttlConfig, InternalTypeInfo<RowData> inputRowType, RowDataKeySelector rowKeySelector, GeneratedRecordComparator generatedRecordComparator, RowDataKeySelector sortKeySelector, RankType rankType, RankRange rankRange, boolean generateUpdateBefore, boolean outputRankNumber, long cacheSize)
-
-
Method Detail
-
open
public void open(OpenContext openContext) throws Exception
Description copied from interface:RichFunction
Initialization method for the function. It is called before the actual working methods (like map or join) and thus suitable for one time setup work. For functions that are part of an iteration, this method will be invoked at the beginning of each iteration superstep.The openContext object passed to the function can be used for configuration and initialization. The openContext contains some necessary information that were configured on the function in the program composition.
public class MyFilter extends RichFilterFunction<String> { private String searchString; public void open(OpenContext openContext) { // initialize the value of searchString } public boolean filter(String value) { return value.equals(searchString); } }
- Specified by:
open
in interfaceRichFunction
- Overrides:
open
in classAbstractTopNFunction
- Parameters:
openContext
- The context containing information about the context in which the function is opened.- Throws:
Exception
- Implementations may forward exceptions, which are caught by the runtime. When the runtime catches an exception, it aborts the task and lets the fail-over logic decide whether to retry the task execution.
-
initializeState
public void initializeState(FunctionInitializationContext context) throws Exception
Description copied from interface:CheckpointedFunction
This method is called when the parallel function instance is created during distributed execution. Functions typically set up their state storing data structures in this method.- Specified by:
initializeState
in interfaceCheckpointedFunction
- Parameters:
context
- the context for initializing the operator- Throws:
Exception
- Thrown, if state could not be created ot restored.
-
processElement
public void processElement(RowData input, KeyedProcessFunction.Context context, Collector<RowData> out) throws Exception
Description copied from class:KeyedProcessFunction
Process one element from the input stream.This function can output zero or more elements using the
Collector
parameter and also update internal state or set timers using theKeyedProcessFunction.Context
parameter.- Specified by:
processElement
in classKeyedProcessFunction<RowData,RowData,RowData>
- Parameters:
input
- The input value.context
- AKeyedProcessFunction.Context
that allows querying the timestamp of the element and getting aTimerService
for registering timers and querying the time. The context is only valid during the invocation of this method, do not store it.out
- The collector for returning result values.- Throws:
Exception
- This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.
-
snapshotState
public void snapshotState(FunctionSnapshotContext context) throws Exception
Description copied from interface:CheckpointedFunction
This method is called when a snapshot for a checkpoint is requested. This acts as a hook to the function to ensure that all state is exposed by means previously offered throughFunctionInitializationContext
when the Function was initialized, or offered now byFunctionSnapshotContext
itself.- Specified by:
snapshotState
in interfaceCheckpointedFunction
- Parameters:
context
- the context for drawing a snapshot of the operator- Throws:
Exception
- Thrown, if state could not be created ot restored.
-
-