public class UpdatableTopNFunction extends AbstractTopNFunction implements CheckpointedFunction
RetractableTopNFunction which only hold top n data in state, and keep sorted map in heap.
However, the function only works in some special scenarios: 1. sort field collation is ascending
and its mono is decreasing, or sort field collation is descending and its mono is increasing 2.
input data has unique keys and unique key must contain partition key 3. input stream could not
contain DELETE record or UPDATE_BEFORE recordKeyedProcessFunction.Context, KeyedProcessFunction.OnTimerContexthitCount, inputRowType, keyContext, outputRankNumber, rankEnd, requestCount, sortKeyComparator, sortKeySelectorstateCleaningEnabled| Constructor and Description |
|---|
UpdatableTopNFunction(long minRetentionTime,
long maxRetentionTime,
InternalTypeInfo<RowData> inputRowType,
RowDataKeySelector rowKeySelector,
GeneratedRecordComparator generatedRecordComparator,
RowDataKeySelector sortKeySelector,
RankType rankType,
RankRange rankRange,
boolean generateUpdateBefore,
boolean outputRankNumber,
long cacheSize) |
| Modifier and Type | Method and Description |
|---|---|
void |
initializeState(FunctionInitializationContext context)
This method is called when the parallel function instance is created during distributed
execution.
|
void |
onTimer(long timestamp,
KeyedProcessFunction.OnTimerContext ctx,
Collector<RowData> out)
Called when a timer set using
TimerService fires. |
void |
open(Configuration parameters)
Initialization method for the function.
|
void |
processElement(RowData input,
KeyedProcessFunction.Context context,
Collector<RowData> out)
Process one element from the input stream.
|
void |
snapshotState(FunctionSnapshotContext context)
This method is called when a snapshot for a checkpoint is requested.
|
checkSortKeyInBufferRange, collectDelete, collectDelete, collectInsert, collectInsert, collectUpdateAfter, collectUpdateAfter, collectUpdateBefore, collectUpdateBefore, getDefaultTopNSize, hasOffset, initRankEnd, isInRankEnd, isInRankRange, registerMetric, setKeyContextcleanupState, initCleanupTimeState, isProcessingTimeTimer, needToCleanupState, registerProcessingCleanupTimerclose, getIterationRuntimeContext, getRuntimeContext, setRuntimeContextclone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitregisterProcessingCleanupTimerpublic UpdatableTopNFunction(long minRetentionTime,
long maxRetentionTime,
InternalTypeInfo<RowData> inputRowType,
RowDataKeySelector rowKeySelector,
GeneratedRecordComparator generatedRecordComparator,
RowDataKeySelector sortKeySelector,
RankType rankType,
RankRange rankRange,
boolean generateUpdateBefore,
boolean outputRankNumber,
long cacheSize)
public void open(Configuration parameters) throws Exception
RichFunctionThe configuration object passed to the function can be used for configuration and initialization. The configuration contains all parameters that were configured on the function in the program composition.
public class MyFilter extends RichFilterFunction<String> {
private String searchString;
public void open(Configuration parameters) {
this.searchString = parameters.getString("foo");
}
public boolean filter(String value) {
return value.equals(searchString);
}
}
By default, this method does nothing.
open in interface RichFunctionopen in class AbstractTopNFunctionparameters - The configuration containing the parameters attached to the contract.Exception - Implementations may forward exceptions, which are caught by the runtime.
When the runtime catches an exception, it aborts the task and lets the fail-over logic
decide whether to retry the task execution.Configurationpublic void onTimer(long timestamp,
KeyedProcessFunction.OnTimerContext ctx,
Collector<RowData> out)
throws Exception
KeyedProcessFunctionTimerService fires.onTimer in class KeyedProcessFunction<RowData,RowData,RowData>timestamp - The timestamp of the firing timer.ctx - An KeyedProcessFunction.OnTimerContext that allows querying the timestamp, the TimeDomain, and the key of the firing timer and getting a TimerService for
registering timers and querying the time. The context is only valid during the invocation
of this method, do not store it.out - The collector for returning result values.Exception - This method may throw exceptions. Throwing an exception will cause the
operation to fail and may trigger recovery.public void initializeState(FunctionInitializationContext context) throws Exception
CheckpointedFunctioninitializeState in interface CheckpointedFunctioncontext - the context for initializing the operatorException - Thrown, if state could not be created ot restored.public void processElement(RowData input, KeyedProcessFunction.Context context, Collector<RowData> out) throws Exception
KeyedProcessFunctionThis function can output zero or more elements using the Collector parameter and
also update internal state or set timers using the KeyedProcessFunction.Context parameter.
processElement in class KeyedProcessFunction<RowData,RowData,RowData>input - The input value.context - A KeyedProcessFunction.Context that allows querying the timestamp of the element and getting a
TimerService for registering timers and querying the time. The context is only
valid during the invocation of this method, do not store it.out - The collector for returning result values.Exception - This method may throw exceptions. Throwing an exception will cause the
operation to fail and may trigger recovery.public void snapshotState(FunctionSnapshotContext context) throws Exception
CheckpointedFunctionFunctionInitializationContext when the Function was initialized, or offered now by FunctionSnapshotContext itself.snapshotState in interface CheckpointedFunctioncontext - the context for drawing a snapshot of the operatorException - Thrown, if state could not be created ot restored.Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.