Class ProcTimeDeduplicateKeepLastRowFunction
- java.lang.Object
-
- org.apache.flink.api.common.functions.AbstractRichFunction
-
- org.apache.flink.streaming.api.functions.KeyedProcessFunction<K,IN,OUT>
-
- org.apache.flink.table.runtime.operators.deduplicate.ProcTimeDeduplicateKeepLastRowFunction
-
- All Implemented Interfaces:
Serializable
,Function
,RichFunction
public class ProcTimeDeduplicateKeepLastRowFunction extends KeyedProcessFunction<K,IN,OUT>
This function is used to deduplicate on keys and keeps only last row.- See Also:
- Serialized Form
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.flink.streaming.api.functions.KeyedProcessFunction
KeyedProcessFunction.Context, KeyedProcessFunction.OnTimerContext
-
-
Field Summary
Fields Modifier and Type Field Description protected TypeSerializer<OUT>
serializer
protected ValueState<T>
state
protected long
stateRetentionTime
protected TypeInformation<T>
typeInfo
-
Constructor Summary
Constructors Constructor Description ProcTimeDeduplicateKeepLastRowFunction(InternalTypeInfo<RowData> typeInfo, long stateRetentionTime, boolean generateUpdateBefore, boolean generateInsert, boolean inputInsertOnly, GeneratedRecordEqualiser genRecordEqualiser)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
open(OpenContext openContext)
Initialization method for the function.void
processElement(RowData input, KeyedProcessFunction.Context ctx, Collector<RowData> out)
Process one element from the input stream.-
Methods inherited from class org.apache.flink.streaming.api.functions.KeyedProcessFunction
onTimer
-
Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction
close, getIterationRuntimeContext, getRuntimeContext, setRuntimeContext
-
-
-
-
Field Detail
-
typeInfo
protected final TypeInformation<T> typeInfo
-
stateRetentionTime
protected final long stateRetentionTime
-
serializer
protected final TypeSerializer<OUT> serializer
-
state
protected ValueState<T> state
-
-
Constructor Detail
-
ProcTimeDeduplicateKeepLastRowFunction
public ProcTimeDeduplicateKeepLastRowFunction(InternalTypeInfo<RowData> typeInfo, long stateRetentionTime, boolean generateUpdateBefore, boolean generateInsert, boolean inputInsertOnly, GeneratedRecordEqualiser genRecordEqualiser)
-
-
Method Detail
-
open
public void open(OpenContext openContext) throws Exception
Description copied from interface:RichFunction
Initialization method for the function. It is called before the actual working methods (like map or join) and thus suitable for one time setup work. For functions that are part of an iteration, this method will be invoked at the beginning of each iteration superstep.The openContext object passed to the function can be used for configuration and initialization. The openContext contains some necessary information that were configured on the function in the program composition.
public class MyFilter extends RichFilterFunction<String> { private String searchString; public void open(OpenContext openContext) { // initialize the value of searchString } public boolean filter(String value) { return value.equals(searchString); } }
- Specified by:
open
in interfaceRichFunction
- Parameters:
openContext
- The context containing information about the context in which the function is opened.- Throws:
Exception
- Implementations may forward exceptions, which are caught by the runtime. When the runtime catches an exception, it aborts the task and lets the fail-over logic decide whether to retry the task execution.
-
processElement
public void processElement(RowData input, KeyedProcessFunction.Context ctx, Collector<RowData> out) throws Exception
Description copied from class:KeyedProcessFunction
Process one element from the input stream.This function can output zero or more elements using the
Collector
parameter and also update internal state or set timers using theKeyedProcessFunction.Context
parameter.- Specified by:
processElement
in classKeyedProcessFunction<RowData,RowData,RowData>
- Parameters:
input
- The input value.ctx
- AKeyedProcessFunction.Context
that allows querying the timestamp of the element and getting aTimerService
for registering timers and querying the time. The context is only valid during the invocation of this method, do not store it.out
- The collector for returning result values.- Throws:
Exception
- This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.
-
-