org.apache.flink.table.runtime.operators.deduplicate

Class ProcTimeDeduplicateKeepLastRowFunction

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.streaming.api.functions.KeyedProcessFunction<K,IN,OUT>

org.apache.flink.table.runtime.operators.deduplicate.ProcTimeDeduplicateKeepLastRowFunction

All Implemented Interfaces:

Serializable, Function, RichFunction

public class ProcTimeDeduplicateKeepLastRowFunction
extends KeyedProcessFunction<K,IN,OUT>

This function is used to deduplicate on keys and keeps only last row.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.streaming.api.functions.KeyedProcessFunction

KeyedProcessFunction.Context, KeyedProcessFunction.OnTimerContext

Field Summary

Fields

Modifier and Type	Field and Description
protected TypeSerializer<OUT>	serializer
protected ValueState<T>	state
protected long	stateRetentionTime
protected TypeInformation<T>	typeInfo

Constructor Summary

Constructors

Constructor and Description
ProcTimeDeduplicateKeepLastRowFunction(InternalTypeInfo<RowData> typeInfo, long stateRetentionTime, boolean generateUpdateBefore, boolean generateInsert, boolean inputInsertOnly, GeneratedRecordEqualiser genRecordEqualiser)

Method Summary

Modifier and Type	Method and Description
void	open(Configuration configure) Initialization method for the function.
void	processElement(RowData input, KeyedProcessFunction.Context ctx, Collector<RowData> out) Process one element from the input stream.

Methods inherited from class org.apache.flink.streaming.api.functions.KeyedProcessFunction

onTimer

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

close, getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

typeInfo

protected final TypeInformation<T> typeInfo

stateRetentionTime

protected final long stateRetentionTime

serializer

protected final TypeSerializer<OUT> serializer

state

protected ValueState<T> state

Constructor Detail

ProcTimeDeduplicateKeepLastRowFunction

public ProcTimeDeduplicateKeepLastRowFunction(InternalTypeInfo<RowData> typeInfo,
                                              long stateRetentionTime,
                                              boolean generateUpdateBefore,
                                              boolean generateInsert,
                                              boolean inputInsertOnly,
                                              GeneratedRecordEqualiser genRecordEqualiser)

Method Detail

open

public void open(Configuration configure)
          throws Exception

Description copied from interface: RichFunction

Initialization method for the function. It is called before the actual working methods (like map or join) and thus suitable for one time setup work. For functions that are part of an iteration, this method will be invoked at the beginning of each iteration superstep.

The configuration object passed to the function can be used for configuration and initialization. The configuration contains all parameters that were configured on the function in the program composition.

 public class MyFilter extends RichFilterFunction<String> {

     private String searchString;

     public void open(Configuration parameters) {
         this.searchString = parameters.getString("foo");
     }

     public boolean filter(String value) {
         return value.equals(searchString);
     }
 }
 

By default, this method does nothing.

Specified by:

open in interface RichFunction

Parameters:

configure - The configuration containing the parameters attached to the contract.

Throws:

Exception - Implementations may forward exceptions, which are caught by the runtime. When the runtime catches an exception, it aborts the task and lets the fail-over logic decide whether to retry the task execution.

See Also:

Configuration

processElement

public void processElement(RowData input,
                           KeyedProcessFunction.Context ctx,
                           Collector<RowData> out)
                    throws Exception

Description copied from class: KeyedProcessFunction

Process one element from the input stream.

This function can output zero or more elements using the Collector parameter and also update internal state or set timers using the KeyedProcessFunction.Context parameter.

Specified by:

processElement in class KeyedProcessFunction<RowData,RowData,RowData>

Parameters:

input - The input value.

ctx - A KeyedProcessFunction.Context that allows querying the timestamp of the element and getting a TimerService for registering timers and querying the time. The context is only valid during the invocation of this method, do not store it.

out - The collector for returning result values.

Throws:

Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.