Class StreamExecDeduplicate
- java.lang.Object
-
- org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase<RowData>
-
- org.apache.flink.table.planner.plan.nodes.exec.stream.StreamExecDeduplicate
-
- All Implemented Interfaces:
ExecNode<RowData>
,ExecNodeTranslator<RowData>
,FusionCodegenExecNode
,SingleTransformationTranslator<RowData>
,StreamExecNode<RowData>
@ExecNodeMetadata(name="stream-exec-deduplicate", version=1, consumedOptions={"table.exec.mini-batch.enabled","table.exec.mini-batch.size","table.exec.deduplicate.insert-update-after-sensitive-enabled","table.exec.deduplicate.mini-batch.compact-changes-enabled"}, producedTransformations="deduplicate", minPlanVersion=v1_15, minStateVersion=v1_15) public class StreamExecDeduplicate extends ExecNodeBase<RowData> implements StreamExecNode<RowData>, SingleTransformationTranslator<RowData>
StreamExecNode
which deduplicate on keys and keeps only first row or last row. This node is an optimization ofStreamExecRank
for some special cases. Compared toStreamExecRank
, this node could use mini-batch and access less state.
-
-
Field Summary
Fields Modifier and Type Field Description static String
DEDUPLICATE_TRANSFORMATION
static String
FIELD_NAME_GENERATE_UPDATE_BEFORE
static String
FIELD_NAME_IS_ROWTIME
static String
FIELD_NAME_KEEP_LAST_ROW
static String
FIELD_NAME_UNIQUE_KEYS
static String
STATE_NAME
-
Fields inherited from interface org.apache.flink.table.planner.plan.nodes.exec.ExecNode
FIELD_NAME_CONFIGURATION, FIELD_NAME_DESCRIPTION, FIELD_NAME_ID, FIELD_NAME_INPUT_PROPERTIES, FIELD_NAME_OUTPUT_TYPE, FIELD_NAME_STATE, FIELD_NAME_TYPE
-
-
Constructor Summary
Constructors Constructor Description StreamExecDeduplicate(int id, ExecNodeContext context, ReadableConfig persistedConfig, int[] uniqueKeys, boolean isRowtime, boolean keepLastRow, boolean generateUpdateBefore, List<StateMetadata> stateMetadataList, List<InputProperty> inputProperties, RowType outputType, String description)
StreamExecDeduplicate(ReadableConfig tableConfig, int[] uniqueKeys, boolean isRowtime, boolean keepLastRow, boolean generateUpdateBefore, InputProperty inputProperty, RowType outputType, String description)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description protected Transformation<RowData>
translateToPlanInternal(org.apache.flink.table.planner.delegation.PlannerBase planner, ExecNodeConfig config)
Internal method, translates this node into a Flink operator.-
Methods inherited from class org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase
accept, createFormattedTransformationDescription, createFormattedTransformationName, createTransformationDescription, createTransformationMeta, createTransformationMeta, createTransformationName, createTransformationUid, getContextFromAnnotation, getDescription, getId, getInputEdges, getInputProperties, getOutputType, getPersistedConfig, getSimplifiedName, getTransformation, inputsContainSingleton, replaceInputEdge, setCompiled, setInputEdges, supportFusionCodegen, translateToFusionCodegenSpec, translateToFusionCodegenSpecInternal, translateToPlan
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.flink.table.planner.plan.nodes.exec.ExecNode
accept, getDescription, getId, getInputEdges, getInputProperties, getOutputType, replaceInputEdge, setCompiled, setInputEdges
-
Methods inherited from interface org.apache.flink.table.planner.plan.nodes.exec.ExecNodeTranslator
translateToPlan
-
Methods inherited from interface org.apache.flink.table.planner.plan.nodes.exec.FusionCodegenExecNode
supportFusionCodegen, translateToFusionCodegenSpec
-
-
-
-
Field Detail
-
DEDUPLICATE_TRANSFORMATION
public static final String DEDUPLICATE_TRANSFORMATION
- See Also:
- Constant Field Values
-
FIELD_NAME_UNIQUE_KEYS
public static final String FIELD_NAME_UNIQUE_KEYS
- See Also:
- Constant Field Values
-
FIELD_NAME_IS_ROWTIME
public static final String FIELD_NAME_IS_ROWTIME
- See Also:
- Constant Field Values
-
FIELD_NAME_KEEP_LAST_ROW
public static final String FIELD_NAME_KEEP_LAST_ROW
- See Also:
- Constant Field Values
-
FIELD_NAME_GENERATE_UPDATE_BEFORE
public static final String FIELD_NAME_GENERATE_UPDATE_BEFORE
- See Also:
- Constant Field Values
-
STATE_NAME
public static final String STATE_NAME
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
StreamExecDeduplicate
public StreamExecDeduplicate(ReadableConfig tableConfig, int[] uniqueKeys, boolean isRowtime, boolean keepLastRow, boolean generateUpdateBefore, InputProperty inputProperty, RowType outputType, String description)
-
StreamExecDeduplicate
public StreamExecDeduplicate(int id, ExecNodeContext context, ReadableConfig persistedConfig, int[] uniqueKeys, boolean isRowtime, boolean keepLastRow, boolean generateUpdateBefore, @Nullable List<StateMetadata> stateMetadataList, List<InputProperty> inputProperties, RowType outputType, String description)
-
-
Method Detail
-
translateToPlanInternal
protected Transformation<RowData> translateToPlanInternal(org.apache.flink.table.planner.delegation.PlannerBase planner, ExecNodeConfig config)
Description copied from class:ExecNodeBase
Internal method, translates this node into a Flink operator.- Specified by:
translateToPlanInternal
in classExecNodeBase<RowData>
- Parameters:
planner
- The planner.config
- per-ExecNode
configuration that contains the merged configuration from various layers which all the nodes implementing this method should use, instead of retrieving configuration from theplanner
. For more details checkExecNodeConfig
.
-
-