public class StreamNonDeterministicUpdatePlanVisitor extends Object
The visitor will try to satisfy the required determinism(represent by ImmutableBitSet) from root. The transmission rule of required determinism:
0. all required determinism is under the precondition: input has updates, that is say no update determinism will be passed to an insert only stream
1. the initial required determinism to the root node(e.g., sink node) was none
2. for a relNode, it will process on two aspects: - can satisfy non-empty required determinism - actively requires determinism from input by self requirements(e.g., stateful node works on retract by row mode)
Rel3
| require input
v
Rel2 {1. satisfy Rel3's requirement 2. append new requirement to input Rel1}
| require input
v
Rel1
the requiredDeterminism passed to input will exclude columns which were in upsertKey, e.g.,
Sink {pk=(c3)} requiredDeterminism=(c3)
| passed requiredDeterminism={}
GroupAgg{group by c3, day} append requiredDeterminism=(c3, day)
| passed requiredDeterminism=(c3, day)
Project{select c1,c2,DATE_FORMAT(CURRENT_TIMESTAMP, 'yyMMdd') day,...} [x] can not satisfy
|
Deduplicate{keep last row, dedup on c1,c2}
|
Scan
3. for a sink node, it will require key columns' determinism when primary key is defined or require all columns' determinism when no primary key is defined
4. for a cdc source node(which will generate updates), the metadata columns are treated as non-deterministic.
Constructor and Description |
---|
StreamNonDeterministicUpdatePlanVisitor() |
Modifier and Type | Method and Description |
---|---|
org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalRel |
visit(org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalRel rel) |
org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalRel |
visit(org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalRel rel,
org.apache.calcite.util.ImmutableBitSet requireDeterminism) |
public StreamNonDeterministicUpdatePlanVisitor()
public org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalRel visit(org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalRel rel)
public org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalRel visit(org.apache.flink.table.planner.plan.nodes.physical.stream.StreamPhysicalRel rel, org.apache.calcite.util.ImmutableBitSet requireDeterminism)
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.