@Documented @Target(value=TYPE) @Retention(value=RUNTIME) @Repeatable(value=MultipleExecNodeMetadata.class) @Internal public @interface ExecNodeMetadata
ExecNode
s to keep necessary metadata when
serializing/deserializing them in a plan. It's used for internal bookkeeping across Flink
versions, and to provide necessary information to the testing infrastructure.
Each ExecNode
needs to be annotated and provide the necessary metadata info so that it
can be correctly serialized and later on instantiated from a string (JSON) plan.
It's possible for one ExecNode
class to use multiple annotations to denote ability to
upgrade to more versions. an ExecNode
class can be annotated directly with multiple
ExecNodeMetadata
annotations, or with a single MultipleExecNodeMetadata
annotation where the MultipleExecNodeMetadata.value()
is an array of ExecNodeMetadata
annotations.
Modifier and Type | Required Element and Description |
---|---|
FlinkVersion |
minPlanVersion
Used for plan validation and potentially plan migration.
|
FlinkVersion |
minStateVersion
Used for operator and potentially savepoint migration.
|
String |
name
Unique name of the
ExecNode for serialization/deserialization and uid() generation. |
int |
version
A positive integer denoting the evolving version of an
ExecNode , used for
serialization/deserialization and uid() generation. |
Modifier and Type | Optional Element and Description |
---|---|
String[] |
consumedOptions
Hard coded list of
ExecutionConfigOptions keys of in the Flink version when the
ExecNode was added. |
String[] |
producedTransformations
Set of transformation names that can be part of the resulting
Transformation s. |
public abstract String name
public abstract FlinkVersion minPlanVersion
Needs to be updated when the JSON for the ExecNode
changes: e.g. after adding an
attribute to the JSON spec of the ExecNode.
The annotation does not need to be updated for every Flink version. As the name suggests it is about the "minimum" version for a restore. If the minimum version is higher than the current Flink version, plan migration is necessary.
Changing this version will always result in a new version()
for the ExecNode
.
Plan migration tests can use this information.
Completeness tests can verify that restore tests exist for all JSON plan variations.
public abstract FlinkVersion minStateVersion
Needs to be updated whenever the state layout of an ExecNode changes. In some cases, the operator can implement and perform state migration. If the minimum version is higher than the current Flink version, savepoint migration is necessary.
Changing this version will always result in a new ExecNode version()
.
Restore tests can verify that operator migration works for all Flink state versions.
Completeness tests can verify that restore tests exist for all state variations.
public abstract String[] consumedOptions
ExecutionConfigOptions
keys of in the Flink version when the
ExecNode was added. Does not reference instances in the ExecutionConfigOptions
class
in case those get refactored.
Completeness tests can verify that every option is set once in restore and change detection tests.
Completeness tests can verify that the ExecutionConfigOptions
class still contains
an option (via key or fallback key) for the given key.
Restore can verify whether the restored ExecNode config map contains only options of the given keys.
Common options used for all StreamExecNode
s:
public abstract String[] producedTransformations
Transformation
s.
Restore and completeness tests can verify there exists at least one test that adds each
operator and that the created Transformation
s contain only operators with Transformation.getUid()
containing the given operator names.
The concrete combinations or existence of these operators in the final pipeline depends on various parameters (both configuration and ExecNode-specific arguments such as interval size etc.).
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.