@Documented @Target(value=TYPE) @Retention(value=RUNTIME) @Repeatable(value=MultipleExecNodeMetadata.class) @Internal public @interface ExecNodeMetadata
ExecNodes to keep necessary metadata when serializing/deserializing them in a plan. It's used for internal bookkeeping across Flink versions, and to provide necessary information to the testing infrastructure.
ExecNode needs to be annotated and provide the necessary metadata info so that it
can be correctly serialized and later on instantiated from a string (JSON) plan.
It's possible for one
ExecNode class to use multiple annotations to denote ability to
upgrade to more versions. an
ExecNode class can be annotated directly with multiple
ExecNodeMetadata annotations, or with a single
annotation where the
MultipleExecNodeMetadata.value() is an array of
|Modifier and Type||Required Element and Description|
Used for plan validation and potentially plan migration.
Used for operator and potentially savepoint migration.
Unique name of the
A positive integer denoting the evolving version of an
|Modifier and Type||Optional Element and Description|
Hard coded list of
Set of transformation names that can be part of the resulting
public abstract String name
public abstract FlinkVersion minPlanVersion
Needs to be updated when the JSON for the
ExecNode changes: e.g. after adding an
attribute to the JSON spec of the ExecNode.
The annotation does not need to be updated for every Flink version. As the name suggests it is about the "minimum" version for a restore. If the minimum version is higher than the current Flink version, plan migration is necessary.
Plan migration tests can use this information.
Completeness tests can verify that restore tests exist for all JSON plan variations.
public abstract FlinkVersion minStateVersion
Needs to be updated whenever the state layout of an ExecNode changes. In some cases, the operator can implement and perform state migration. If the minimum version is higher than the current Flink version, savepoint migration is necessary.
Changing this version will always result in a new ExecNode
Restore tests can verify that operator migration works for all Flink state versions.
Completeness tests can verify that restore tests exist for all state variations.
public abstract String consumedOptions
ExecutionConfigOptionskeys of in the Flink version when the ExecNode was added. Does not reference instances in the
ExecutionConfigOptionsclass in case those get refactored.
Completeness tests can verify that every option is set once in restore and change detection tests.
Completeness tests can verify that the
ExecutionConfigOptions class still contains
an option (via key or fallback key) for the given key.
Restore can verify whether the restored ExecNode config map contains only options of the given keys.
Common options used for all
public abstract String producedTransformations
Restore and completeness tests can verify there exists at least one test that adds each
operator and that the created
Transformations contain only operators with
Transformation.getUid() containing the given operator names.
The concrete combinations or existence of these operators in the final pipeline depends on various parameters (both configuration and ExecNode-specific arguments such as interval size etc.).
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.