org.apache.flink.api.common.typeutils

Interface TypeSerializerSnapshot<T>

Type Parameters:

T - The data type that the originating serializer of this configuration serializes.

All Known Implementing Classes:

ArrayDataSerializer.ArrayDataSerializerSnapshot, ArrayDataSerializer.ArrayDataSerializerSnapshot, ArrayListSerializerSnapshot, AvroSerializer.AvroSchemaSerializerConfigSnapshot, AvroSerializerSnapshot, BigDecSerializer.BigDecSerializerSnapshot, BigDecSerializer.BigDecSerializerSnapshot, BigIntSerializer.BigIntSerializerSnapshot, BinaryRowDataSerializer.BinaryRowDataSerializerSnapshot, BooleanPrimitiveArraySerializer.BooleanPrimitiveArraySerializerSnapshot, BooleanSerializer.BooleanSerializerSnapshot, BooleanValueSerializer.BooleanValueSerializerSnapshot, ByteArrayWrapperSerializer.ByteArrayWrapperSerializerSnapshot, BytePrimitiveArraySerializer.BytePrimitiveArraySerializerSnapshot, ByteSerializer.ByteSerializerSnapshot, ByteValueSerializer.ByteValueSerializerSnapshot, CharPrimitiveArraySerializer.CharPrimitiveArraySerializerSnapshot, CharSerializer.CharSerializerSnapshot, CharValueSerializer.CharValueSerializerSnapshot, CoGroupedStreams.UnionSerializerSnapshot, CompositeTypeSerializerSnapshot, CopyableValueSerializer.CopyableValueSerializerSnapshot, CountWindow.Serializer.CountWindowSerializerSnapshot, DateSerializer.DateSerializerSnapshot, DateSerializer.DateSerializerSnapshot, DecimalDataSerializer.DecimalSerializerSnapshot, DecimalDataSerializer.DecimalSerializerSnapshot, DeweyNumber.DeweyNumberSerializer.DeweyNumberSerializerSnapshot, DoublePrimitiveArraySerializer.DoublePrimitiveArraySerializerSnapshot, DoubleSerializer.DoubleSerializerSnapshot, DoubleValueSerializer.DoubleValueSerializerSnapshot, EitherSerializerSnapshot, EnumSerializer.EnumSerializerSnapshot, EventId.EventIdSerializer.EventIdSerializerSnapshot, ExternalSerializer.ExternalSerializerSnapshot, FloatPrimitiveArraySerializer.FloatPrimitiveArraySerializerSnapshot, FloatSerializer.FloatSerializerSnapshot, FloatValueSerializer.FloatValueSerializerSnapshot, GenericArraySerializerConfigSnapshot, GenericArraySerializerSnapshot, GenericTypeSerializerSnapshot, GlobalWindow.Serializer.GlobalWindowSerializerSnapshot, InstantSerializer.InstantSerializerSnapshot, IntervalJoinOperator.BufferEntrySerializerSnapshot, IntPrimitiveArraySerializer.IntPrimitiveArraySerializerSnapshot, IntSerializer.IntSerializerSnapshot, IntValueSerializer.IntValueSerializerSnapshot, JavaEitherSerializerSnapshot, JavaSerializer.JavaSerializerSnapshot, KryoSerializerSnapshot, LinkedListSerializer.LinkedListSerializerSnapshot, ListSerializerSnapshot, ListViewSerializerSnapshot, LocalDateSerializer.LocalDateSerializerSnapshot, LocalDateTimeSerializer.LocalDateTimeSerializerSnapshot, LocalTimeSerializer.LocalTimeSerializerSnapshot, LockableTypeSerializerSnapshot, LongPrimitiveArraySerializer.LongPrimitiveArraySerializerSnapshot, LongSerializer.LongSerializerSnapshot, LongValueSerializer.LongValueSerializerSnapshot, MapDataSerializer.BaseMapSerializerSnapshot, MapDataSerializer.MapDataSerializerSnapshot, MapSerializerSnapshot, MapViewSerializerSnapshot, NFA.MigratedNFASerializerSnapshot, NFAStateSerializerSnapshot, NodeId.NodeIdSerializer.NodeIdSerializerSnapshot, NullableSerializer.NullableSerializerSnapshot, NullAwareMapSerializerSnapshot, NullSerializer.NullSerializerSnapshot, NullValueSerializer.NullValueSerializerSnapshot, PojoSerializerSnapshot, RawValueDataSerializer.RawValueDataSerializerSnapshot, RowDataSerializer.RowDataSerializerSnapshot, RowDataSerializer.RowDataSerializerSnapshot, RowSerializer.RowSerializerSnapshot, ScalaCaseClassSerializerSnapshot, ScalaEitherSerializerSnapshot, ScalaOptionSerializerSnapshot, ScalaTrySerializerSnapshot, SharedBuffer.SharedBufferSerializerSnapshot, SharedBufferEdge.SharedBufferEdgeSerializer.SharedBufferEdgeSerializerSnapshot, SharedBufferNode.SharedBufferNodeSerializer.SharedBufferNodeSerializerSnapshot, SharedBufferNodeSerializerSnapshotV2, ShortPrimitiveArraySerializer.ShortPrimitiveArraySerializerSnapshot, ShortSerializer.ShortSerializerSnapshot, ShortValueSerializer.ShortValueSerializerSnapshot, SimpleTypeSerializerSnapshot, SingleThreadAccessCheckingTypeSerializer.SingleThreadAccessCheckingTypeSerializerSnapshot, SortedMapSerializerSnapshot, SqlDateSerializer.SqlDateSerializerSnapshot, SqlTimeSerializer.SqlTimeSerializerSnapshot, SqlTimestampSerializer.SqlTimestampSerializerSnapshot, StatefulComplexPayloadSerializer.Snapshot, StreamElementSerializer.StreamElementSerializerSnapshot, StringArraySerializer.StringArraySerializerSnapshot, StringDataSerializer.StringDataSerializerSnapshot, StringSerializer.StringSerializerSnapshot, StringSerializer.StringSerializerSnapshot, StringValueSerializer.StringValueSerializerSnapshot, TimerSerializerSnapshot, TimeSerializer.TimeSerializerSnapshot, TimestampDataSerializer.TimestampDataSerializerSnapshot, TimestampSerializer.TimestampSerializerSnapshot, TimeWindow.Serializer.TimeWindowSerializerSnapshot, TimeWindow.Serializer.TimeWindowSerializerSnapshot, TraversableSerializerSnapshot, TtlStateFactory.TtlSerializerSnapshot, Tuple0SerializerSnapshot, Tuple2CaseClassSerializerSnapshot, TupleSerializerSnapshot, TwoPhaseCommitSinkFunction.StateSerializerSnapshot, ValueSerializer.ValueSerializerSnapshot, ValueWithTs.ValueWithTsSerializerSnapshot, VoidNamespaceSerializer.VoidNamespaceSerializerSnapshot, VoidNamespaceSerializer.VoidNamespaceSerializerSnapshot, VoidSerializer.VoidSerializerSnapshot, WindowKeySerializer.WindowKeySerializerSnapshot, WritableSerializer.WritableSerializerSnapshot

@PublicEvolving
public interface TypeSerializerSnapshot<T>

A TypeSerializerSnapshot is a point-in-time view of a TypeSerializer's configuration. The configuration snapshot of a serializer is persisted within checkpoints as a single source of meta information about the schema of serialized data in the checkpoint. This serves three purposes:

• Capturing serializer parameters and schema: a serializer's configuration snapshot represents information about the parameters, state, and schema of a serializer. This is explained in more detail below.
• Compatibility checks for new serializers: when new serializers are available, they need to be checked whether or not they are compatible to read the data written by the previous serializer. This is performed by providing the new serializer snapshots to resolve the compatibility with the corresponding previous serializer snapshots in checkpoints.
• Factory for a read serializer when schema conversion is required: in the case that new serializers are not compatible to read previous data, a schema conversion process executed across all data is required before the new serializer can be continued to be used. This conversion process requires a compatible read serializer to restore serialized bytes as objects, and then written back again using the new serializer. In this scenario, the serializer configuration snapshots in checkpoints doubles as a factory for the read serializer of the conversion process.

Serializer Configuration and Schema

Since serializer configuration snapshots needs to be used to ensure serialization compatibility for the same managed state as well as serving as a factory for compatible read serializers, the configuration snapshot should encode sufficient information about:

• Parameter settings of the serializer: parameters of the serializer include settings required to setup the serializer, or the state of the serializer if it is stateful. If the serializer has nested serializers, then the configuration snapshot should also contain the parameters of the nested serializers.
• Serialization schema of the serializer: the binary format used by the serializer, or in other words, the schema of data written by the serializer.

NOTE: Implementations must contain the default empty nullary constructor. This is required to be able to deserialize the configuration snapshot from its binary form.

Method Summary

Modifier and Type	Method and Description
int	getCurrentVersion() Returns the version of the current snapshot's written binary format.
void	readSnapshot(int readVersion, DataInputView in, ClassLoader userCodeClassLoader) Reads the serializer snapshot from the provided DataInputView.
static <T> TypeSerializerSnapshot<T>	readVersionedSnapshot(DataInputView in, ClassLoader cl) Reads a snapshot from the stream, performing resolving This method reads snapshots written by writeVersionedSnapshot(DataOutputView, TypeSerializerSnapshot).
default TypeSerializerSchemaCompatibility<T>	resolveSchemaCompatibility(TypeSerializer<T> newSerializer) Deprecated. This method has been replaced by #resolveSchemaCompatibility(TypeSerializerSnapshot) and will be removed in the future release. It's strongly recommended to migrate from old method to the new one, see the doc section "Migrating from deprecated `TypeSerializerSnapshot#resolveSchemaCompatibility(TypeSerializer newSerializer)`" for more details.
default TypeSerializerSchemaCompatibility<T>	resolveSchemaCompatibility(TypeSerializerSnapshot<T> oldSerializerSnapshot) Checks current serializer's compatibility to read data written by the prior serializer.
TypeSerializer<T>	restoreSerializer() Recreates a serializer instance from this snapshot.
void	writeSnapshot(DataOutputView out) Writes the serializer snapshot to the provided DataOutputView.
static void	writeVersionedSnapshot(DataOutputView out, TypeSerializerSnapshot<?> snapshot) Writes the given snapshot to the out stream.

Method Detail

getCurrentVersion

int getCurrentVersion()

Returns the version of the current snapshot's written binary format.

Returns:

the version of the current snapshot's written binary format.

writeSnapshot

void writeSnapshot(DataOutputView out)
            throws IOException

Writes the serializer snapshot to the provided DataOutputView. The current version of the written serializer snapshot's binary format is specified by the getCurrentVersion() method.

Parameters:

out - the DataOutputView to write the snapshot to.

Throws:

IOException - Thrown if the snapshot data could not be written.

See Also:

writeVersionedSnapshot(DataOutputView, TypeSerializerSnapshot)

readSnapshot

void readSnapshot(int readVersion,
                  DataInputView in,
                  ClassLoader userCodeClassLoader)
           throws IOException

Reads the serializer snapshot from the provided DataInputView. The version of the binary format that the serializer snapshot was written with is provided. This version can be used to determine how the serializer snapshot should be read.

Parameters:

readVersion - version of the serializer snapshot's written binary format

in - the DataInputView to read the snapshot from.

userCodeClassLoader - the user code classloader

Throws:

IOException - Thrown if the snapshot data could be read or parsed.

See Also:

readVersionedSnapshot(DataInputView, ClassLoader)

restoreSerializer

TypeSerializer<T> restoreSerializer()

Recreates a serializer instance from this snapshot. The returned serializer can be safely used to read data written by the prior serializer (i.e., the serializer that created this snapshot).

Returns:

a serializer instance restored from this serializer snapshot.

resolveSchemaCompatibility

@Deprecated
default TypeSerializerSchemaCompatibility<T> resolveSchemaCompatibility(TypeSerializer<T> newSerializer)

Deprecated. This method has been replaced by #resolveSchemaCompatibility(TypeSerializerSnapshot) and will be removed in the future release. It's strongly recommended to migrate from old method to the new one, see the doc section "Migrating from deprecated `TypeSerializerSnapshot#resolveSchemaCompatibility(TypeSerializer newSerializer)`" for more details.

Checks a new serializer's compatibility to read data written by the prior serializer.

When a checkpoint/savepoint is restored, this method checks whether the serialization format of the data in the checkpoint/savepoint is compatible for the format of the serializer used by the program that restores the checkpoint/savepoint. The outcome can be that the serialization format is compatible, that the program's serializer needs to reconfigure itself (meaning to incorporate some information from the TypeSerializerSnapshot to be compatible), that the format is outright incompatible, or that a migration needed. In the latter case, the TypeSerializerSnapshot produces a serializer to deserialize the data, and the restoring program's serializer re-serializes the data, thus converting the format during the restore operation.

Parameters:

newSerializer - the new serializer to check.

Returns:

the serializer compatibility result.

resolveSchemaCompatibility

default TypeSerializerSchemaCompatibility<T> resolveSchemaCompatibility(TypeSerializerSnapshot<T> oldSerializerSnapshot)

Checks current serializer's compatibility to read data written by the prior serializer.

When a checkpoint/savepoint is restored, this method checks whether the serialization format of the data in the checkpoint/savepoint is compatible for the format of the serializer used by the program that restores the checkpoint/savepoint. The outcome can be that the serialization format is compatible, that the program's serializer needs to reconfigure itself (meaning to incorporate some information from the TypeSerializerSnapshot to be compatible), that the format is outright incompatible, or that a migration needed. In the latter case, the TypeSerializerSnapshot produces a serializer to deserialize the data, and the restoring program's serializer re-serializes the data, thus converting the format during the restore operation.

This method must be implemented to clarify the compatibility. See FLIP-263 for more details.

Parameters:

oldSerializerSnapshot - the old serializer snapshot to check.

Returns:

the serializer compatibility result.

writeVersionedSnapshot

static void writeVersionedSnapshot(DataOutputView out,
                                   TypeSerializerSnapshot<?> snapshot)
                            throws IOException

Writes the given snapshot to the out stream. One should always use this method to write snapshots out, rather than directly calling writeSnapshot(DataOutputView).

The snapshot written with this method can be read via readVersionedSnapshot(DataInputView, ClassLoader).

Throws:

IOException

readVersionedSnapshot

static <T> TypeSerializerSnapshot<T> readVersionedSnapshot(DataInputView in,
                                                           ClassLoader cl)
                                                    throws IOException

Reads a snapshot from the stream, performing resolving

This method reads snapshots written by writeVersionedSnapshot(DataOutputView, TypeSerializerSnapshot).

Throws:

IOException