Package org.apache.flink.state.api

Class SavepointWriter

java.lang.Object

org.apache.flink.state.api.SavepointWriter

@PublicEvolving
public class SavepointWriter
extends Object

A SavepointWriter can create new savepoints from bounded data streams. This can allow for boostrapping state for new applications or modifying the savepoints of existing jobs.

Field Summary

Fields

Modifier and Type	Field	Description
protected SavepointMetadataV2	metadata	The savepoint metadata, which maintains the current set of existing / newly added operator states.
protected StateBackend	stateBackend	The state backend to use when writing this savepoint.

Method Summary

Modifier and Type	Method	Description
SavepointWriter	changeOperatorIdentifier(OperatorIdentifier from, OperatorIdentifier to)	Changes the identifier of an operator.
static SavepointWriter	fromExistingSavepoint(StreamExecutionEnvironment executionEnvironment, String path)	Loads an existing savepoint.
static SavepointWriter	fromExistingSavepoint(StreamExecutionEnvironment executionEnvironment, String path, StateBackend stateBackend)	Loads an existing savepoint.
static SavepointWriter	newSavepoint(StreamExecutionEnvironment executionEnvironment, int maxParallelism)	Creates a new savepoint.
static SavepointWriter	newSavepoint(StreamExecutionEnvironment executionEnvironment, StateBackend stateBackend, int maxParallelism)	Creates a new savepoint.
SavepointWriter	removeOperator(OperatorIdentifier identifier)	Drop an existing operator from the savepoint.
<T> SavepointWriter	withConfiguration(ConfigOption<T> option, T value)	Sets a configuration that will be applied to the stream operators used to bootstrap a new savepoint.
<T> SavepointWriter	withOperator(OperatorIdentifier identifier, StateBootstrapTransformation<T> transformation)	Adds a new operator to the savepoint.
void	write(String path)	Write out a new or updated savepoint.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

metadata

protected final SavepointMetadataV2 metadata

The savepoint metadata, which maintains the current set of existing / newly added operator states.

stateBackend

@Nullable
protected final StateBackend stateBackend

The state backend to use when writing this savepoint.

Method Detail

fromExistingSavepoint

public static SavepointWriter fromExistingSavepoint(StreamExecutionEnvironment executionEnvironment,
                                                    String path)
                                             throws IOException

Loads an existing savepoint. Useful if you want to modify or extend the state of an existing application. The savepoint will be written using the state backend defined via the clusters configuration.

Parameters:

path - The path to an existing savepoint on disk.

Returns:

A SavepointWriter.

Throws:

IOException

See Also:

fromExistingSavepoint(StreamExecutionEnvironment, String, StateBackend), withConfiguration(ConfigOption, Object)

fromExistingSavepoint

public static SavepointWriter fromExistingSavepoint(StreamExecutionEnvironment executionEnvironment,
                                                    String path,
                                                    StateBackend stateBackend)
                                             throws IOException

Loads an existing savepoint. Useful if you want to modify or extend the state of an existing application.

Parameters:

path - The path to an existing savepoint on disk.

stateBackend - The state backend of the savepoint.

Returns:

A SavepointWriter.

Throws:

IOException

newSavepoint

public static SavepointWriter newSavepoint(StreamExecutionEnvironment executionEnvironment,
                                           int maxParallelism)

Creates a new savepoint. The savepoint will be written using the state backend defined via the clusters configuration.

Parameters:

maxParallelism - The max parallelism of the savepoint.

Returns:

A SavepointWriter.

See Also:

newSavepoint(StreamExecutionEnvironment, StateBackend, int), withConfiguration(ConfigOption, Object)

newSavepoint

public static SavepointWriter newSavepoint(StreamExecutionEnvironment executionEnvironment,
                                           StateBackend stateBackend,
                                           int maxParallelism)

Creates a new savepoint.

Parameters:

stateBackend - The state backend of the savepoint used for keyed state.

maxParallelism - The max parallelism of the savepoint.

Returns:

A SavepointWriter.

See Also:

newSavepoint(StreamExecutionEnvironment, int)

removeOperator

public SavepointWriter removeOperator(OperatorIdentifier identifier)

Drop an existing operator from the savepoint.

Parameters:

identifier - The identifier of the operator.

Returns:

A modified savepoint.

withOperator

public <T> SavepointWriter withOperator(OperatorIdentifier identifier,
                                        StateBootstrapTransformation<T> transformation)

Adds a new operator to the savepoint.

Parameters:

identifier - The identifier of the operator.

transformation - The operator to be included.

Returns:

The modified savepoint.

withConfiguration

public <T> SavepointWriter withConfiguration(ConfigOption<T> option,
                                             T value)

Sets a configuration that will be applied to the stream operators used to bootstrap a new savepoint.

Type Parameters:

T - type of the value to be stored

Parameters:

option - metadata information

value - value to be stored

Returns:

The modified savepoint.

changeOperatorIdentifier

public SavepointWriter changeOperatorIdentifier(OperatorIdentifier from,
                                                OperatorIdentifier to)

Changes the identifier of an operator.

This method is comparatively cheap since it only modifies savepoint metadata without reading the entire savepoint data.

Use-cases include, but are not limited to:

• assigning a UID to an operator that did not have a UID assigned before
• changing the UID of an operator
• swapping the states of 2 operators

Identifier changes are applied after all other operations; in the following example the savepoint will only contain UID_2.

     SavepointWriter savepoint = ...
     savepoint.withOperator(UID_1, ...)
     savepoint.changeOperatorIdentifier(UID_1, UID_2)
     savepoint.write(...)
 

You cannot define a chain of changes; in the following example the savepoint will only contain UID_2.

     SavepointWriter savepoint = ...
     savepoint.withOperator(UID_1, ...)
     savepoint.changeOperatorIdentifier(UID_1, UID_2)
     savepoint.changeOperatorIdentifier(UID_2, UID_3)
     savepoint.write(...)
 

Parameters:

from - operator whose identifier should be changed

to - desired identifier

Returns:

The modified savepoint.

write

public final void write(String path)

Write out a new or updated savepoint.

Parameters:

path - The path to where the savepoint should be written.