org.apache.flink.runtime.state.restore

Class FullSnapshotRestoreOperation<K>

java.lang.Object

org.apache.flink.runtime.state.restore.FullSnapshotRestoreOperation<K>

Type Parameters:

K - The data type of the key.

All Implemented Interfaces:

RestoreOperation<ThrowingIterator<SavepointRestoreResult>>

@Internal
public class FullSnapshotRestoreOperation<K>
extends Object
implements RestoreOperation<ThrowingIterator<SavepointRestoreResult>>

The procedure of restoring state from a savepoint written with the unified binary format. All state backends should support restoring from that format.

The format was adopted from the RocksDB state backend. It is as follows:

                    +-------------------------------------------------------------+
                    |                Keyed Backend Meta Information               |
  Meta Information  +--------------------+-----+----------------------------------+
                    |    State Meta 0    | ... |           State Meta M           |
                    +-------------------------------------------------------------+
                    |                       State ID (short)                      |
  State 0           +--------------------+-----+----------------------------------+
                    | State (K,V) pair 0 | ... | State (K,V) pair X (flipped MSB) |
                    +--------------------+-----+----------------------------------+
                    |                       State ID (short)                      |
  State 1           +--------------------+-----+----------------------------------+
                    | State (K,V) pair 0 | ... | State (K,V) pair X (flipped MSB) |
                    +--------------------+-----+----------------------------------+
                    |                       State ID (short)                      |
                    +--------------------+-----+----------------------------------+
  State M           | State (K,V) pair 0 | ... | State (K,V) pair X (flipped MSB) |
                    +--------------------+-----+----------------------------------+
                    |                END_OF_KEY_GROUP_MARK (0xFFFF)               |
                    +-------------------------------------------------------------+
 

Additionally the format of the (K,V) pairs differs slightly depending on the type of the state object:

 +------------------+-------------------------------------+
 |    ValueState    |    [CompositeKey(KG, K, NS), SV]    |
 |------------------+-------------------------------------+
 |    ListState     |    [CompositeKey(KG, K, NS), SV]    |
 +------------------+-------------------------------------+
 |     MapState     | [CompositeKey(KG, K, NS) :: UK, UV] |
 +------------------+-------------------------------------+
 | AggregatingState |    [CompositeKey(KG, K, NS), SV]    |
 +------------------+-------------------------------------+
 |   ReducingState  |    [CompositeKey(KG, K, NS), SV]    |
 |                  +-------------------------------------+
 |                  |    [CompositeKey(KG, K, NS), SV]    |
 +------------------+-------------------------------------+
 |      Timers      |    [KG :: TS :: K :: NS, (empty)]   |
 +------------------+-------------------------------------+
 

For detailed information see FLIP-41: https://cwiki.apache.org/confluence/x/VJDiBg

Constructor Summary

Constructors

Constructor and Description
FullSnapshotRestoreOperation(KeyGroupRange keyGroupRange, ClassLoader userCodeClassLoader, Collection<KeyedStateHandle> restoreStateHandles, StateSerializerProvider<K> keySerializerProvider)

Method Summary

Modifier and Type	Method and Description
ThrowingIterator<SavepointRestoreResult>	restore() Restores state that was previously snapshot-ed from the provided state handles.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FullSnapshotRestoreOperation

public FullSnapshotRestoreOperation(KeyGroupRange keyGroupRange,
                                    ClassLoader userCodeClassLoader,
                                    Collection<KeyedStateHandle> restoreStateHandles,
                                    StateSerializerProvider<K> keySerializerProvider)

Method Detail

restore

public ThrowingIterator<SavepointRestoreResult> restore()
                                                 throws IOException,
                                                        StateMigrationException

Description copied from interface: RestoreOperation

Restores state that was previously snapshot-ed from the provided state handles.

Specified by:

restore in interface RestoreOperation<ThrowingIterator<SavepointRestoreResult>>

Throws:

IOException

StateMigrationException