Package org.apache.flink.runtime.state.restore

Class FullSnapshotRestoreOperation<K>

  • Type Parameters:
    K - The data type of the key.
    All Implemented Interfaces:
    RestoreOperation<ThrowingIterator<SavepointRestoreResult>>
    @Internal
public class FullSnapshotRestoreOperation<K>
extends Object
implements RestoreOperation<ThrowingIterator<SavepointRestoreResult>>
    The procedure of restoring state from a savepoint written with the unified binary format. All state backends should support restoring from that format.

    The format was adopted from the RocksDB state backend. It is as follows: 

                        +-------------------------------------------------------------+
                    |                Keyed Backend Meta Information               |
  Meta Information  +--------------------+-----+----------------------------------+
                    |    State Meta 0    | ... |           State Meta M           |
                    +-------------------------------------------------------------+
                    |                       State ID (short)                      |
  State 0           +--------------------+-----+----------------------------------+
                    | State (K,V) pair 0 | ... | State (K,V) pair X (flipped MSB) |
                    +--------------------+-----+----------------------------------+
                    |                       State ID (short)                      |
  State 1           +--------------------+-----+----------------------------------+
                    | State (K,V) pair 0 | ... | State (K,V) pair X (flipped MSB) |
                    +--------------------+-----+----------------------------------+
                    |                       State ID (short)                      |
                    +--------------------+-----+----------------------------------+
  State M           | State (K,V) pair 0 | ... | State (K,V) pair X (flipped MSB) |
                    +--------------------+-----+----------------------------------+
                    |                END_OF_KEY_GROUP_MARK (0xFFFF)               |
                    +-------------------------------------------------------------+

    Additionally the format of the (K,V) pairs differs slightly depending on the type of the state object: 

     +------------------+-------------------------------------+
 |    ValueState    |    [CompositeKey(KG, K, NS), SV]    |
 |------------------+-------------------------------------+
 |    ListState     |    [CompositeKey(KG, K, NS), SV]    |
 +------------------+-------------------------------------+
 |     MapState     | [CompositeKey(KG, K, NS) :: UK, UV] |
 +------------------+-------------------------------------+
 | AggregatingState |    [CompositeKey(KG, K, NS), SV]    |
 +------------------+-------------------------------------+
 |   ReducingState  |    [CompositeKey(KG, K, NS), SV]    |
 |                  +-------------------------------------+
 |                  |    [CompositeKey(KG, K, NS), SV]    |
 +------------------+-------------------------------------+
 |      Timers      |    [KG :: TS :: K :: NS, (empty)]   |
 +------------------+-------------------------------------+

    For detailed information see FLIP-41: https://cwiki.apache.org/confluence/x/VJDiBg