Package org.apache.flink.runtime.state.heap

Class HeapKeyedStateBackend<K>

    • Method Detail

      • create

        public <T extends HeapPriorityQueueElement & PriorityComparable<? super T> & Keyed<?>> KeyGroupedInternalPriorityQueue<T> create​(@Nonnull
                                                                                                                                 String stateName,
                                                                                                                                 @Nonnull
                                                                                                                                 TypeSerializer<T> byteOrderedElementSerializer,
                                                                                                                                 boolean allowFutureMetadataUpdates)
        Description copied from interface: PriorityQueueSetFactory
        Creates a KeyGroupedInternalPriorityQueue.
        Type Parameters:
        T - type of the stored elements.
        Parameters:
        stateName - unique name for associated with this queue.
        byteOrderedElementSerializer - a serializer that with a format that is lexicographically ordered in alignment with elementPriorityComparator.
        allowFutureMetadataUpdates - whether allow metadata to update in the future or not.
        Returns:
        the queue with the specified unique name.

      • getKeys

        public <N> Stream<K> getKeys​(String state,
                             N namespace)
        Parameters:
        state - State variable for which existing keys will be returned.
        namespace - Namespace for which existing keys will be returned.
        Returns:
        A stream of all keys for the given state and namespace. Modifications to the state during iterating over it keys are not supported.

      • getKeysAndNamespaces

        public <N> Stream<Tuple2<K,​N>> getKeysAndNamespaces​(String state)
        Parameters:
        state - State variable for which existing keys will be returned.
        Returns:
        A stream of all keys for the given state and namespace. Modifications to the state during iterating over it keys are not supported. Implementations go not make any ordering guarantees about the returned tupes. Two records with the same key or namespace may not be returned near each other in the stream.

      • createOrUpdateInternalState

        @Nonnull
public <N,​SV,​SEV,​S extends State,​IS extends S> IS createOrUpdateInternalState​(@Nonnull
                                                                                                      TypeSerializer<N> namespaceSerializer,
                                                                                                      @Nonnull
                                                                                                      StateDescriptor<S,​SV> stateDesc,
                                                                                                      @Nonnull
                                                                                                      StateSnapshotTransformer.StateSnapshotTransformFactory<SEV> snapshotTransformFactory)
                                                                                               throws Exception
        Description copied from interface: KeyedStateFactory
        Creates or updates internal state and returns a new InternalKvState.
        Type Parameters:
        N - The type of the namespace.
        SV - The type of the stored state value.
        SEV - The type of the stored state value or entry for collection types (list or map).
        S - The type of the public API state.
        IS - The type of internal state.
        Parameters:
        namespaceSerializer - TypeSerializer for the state namespace.
        stateDesc - The StateDescriptor that contains the name of the state.
        snapshotTransformFactory - factory of state snapshot transformer.
        Throws:
        Exception

      • createOrUpdateInternalState

        @Nonnull
public <N,​SV,​SEV,​S extends State,​IS extends S> IS createOrUpdateInternalState​(@Nonnull
                                                                                                      TypeSerializer<N> namespaceSerializer,
                                                                                                      @Nonnull
                                                                                                      StateDescriptor<S,​SV> stateDesc,
                                                                                                      @Nonnull
                                                                                                      StateSnapshotTransformer.StateSnapshotTransformFactory<SEV> snapshotTransformFactory,
                                                                                                      boolean allowFutureMetadataUpdates)
                                                                                               throws Exception
        Description copied from interface: KeyedStateFactory
        Creates or updates internal state and returns a new InternalKvState.
        Type Parameters:
        N - The type of the namespace.
        SV - The type of the stored state value.
        SEV - The type of the stored state value or entry for collection types (list or map).
        S - The type of the public API state.
        IS - The type of internal state.
        Parameters:
        namespaceSerializer - TypeSerializer for the state namespace.
        stateDesc - The StateDescriptor that contains the name of the state.
        snapshotTransformFactory - factory of state snapshot transformer.
        allowFutureMetadataUpdates - whether allow metadata to update in the future or not.
        Throws:
        Exception

      • snapshot

        @Nonnull
public RunnableFuture<SnapshotResult<KeyedStateHandle>> snapshot​(long checkpointId,
                                                                 long timestamp,
                                                                 @Nonnull
                                                                 CheckpointStreamFactory streamFactory,
                                                                 @Nonnull
                                                                 CheckpointOptions checkpointOptions)
                                                          throws Exception
        Description copied from interface: Snapshotable
        Operation that writes a snapshot into a stream that is provided by the given CheckpointStreamFactory and returns a @RunnableFuture that gives a state handle to the snapshot. It is up to the implementation if the operation is performed synchronous or asynchronous. In the later case, the returned Runnable must be executed first before obtaining the handle.
        Parameters:
        checkpointId - The ID of the checkpoint.
        timestamp - The timestamp of the checkpoint.
        streamFactory - The factory that we can use for writing our state to streams.
        checkpointOptions - Options for how to perform this checkpoint.
        Returns:
        A runnable future that will yield a StateObject.
        Throws:
        Exception

      • notifyCheckpointComplete

        public void notifyCheckpointComplete​(long checkpointId)
        Description copied from interface: CheckpointListener
        Notifies the listener that the checkpoint with the given checkpointId completed and was committed.

        These notifications are "best effort", meaning they can sometimes be skipped. To behave properly, implementers need to follow the "Checkpoint Subsuming Contract". Please see the class-level JavaDocs for details.

        Please note that checkpoints may generally overlap, so you cannot assume that the notifyCheckpointComplete() call is always for the latest prior checkpoint (or snapshot) that was taken on the function/operator implementing this interface. It might be for a checkpoint that was triggered earlier. Implementing the "Checkpoint Subsuming Contract" (see above) properly handles this situation correctly as well.

        Please note that throwing exceptions from this method will not cause the completed checkpoint to be revoked. Throwing exceptions will typically cause task/job failure and trigger recovery.

        Parameters:
        checkpointId - The ID of the checkpoint that has been completed.

      • notifyCheckpointAborted

        public void notifyCheckpointAborted​(long checkpointId)
        Description copied from interface: CheckpointListener
        This method is called as a notification once a distributed checkpoint has been aborted.

        Important: The fact that a checkpoint has been aborted does NOT mean that the data and artifacts produced between the previous checkpoint and the aborted checkpoint are to be discarded. The expected behavior is as if this checkpoint was never triggered in the first place, and the next successful checkpoint simply covers a longer time span. See the "Checkpoint Subsuming Contract" in the class-level JavaDocs for details.

        These notifications are "best effort", meaning they can sometimes be skipped.

        This method is very rarely necessary to implement. The "best effort" guarantee, together with the fact that this method should not result in discarding any data (per the "Checkpoint Subsuming Contract") means it is mainly useful for earlier cleanups of auxiliary resources. One example is to pro-actively clear a local per-checkpoint state cache upon checkpoint failure.

        Parameters:
        checkpointId - The ID of the checkpoint that has been aborted.

      • numKeyValueStateEntries

        @VisibleForTesting
public int numKeyValueStateEntries()
        Returns the total number of state entries across all keys/namespaces.

      • numKeyValueStateEntries

        @VisibleForTesting
public int numKeyValueStateEntries​(Object namespace)
        Returns the total number of state entries across all keys for the given namespace.