org.apache.flink.runtime.state

Class AbstractKeyedStateBackend<K>

java.lang.Object

org.apache.flink.runtime.state.AbstractKeyedStateBackend<K>

Type Parameters:

K - Type of the key by which state is keyed.

All Implemented Interfaces:

Closeable, AutoCloseable, CheckpointListener, InternalKeyContext<K>, KeyedStateBackend<K>, KeyedStateFactory, PriorityQueueSetFactory, SnapshotStrategy<SnapshotResult<KeyedStateHandle>>, Disposable

Direct Known Subclasses:

HeapKeyedStateBackend, RocksDBKeyedStateBackend

public abstract class AbstractKeyedStateBackend<K>
extends Object
implements KeyedStateBackend<K>, SnapshotStrategy<SnapshotResult<KeyedStateHandle>>, Closeable, CheckpointListener

Base implementation of KeyedStateBackend. The state can be checkpointed to streams using SnapshotStrategy.snapshot(long, long, CheckpointStreamFactory, CheckpointOptions).

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.state.KeyedStateBackend

KeyedStateBackend.KeySelectionListener<K>

Field Summary

Fields

Modifier and Type	Field and Description
protected CloseableRegistry	cancelStreamRegistry Registry for all opened streams, so they can be closed if the task using this backend is closed.
protected StreamCompressionDecorator	keyGroupCompressionDecorator Decorates the input and output streams to write key-groups compressed.
protected KeyGroupRange	keyGroupRange Range of key-groups for which this backend is responsible.
protected StateSerializerProvider<K>	keySerializerProvider StateSerializerProvider for our key serializer.
protected TaskKvStateRegistry	kvStateRegistry KvStateRegistry helper for this task.
protected int	numberOfKeyGroups The number of key-groups aka max parallelism.
protected TtlTimeProvider	ttlTimeProvider
protected ClassLoader	userCodeClassLoader

Constructor Summary

Constructors

Constructor and Description
AbstractKeyedStateBackend(TaskKvStateRegistry kvStateRegistry, StateSerializerProvider<K> keySerializerProvider, ClassLoader userCodeClassLoader, int numberOfKeyGroups, KeyGroupRange keyGroupRange, ExecutionConfig executionConfig, TtlTimeProvider ttlTimeProvider, CloseableRegistry cancelStreamRegistry, StreamCompressionDecorator keyGroupCompressionDecorator)
AbstractKeyedStateBackend(TaskKvStateRegistry kvStateRegistry, TypeSerializer<K> keySerializer, ClassLoader userCodeClassLoader, int numberOfKeyGroups, KeyGroupRange keyGroupRange, ExecutionConfig executionConfig, TtlTimeProvider ttlTimeProvider, CloseableRegistry cancelStreamRegistry)

Method Summary

Modifier and Type	Method and Description
<N,S extends State,T> void	applyToAllKeys(N namespace, TypeSerializer<N> namespaceSerializer, StateDescriptor<S,T> stateDescriptor, KeyedStateFunction<K,S> function) Applies the provided KeyedStateFunction to the state with the provided StateDescriptor of all the currently active keys.
void	close()
boolean	deregisterKeySelectionListener(KeyedStateBackend.KeySelectionListener<K> listener) Stop calling listener registered in KeyedStateBackend.registerKeySelectionListener(org.apache.flink.runtime.state.KeyedStateBackend.KeySelectionListener<K>).
void	dispose() Closes the state backend, releasing all internal resources, but does not delete any persistent checkpoint data.
K	getCurrentKey() Used by states to access the current key.
int	getCurrentKeyGroupIndex() Returns the key-group to which the current key belongs.
StreamCompressionDecorator	getKeyGroupCompressionDecorator()
KeyGroupRange	getKeyGroupRange() Returns the key groups for this backend.
TypeSerializer<K>	getKeySerializer() TypeSerializer for the state backend key type.
int	getNumberOfKeyGroups() Returns the number of key-groups aka max parallelism.
<N,S extends State,V> S	getOrCreateKeyedState(TypeSerializer<N> namespaceSerializer, StateDescriptor<S,V> stateDescriptor) Creates or retrieves a keyed state backed by this state backend.
<N,S extends State> S	getPartitionedState(N namespace, TypeSerializer<N> namespaceSerializer, StateDescriptor<S,?> stateDescriptor) TODO: NOTE: This method does a lot of work caching / retrieving states just to update the namespace.
abstract int	numKeyValueStateEntries() Returns the total number of state entries across all keys/namespaces.
void	registerKeySelectionListener(KeyedStateBackend.KeySelectionListener<K> listener) State backend will call KeyedStateBackend.KeySelectionListener.keySelected(K) when key context is switched if supported.
boolean	requiresLegacySynchronousTimerSnapshots()
void	setCurrentKey(K newKey) Sets the current key that is used for partitioned state.
boolean	supportsAsynchronousSnapshots()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.state.KeyedStateBackend

getKeys

Methods inherited from interface org.apache.flink.runtime.state.KeyedStateFactory

createInternalState, createInternalState

Methods inherited from interface org.apache.flink.runtime.state.PriorityQueueSetFactory

create

Methods inherited from interface org.apache.flink.runtime.state.SnapshotStrategy

snapshot

Methods inherited from interface org.apache.flink.runtime.state.CheckpointListener

notifyCheckpointComplete

Field Detail

keySerializerProvider

protected final StateSerializerProvider<K> keySerializerProvider

StateSerializerProvider for our key serializer.

numberOfKeyGroups

protected final int numberOfKeyGroups

The number of key-groups aka max parallelism.

keyGroupRange

protected final KeyGroupRange keyGroupRange

Range of key-groups for which this backend is responsible.

kvStateRegistry

protected final TaskKvStateRegistry kvStateRegistry

KvStateRegistry helper for this task.

cancelStreamRegistry

protected CloseableRegistry cancelStreamRegistry

Registry for all opened streams, so they can be closed if the task using this backend is closed.

userCodeClassLoader

protected final ClassLoader userCodeClassLoader

ttlTimeProvider

protected final TtlTimeProvider ttlTimeProvider

keyGroupCompressionDecorator

protected final StreamCompressionDecorator keyGroupCompressionDecorator

Decorates the input and output streams to write key-groups compressed.

Constructor Detail

AbstractKeyedStateBackend

public AbstractKeyedStateBackend(TaskKvStateRegistry kvStateRegistry,
                                 TypeSerializer<K> keySerializer,
                                 ClassLoader userCodeClassLoader,
                                 int numberOfKeyGroups,
                                 KeyGroupRange keyGroupRange,
                                 ExecutionConfig executionConfig,
                                 TtlTimeProvider ttlTimeProvider,
                                 CloseableRegistry cancelStreamRegistry)

AbstractKeyedStateBackend

public AbstractKeyedStateBackend(TaskKvStateRegistry kvStateRegistry,
                                 StateSerializerProvider<K> keySerializerProvider,
                                 ClassLoader userCodeClassLoader,
                                 int numberOfKeyGroups,
                                 KeyGroupRange keyGroupRange,
                                 ExecutionConfig executionConfig,
                                 TtlTimeProvider ttlTimeProvider,
                                 CloseableRegistry cancelStreamRegistry,
                                 StreamCompressionDecorator keyGroupCompressionDecorator)

Method Detail

dispose

public void dispose()

Closes the state backend, releasing all internal resources, but does not delete any persistent checkpoint data.

Specified by:

dispose in interface KeyedStateBackend<K>

Specified by:

dispose in interface Disposable

setCurrentKey

public void setCurrentKey(K newKey)

Description copied from interface: KeyedStateBackend

Sets the current key that is used for partitioned state.

Specified by:

setCurrentKey in interface KeyedStateBackend<K>

Parameters:

newKey - The new current key.

See Also:

KeyedStateBackend

registerKeySelectionListener

public void registerKeySelectionListener(KeyedStateBackend.KeySelectionListener<K> listener)

Description copied from interface: KeyedStateBackend

State backend will call KeyedStateBackend.KeySelectionListener.keySelected(K) when key context is switched if supported.

Specified by:

registerKeySelectionListener in interface KeyedStateBackend<K>

deregisterKeySelectionListener

public boolean deregisterKeySelectionListener(KeyedStateBackend.KeySelectionListener<K> listener)

Description copied from interface: KeyedStateBackend

Stop calling listener registered in KeyedStateBackend.registerKeySelectionListener(org.apache.flink.runtime.state.KeyedStateBackend.KeySelectionListener<K>).

Specified by:

deregisterKeySelectionListener in interface KeyedStateBackend<K>

Returns:

returns true iff listener was registered before.

getKeySerializer

public TypeSerializer<K> getKeySerializer()

Description copied from interface: InternalKeyContext

TypeSerializer for the state backend key type.

Specified by:

getKeySerializer in interface InternalKeyContext<K>

See Also:

KeyedStateBackend

getCurrentKey

public K getCurrentKey()

Description copied from interface: InternalKeyContext

Used by states to access the current key.

Specified by:

getCurrentKey in interface InternalKeyContext<K>

See Also:

KeyedStateBackend

getCurrentKeyGroupIndex

public int getCurrentKeyGroupIndex()

Description copied from interface: InternalKeyContext

Returns the key-group to which the current key belongs.

Specified by:

getCurrentKeyGroupIndex in interface InternalKeyContext<K>

See Also:

KeyedStateBackend

getNumberOfKeyGroups

public int getNumberOfKeyGroups()

Description copied from interface: InternalKeyContext

Returns the number of key-groups aka max parallelism.

Specified by:

getNumberOfKeyGroups in interface InternalKeyContext<K>

See Also:

KeyedStateBackend

getKeyGroupRange

public KeyGroupRange getKeyGroupRange()

Description copied from interface: InternalKeyContext

Returns the key groups for this backend.

Specified by:

getKeyGroupRange in interface InternalKeyContext<K>

See Also:

KeyedStateBackend

applyToAllKeys

public <N,S extends State,T> void applyToAllKeys(N namespace,
                                                 TypeSerializer<N> namespaceSerializer,
                                                 StateDescriptor<S,T> stateDescriptor,
                                                 KeyedStateFunction<K,S> function)
                                          throws Exception

Description copied from interface: KeyedStateBackend

Applies the provided KeyedStateFunction to the state with the provided StateDescriptor of all the currently active keys.

Specified by:

applyToAllKeys in interface KeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

S - The type of the state.

Parameters:

namespace - the namespace of the state.

namespaceSerializer - the serializer for the namespace.

stateDescriptor - the descriptor of the state to which the function is going to be applied.

function - the function to be applied to the keyed state.

Throws:

Exception

See Also:

KeyedStateBackend

getOrCreateKeyedState

public <N,S extends State,V> S getOrCreateKeyedState(TypeSerializer<N> namespaceSerializer,
                                                     StateDescriptor<S,V> stateDescriptor)
                                              throws Exception

Description copied from interface: KeyedStateBackend

Creates or retrieves a keyed state backed by this state backend.

Specified by:

getOrCreateKeyedState in interface KeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

S - The type of the state.

Parameters:

namespaceSerializer - The serializer used for the namespace type of the state

stateDescriptor - The identifier for the state. This contains name and can create a default state value.

Returns:

A new key/value state backed by this backend.

Throws:

Exception - Exceptions may occur during initialization of the state and should be forwarded.

See Also:

KeyedStateBackend

getPartitionedState

public <N,S extends State> S getPartitionedState(N namespace,
                                                 TypeSerializer<N> namespaceSerializer,
                                                 StateDescriptor<S,?> stateDescriptor)
                                          throws Exception

TODO: NOTE: This method does a lot of work caching / retrieving states just to update the namespace. This method should be removed for the sake of namespaces being lazily fetched from the keyed state backend, or being set on the state directly.

Specified by:

getPartitionedState in interface KeyedStateBackend<K>

Type Parameters:

N - The type of the namespace.

S - The type of the state.

stateDescriptor - The identifier for the state. This contains name and can create a default state value.

Returns:

A new key/value state backed by this backend.

Throws:

Exception - Exceptions may occur during initialization of the state and should be forwarded.

See Also:

KeyedStateBackend

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

supportsAsynchronousSnapshots

@VisibleForTesting
public boolean supportsAsynchronousSnapshots()

getKeyGroupCompressionDecorator

@VisibleForTesting
public StreamCompressionDecorator getKeyGroupCompressionDecorator()

numKeyValueStateEntries

@VisibleForTesting
public abstract int numKeyValueStateEntries()

Returns the total number of state entries across all keys/namespaces.

requiresLegacySynchronousTimerSnapshots

public boolean requiresLegacySynchronousTimerSnapshots()