org.apache.flink.contrib.streaming.state

Class AbstractRocksDBState<K,N,V>

java.lang.Object

org.apache.flink.contrib.streaming.state.AbstractRocksDBState<K,N,V>

Type Parameters:

K - The type of the key.

N - The type of the namespace.

V - The type of values kept internally in state.

All Implemented Interfaces:

State, InternalKvState<K,N,V>

public abstract class AbstractRocksDBState<K,N,V>
extends Object
implements InternalKvState<K,N,V>, State

Base class for State implementations that store state in a RocksDB database.

State is not stored in this class but in the RocksDB instance that the EmbeddedRocksDBStateBackend manages and checkpoints.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.state.internal.InternalKvState

InternalKvState.StateIncrementalVisitor<K,N,V>

Field Summary

Fields

Modifier and Type	Field and Description
protected RocksDBKeyedStateBackend<K>	backend Backend that holds the actual RocksDB instance where we store state.
protected org.rocksdb.ColumnFamilyHandle	columnFamily The column family of this particular instance of state.
protected DataInputDeserializer	dataInputView
protected DataOutputSerializer	dataOutputView
protected V	defaultValue
protected org.rocksdb.WriteOptions	writeOptions

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractRocksDBState(org.rocksdb.ColumnFamilyHandle columnFamily, TypeSerializer<N> namespaceSerializer, TypeSerializer<V> valueSerializer, V defaultValue, RocksDBKeyedStateBackend<K> backend) Creates a new RocksDB backed state.

Method Summary

Modifier and Type	Method and Description
void	clear() Removes the value mapped under the current key.
protected V	getDefaultValue()
byte[]	getSerializedValue(byte[] serializedKeyAndNamespace, TypeSerializer<K> safeKeySerializer, TypeSerializer<N> safeNamespaceSerializer, TypeSerializer<V> safeValueSerializer) Returns the serialized value for the given key and namespace.
InternalKvState.StateIncrementalVisitor<K,N,V>	getStateIncrementalVisitor(int recommendedMaxNumberOfReturnedRecords) Get global visitor of state entries.
void	migrateSerializedValue(DataInputDeserializer serializedOldValueInput, DataOutputSerializer serializedMigratedValueOutput, TypeSerializer<V> priorSerializer, TypeSerializer<V> newSerializer)
void	setCurrentNamespace(N namespace) Sets the current namespace, which will be used when using the state access methods.
protected AbstractRocksDBState<K,N,V>	setDefaultValue(V defaultValue)
protected AbstractRocksDBState<K,N,V>	setNamespaceSerializer(TypeSerializer<N> namespaceSerializer)
protected AbstractRocksDBState<K,N,V>	setValueSerializer(TypeSerializer<V> valueSerializer)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.state.internal.InternalKvState

getKeySerializer, getNamespaceSerializer, getValueSerializer

Field Detail

backend

protected RocksDBKeyedStateBackend<K> backend

Backend that holds the actual RocksDB instance where we store state.

columnFamily

protected org.rocksdb.ColumnFamilyHandle columnFamily

The column family of this particular instance of state.

defaultValue

protected V defaultValue

writeOptions

protected final org.rocksdb.WriteOptions writeOptions

dataOutputView

protected final DataOutputSerializer dataOutputView

dataInputView

protected final DataInputDeserializer dataInputView

Constructor Detail

AbstractRocksDBState

protected AbstractRocksDBState(org.rocksdb.ColumnFamilyHandle columnFamily,
                               TypeSerializer<N> namespaceSerializer,
                               TypeSerializer<V> valueSerializer,
                               V defaultValue,
                               RocksDBKeyedStateBackend<K> backend)

Creates a new RocksDB backed state.

Parameters:

columnFamily - The RocksDB column family that this state is associated to.

namespaceSerializer - The serializer for the namespace.

valueSerializer - The serializer for the state.

defaultValue - The default value for the state.

backend - The backend for which this state is bind to.

Method Detail

clear

public void clear()

Description copied from interface: State

Removes the value mapped under the current key.

Specified by:

clear in interface State

setCurrentNamespace

public void setCurrentNamespace(N namespace)

Description copied from interface: InternalKvState

Sets the current namespace, which will be used when using the state access methods.

Specified by:

setCurrentNamespace in interface InternalKvState<K,N,V>

Parameters:

namespace - The namespace.

getSerializedValue

public byte[] getSerializedValue(byte[] serializedKeyAndNamespace,
                                 TypeSerializer<K> safeKeySerializer,
                                 TypeSerializer<N> safeNamespaceSerializer,
                                 TypeSerializer<V> safeValueSerializer)
                          throws Exception

Description copied from interface: InternalKvState

Returns the serialized value for the given key and namespace.

If no value is associated with key and namespace, null is returned.

TO IMPLEMENTERS: This method is called by multiple threads. Anything stateful (e.g. serializers) should be either duplicated or protected from undesired consequences of concurrent invocations.

Specified by:

getSerializedValue in interface InternalKvState<K,N,V>

Parameters:

serializedKeyAndNamespace - Serialized key and namespace

safeKeySerializer - A key serializer which is safe to be used even in multi-threaded context

safeNamespaceSerializer - A namespace serializer which is safe to be used even in multi-threaded context

safeValueSerializer - A value serializer which is safe to be used even in multi-threaded context

Returns:

Serialized value or null if no value is associated with the key and namespace.

Throws:

Exception - Exceptions during serialization are forwarded

migrateSerializedValue

public void migrateSerializedValue(DataInputDeserializer serializedOldValueInput,
                                   DataOutputSerializer serializedMigratedValueOutput,
                                   TypeSerializer<V> priorSerializer,
                                   TypeSerializer<V> newSerializer)
                            throws StateMigrationException

Throws:

StateMigrationException

getDefaultValue

protected V getDefaultValue()

setNamespaceSerializer

protected AbstractRocksDBState<K,N,V> setNamespaceSerializer(TypeSerializer<N> namespaceSerializer)

setValueSerializer

protected AbstractRocksDBState<K,N,V> setValueSerializer(TypeSerializer<V> valueSerializer)

setDefaultValue

protected AbstractRocksDBState<K,N,V> setDefaultValue(V defaultValue)

getStateIncrementalVisitor

public InternalKvState.StateIncrementalVisitor<K,N,V> getStateIncrementalVisitor(int recommendedMaxNumberOfReturnedRecords)

Description copied from interface: InternalKvState

Get global visitor of state entries.

Specified by:

getStateIncrementalVisitor in interface InternalKvState<K,N,V>

Parameters:

recommendedMaxNumberOfReturnedRecords - hint to the visitor not to exceed this number of returned records per nextEntries call, it can still be exceeded by some smaller constant.

Returns:

global iterator over state entries