org.apache.flink.contrib.streaming.state

Class LazyDbValueState<K,N,V>

java.lang.Object

org.apache.flink.contrib.streaming.state.LazyDbValueState<K,N,V>

All Implemented Interfaces:

OperatorState<V>, State, ValueState<V>, CheckpointListener, KvState<K,N,ValueState<V>,ValueStateDescriptor<V>,DbStateBackend>

public class LazyDbValueState<K,N,V>
extends Object
implements KvState<K,N,ValueState<V>,ValueStateDescriptor<V>,DbStateBackend>, ValueState<V>, CheckpointListener

Lazily fetched KvState using a SQL backend. Key-value pairs are cached on heap and are lazily retrieved on access.

Constructor Summary

Constructors

Constructor and Description
LazyDbValueState(String kvStateId, boolean compact, ShardedConnection cons, DbBackendConfig conf, TypeSerializer<K> keySerializer, TypeSerializer<N> namespaceSerializer, ValueStateDescriptor<V> stateDesc) Constructor to initialize the LazyDbValueState the first time the job starts.
LazyDbValueState(String kvStateId, boolean compact, ShardedConnection cons, DbBackendConfig conf, TypeSerializer<K> keySerializer, TypeSerializer<N> namespaceSerializer, ValueStateDescriptor<V> stateDesc, long nextTs, long lastCompactedTs) Initialize the LazyDbValueState from a snapshot.

Method Summary

Modifier and Type	Method and Description
void	clear() Removes the value mapped under the current key.
void	dispose() Disposes the key/value state, releasing all occupied resources.
ExecutorService	getExecutor() Used for testing purposes
Map<Tuple2<K,N>,com.google.common.base.Optional<V>>	getModified() Return the Map of modified states that hasn't been written to the database yet.
Map<Tuple2<K,N>,com.google.common.base.Optional<V>>	getStateCache() Return the Map of cached states.
boolean	isCompactor() Used for testing purposes
void	notifyCheckpointComplete(long checkpointId) This method is called as a notification once a distributed checkpoint has been completed.
void	setCurrentKey(K key) Sets the current key, which will be used when using the state access methods.
void	setCurrentNamespace(N namespace) Sets the current namespace, which will be used when using the state access methods.
int	size() Returns the number of elements currently stored in the task's cache.
org.apache.flink.contrib.streaming.state.LazyDbValueState.DbKvStateSnapshot<K,N,V>	snapshot(long checkpointId, long timestamp) Creates a snapshot of this state.
void	update(V value) Updates the operator state accessible by ValueState.value() to the given value.
V	value() Returns the current value for the state.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LazyDbValueState

public LazyDbValueState(String kvStateId,
                        boolean compact,
                        ShardedConnection cons,
                        DbBackendConfig conf,
                        TypeSerializer<K> keySerializer,
                        TypeSerializer<N> namespaceSerializer,
                        ValueStateDescriptor<V> stateDesc)
                 throws IOException

Constructor to initialize the LazyDbValueState the first time the job starts.

Throws:

IOException

LazyDbValueState

public LazyDbValueState(String kvStateId,
                        boolean compact,
                        ShardedConnection cons,
                        DbBackendConfig conf,
                        TypeSerializer<K> keySerializer,
                        TypeSerializer<N> namespaceSerializer,
                        ValueStateDescriptor<V> stateDesc,
                        long nextTs,
                        long lastCompactedTs)
                 throws IOException

Initialize the LazyDbValueState from a snapshot.

Throws:

IOException

Method Detail

setCurrentKey

public void setCurrentKey(K key)

Description copied from interface: KvState

Sets the current key, which will be used when using the state access methods.

Specified by:

setCurrentKey in interface KvState<K,N,ValueState<V>,ValueStateDescriptor<V>,DbStateBackend>

Parameters:

key - The key.

setCurrentNamespace

public void setCurrentNamespace(N namespace)

Description copied from interface: KvState

Sets the current namespace, which will be used when using the state access methods.

Specified by:

setCurrentNamespace in interface KvState<K,N,ValueState<V>,ValueStateDescriptor<V>,DbStateBackend>

Parameters:

namespace - The namespace.

update

public void update(V value)
            throws IOException

Description copied from interface: ValueState

Updates the operator state accessible by ValueState.value() to the given value. The next time ValueState.value() is called (for the same state partition) the returned state will represent the updated value. When a partitioned state is updated with null, the state for the current key will be removed and the default value is returned on the next access.

Specified by:

update in interface OperatorState<V>

Specified by:

update in interface ValueState<V>

Parameters:

value - The new value for the state.

Throws:

IOException - Thrown if the system cannot access the state.

value

public V value()
        throws IOException

Description copied from interface: ValueState

Returns the current value for the state. When the state is not partitioned the returned value is the same for all inputs in a given operator instance. If state partitioning is applied, the value returned depends on the current operator input, as the operator maintains an independent state for each partition.

Specified by:

value in interface OperatorState<V>

Specified by:

value in interface ValueState<V>

Returns:

The operator state value corresponding to the current input.

Throws:

IOException - Thrown if the system cannot access the state.

clear

public void clear()

Description copied from interface: State

Removes the value mapped under the current key.

Specified by:

clear in interface State

snapshot

public org.apache.flink.contrib.streaming.state.LazyDbValueState.DbKvStateSnapshot<K,N,V> snapshot(long checkpointId,
                                                                                                   long timestamp)
                                                                                            throws IOException

Description copied from interface: KvState

Creates a snapshot of this state.

Specified by:

snapshot in interface KvState<K,N,ValueState<V>,ValueStateDescriptor<V>,DbStateBackend>

Parameters:

checkpointId - The ID of the checkpoint for which the snapshot should be created.

timestamp - The timestamp of the checkpoint.

Returns:

A snapshot handle for this key/value state.

Throws:

IOException

size

public int size()

Returns the number of elements currently stored in the task's cache. Note that the number of elements in the database is not counted here.

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)

Description copied from interface: CheckpointListener

This method is called as a notification once a distributed checkpoint has been completed. Note that any exception during this method will not cause the checkpoint to fail any more.

Specified by:

notifyCheckpointComplete in interface CheckpointListener

Parameters:

checkpointId - The ID of the checkpoint that has been completed.

dispose

public void dispose()

Description copied from interface: KvState

Disposes the key/value state, releasing all occupied resources.

Specified by:

dispose in interface KvState<K,N,ValueState<V>,ValueStateDescriptor<V>,DbStateBackend>

getStateCache

public Map<Tuple2<K,N>,com.google.common.base.Optional<V>> getStateCache()

Return the Map of cached states.

getModified

public Map<Tuple2<K,N>,com.google.common.base.Optional<V>> getModified()

Return the Map of modified states that hasn't been written to the database yet.

isCompactor

public boolean isCompactor()

Used for testing purposes

getExecutor

public ExecutorService getExecutor()

Used for testing purposes