org.apache.flink.contrib.streaming.state

Class RocksDBCachingPriorityQueueSet<E extends HeapPriorityQueueElement>

java.lang.Object

org.apache.flink.contrib.streaming.state.RocksDBCachingPriorityQueueSet<E>

Type Parameters:

E - the type of the contained elements in the queue.

All Implemented Interfaces:

HeapPriorityQueueElement, InternalPriorityQueue<E>

public class RocksDBCachingPriorityQueueSet<E extends HeapPriorityQueueElement>
extends Object
implements InternalPriorityQueue<E>, HeapPriorityQueueElement

A priority queue with set semantics, implemented on top of RocksDB. This uses a TreeSet to cache the bytes of up to the first n elements from RocksDB in memory to reduce interaction with RocksDB, in particular seek operations. Cache uses a simple write-through policy.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	RocksDBCachingPriorityQueueSet.OrderedByteArraySetCache Cache that is organized as an ordered set for byte-arrays.

Field Summary

Fields inherited from interface org.apache.flink.runtime.state.heap.HeapPriorityQueueElement

NOT_CONTAINED

Method Summary

Modifier and Type	Method and Description
boolean	add(E toAdd) Adds the given element to the set, if it is not already contained.
void	addAll(Collection<? extends E> toAdd) Adds all the given elements to the set.
int	getInternalIndex() Returns the current index of this object in the internal array of HeapPriorityQueue.
boolean	isEmpty() Check if the set contains any elements.
CloseableIterator<E>	iterator() Iterator over all elements, no order guaranteed.
E	peek() Retrieves, but does not remove, the element (w.r.t.
E	poll() Retrieves and removes the first element (w.r.t.
boolean	remove(E toRemove) Removes the given element from the set, if is contained in the set.
void	setInternalIndex(int newIndex) Sets the current index of this object in the HeapPriorityQueue and should only be called by the owning HeapPriorityQueue.
int	size() This implementation comes at a relatively high cost per invocation.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

peek

@Nullable
public E peek()

Description copied from interface: InternalPriorityQueue

Retrieves, but does not remove, the element (w.r.t. order) of this set, or returns null if this set is empty.

Specified by:

peek in interface InternalPriorityQueue<E extends HeapPriorityQueueElement>

Returns:

the first element (w.r.t. order) of this ordered set, or null if this set is empty.

poll

@Nullable
public E poll()

Description copied from interface: InternalPriorityQueue

Retrieves and removes the first element (w.r.t. the order) of this set, or returns null if this set is empty.

NOTE: Correct key (i.e. the key of the polled element) must be set on KeyContext before calling this method.

Specified by:

poll in interface InternalPriorityQueue<E extends HeapPriorityQueueElement>

Returns:

the first element of this ordered set, or null if this set is empty.

add

public boolean add(@Nonnull
                   E toAdd)

Description copied from interface: InternalPriorityQueue

Adds the given element to the set, if it is not already contained.

Specified by:

add in interface InternalPriorityQueue<E extends HeapPriorityQueueElement>

Parameters:

toAdd - the element to add to the set.

Returns:

true if the operation changed the head element or if it is unclear if the head element changed. Only returns false if the head element was not changed by this operation.

remove

public boolean remove(@Nonnull
                      E toRemove)

Description copied from interface: InternalPriorityQueue

Removes the given element from the set, if is contained in the set.

NOTE: Correct key (i.e. the key of the polled element) must be set on KeyContext before calling this method.

Specified by:

remove in interface InternalPriorityQueue<E extends HeapPriorityQueueElement>

Parameters:

toRemove - the element to remove.

Returns:

true if the operation changed the head element or if it is unclear if the head element changed. Only returns false if the head element was not changed by this operation.

addAll

public void addAll(@Nullable
                   Collection<? extends E> toAdd)

Description copied from interface: InternalPriorityQueue

Adds all the given elements to the set.

Specified by:

addAll in interface InternalPriorityQueue<E extends HeapPriorityQueueElement>

isEmpty

public boolean isEmpty()

Description copied from interface: InternalPriorityQueue

Check if the set contains any elements.

Specified by:

isEmpty in interface InternalPriorityQueue<E extends HeapPriorityQueueElement>

Returns:

true if the set is empty, i.e. no element is contained.

iterator

@Nonnull
public CloseableIterator<E> iterator()

Description copied from interface: InternalPriorityQueue

Iterator over all elements, no order guaranteed. Iterator must be closed after usage.

Specified by:

iterator in interface InternalPriorityQueue<E extends HeapPriorityQueueElement>

size

public int size()

This implementation comes at a relatively high cost per invocation. It should not be called repeatedly when it is clear that the value did not change. Currently this is only truly used to realize certain higher-level tests.

Specified by:

size in interface InternalPriorityQueue<E extends HeapPriorityQueueElement>

Returns:

the number of elements in this set.

getInternalIndex

public int getInternalIndex()

Description copied from interface: HeapPriorityQueueElement

Returns the current index of this object in the internal array of HeapPriorityQueue.

Specified by:

getInternalIndex in interface HeapPriorityQueueElement

setInternalIndex

public void setInternalIndex(int newIndex)

Description copied from interface: HeapPriorityQueueElement

Sets the current index of this object in the HeapPriorityQueue and should only be called by the owning HeapPriorityQueue.

Specified by:

setInternalIndex in interface HeapPriorityQueueElement

Parameters:

newIndex - the new index in the timer heap.