T
- Type of statepublic class ZooKeeperStateHandleStore<T extends Serializable> extends Object
Added state is persisted via StateHandle
s, which in turn are written to
ZooKeeper. This level of indirection is necessary to keep the amount of data in ZooKeeper
small. ZooKeeper is build for data in the KB range whereas state can grow to multiple MBs.
State modifications require some care, because it is possible that certain failures bring the state handle backend and ZooKeeper out of sync.
ZooKeeper holds the ground truth about state handles, i.e. the following holds:
State handle in ZooKeeper => State handle existsBut not:
State handle exists => State handle in ZooKeeperThere can be lingering state handles when failures happen during operation. They need to be cleaned up manually (see FLINK-2513 about a possible way to overcome this).
Modifier and Type | Field and Description |
---|---|
static org.slf4j.Logger |
LOG |
Constructor and Description |
---|
ZooKeeperStateHandleStore(org.apache.curator.framework.CuratorFramework client,
StateStorageHelper storage)
Creates a
ZooKeeperStateHandleStore . |
Modifier and Type | Method and Description |
---|---|
StateHandle<T> |
add(String pathInZooKeeper,
T state)
Creates a state handle and stores it in ZooKeeper with create mode
CreateMode.PERSISTENT . |
StateHandle<T> |
add(String pathInZooKeeper,
T state,
org.apache.zookeeper.CreateMode createMode)
Creates a state handle and stores it in ZooKeeper.
|
int |
exists(String pathInZooKeeper)
Returns the version of the node if it exists or
-1 if it doesn't. |
StateHandle<T> |
get(String pathInZooKeeper)
Gets a state handle from ZooKeeper.
|
List<Tuple2<StateHandle<T>,String>> |
getAll()
Gets all available state handles from ZooKeeper.
|
List<Tuple2<StateHandle<T>,String>> |
getAllSortedByName()
Gets all available state handles from ZooKeeper sorted by name (ascending).
|
void |
remove(String pathInZooKeeper)
Removes a state handle from ZooKeeper.
|
void |
remove(String pathInZooKeeper,
org.apache.curator.framework.api.BackgroundCallback callback)
Removes a state handle from ZooKeeper asynchronously.
|
void |
removeAndDiscardAllState()
Discards all available state handles and removes them from ZooKeeper.
|
void |
removeAndDiscardState(String pathInZooKeeper)
Discards a state handle and removes it from ZooKeeper.
|
void |
replace(String pathInZooKeeper,
int expectedVersion,
T state)
Replaces a state handle in ZooKeeper and discards the old state handle.
|
public ZooKeeperStateHandleStore(org.apache.curator.framework.CuratorFramework client, StateStorageHelper storage) throws IOException
ZooKeeperStateHandleStore
.client
- The Curator ZooKeeper client. Important: It is
expected that the client's namespace ensures that the root
path is exclusive for all state handles managed by this
instance, e.g. client.usingNamespace("/stateHandles")
IOException
public StateHandle<T> add(String pathInZooKeeper, T state) throws Exception
CreateMode.PERSISTENT
.Exception
add(String, Serializable, CreateMode)
public StateHandle<T> add(String pathInZooKeeper, T state, org.apache.zookeeper.CreateMode createMode) throws Exception
Important: This will not store the actual state in ZooKeeper, but create a state handle and store it in ZooKeeper. This level of indirection makes sure that data in ZooKeeper is small.
pathInZooKeeper
- Destination path in ZooKeeper (expected to *not* exist yet and
start with a '/')state
- State to be addedcreateMode
- The create mode for the new path in ZooKeeperStateHandle
Exception
- If a ZooKeeper or state handle operation failspublic void replace(String pathInZooKeeper, int expectedVersion, T state) throws Exception
pathInZooKeeper
- Destination path in ZooKeeper (expected to exist and start with a '/')expectedVersion
- Expected version of the node to replacestate
- The new state to replace the old oneException
- If a ZooKeeper or state handle operation failspublic int exists(String pathInZooKeeper) throws Exception
-1
if it doesn't.pathInZooKeeper
- Path in ZooKeeper to check-1
otherwise.Exception
- If the ZooKeeper operation failspublic StateHandle<T> get(String pathInZooKeeper) throws Exception
pathInZooKeeper
- Path in ZooKeeper to get the state handle from (expected to
exist and start with a '/').Exception
- If a ZooKeeper or state handle operation failspublic List<Tuple2<StateHandle<T>,String>> getAll() throws Exception
If there is a concurrent modification, the operation is retried until it succeeds.
Exception
- If a ZooKeeper or state handle operation failspublic List<Tuple2<StateHandle<T>,String>> getAllSortedByName() throws Exception
If there is a concurrent modification, the operation is retried until it succeeds.
Exception
- If a ZooKeeper or state handle operation failspublic void remove(String pathInZooKeeper) throws Exception
Important: this does not discard the state handle. If you want to
discard the state handle call removeAndDiscardState(String)
.
pathInZooKeeper
- Path of state handle to remove (expected to start with a '/')Exception
- If the ZooKeeper operation failspublic void remove(String pathInZooKeeper, org.apache.curator.framework.api.BackgroundCallback callback) throws Exception
Important: this does not discard the state handle. If you want to
discard the state handle call removeAndDiscardState(String)
.
pathInZooKeeper
- Path of state handle to remove (expected to start with a '/')callback
- The callback after the operation finishesException
- If the ZooKeeper operation failspublic void removeAndDiscardState(String pathInZooKeeper) throws Exception
If you only want to remove the state handle in ZooKeeper call remove(String)
.
pathInZooKeeper
- Path of state handle to discard (expected to start with a '/')Exception
- If the ZooKeeper or state handle operation failsCopyright © 2014–2017 The Apache Software Foundation. All rights reserved.