org.apache.flink.runtime.util

Class ZooKeeperUtils

java.lang.Object

org.apache.flink.runtime.util.ZooKeeperUtils

public class ZooKeeperUtils
extends Object

Class containing helper functions to interact with ZooKeeper.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ZooKeeperUtils.SecureAclProvider Secure ACLProvider implementation.
static class	ZooKeeperUtils.ZkClientACLMode ZooKeeper client ACL mode enum.

Field Summary

Fields

Modifier and Type	Field and Description
static String	HA_STORAGE_COMPLETED_CHECKPOINT The prefix of the completed checkpoint file.
static String	HA_STORAGE_SUBMITTED_JOBGRAPH_PREFIX The prefix of the submitted job graph file.

Method Summary

Modifier and Type	Method and Description
static ZooKeeperCheckpointIDCounter	createCheckpointIDCounter(org.apache.curator.framework.CuratorFramework client, Configuration configuration, JobID jobId) Creates a ZooKeeperCheckpointIDCounter instance.
static CompletedCheckpointStore	createCompletedCheckpoints(org.apache.curator.framework.CuratorFramework client, Configuration configuration, JobID jobId, int maxNumberOfCheckpointsToRetain, Executor executor) Creates a ZooKeeperCompletedCheckpointStore instance.
static <T extends Serializable> FileSystemStateStorageHelper<T>	createFileSystemStateStorage(Configuration configuration, String prefix) Creates a FileSystemStateStorageHelper instance.
static ZooKeeperJobGraphStore	createJobGraphs(org.apache.curator.framework.CuratorFramework client, Configuration configuration) Creates a ZooKeeperJobGraphStore instance.
static ZooKeeperLeaderElectionService	createLeaderElectionService(org.apache.curator.framework.CuratorFramework client, Configuration configuration) Creates a ZooKeeperLeaderElectionService instance.
static ZooKeeperLeaderElectionService	createLeaderElectionService(org.apache.curator.framework.CuratorFramework client, Configuration configuration, String pathSuffix) Creates a ZooKeeperLeaderElectionService instance.
static ZooKeeperLeaderRetrievalService	createLeaderRetrievalService(org.apache.curator.framework.CuratorFramework client, Configuration configuration) Creates a ZooKeeperLeaderRetrievalService instance.
static ZooKeeperLeaderRetrievalService	createLeaderRetrievalService(org.apache.curator.framework.CuratorFramework client, Configuration configuration, String pathSuffix) Creates a ZooKeeperLeaderRetrievalService instance.
static <T extends Serializable> ZooKeeperStateHandleStore<T>	createZooKeeperStateHandleStore(org.apache.curator.framework.CuratorFramework client, String path, RetrievableStateStorageHelper<T> stateStorage) Creates an instance of ZooKeeperStateHandleStore.
static String	generateZookeeperPath(String root, String namespace)
static String	getZooKeeperEnsemble(Configuration flinkConf) Returns the configured ZooKeeper quorum (and removes whitespace, because ZooKeeper does not tolerate it).
static boolean	isZooKeeperRecoveryMode(Configuration flinkConf) Returns whether HighAvailabilityMode.ZOOKEEPER is configured.
static org.apache.curator.framework.CuratorFramework	startCuratorFramework(Configuration configuration) Starts a CuratorFramework instance and connects it to the given ZooKeeper quorum.
static org.apache.curator.framework.CuratorFramework	useNamespaceAndEnsurePath(org.apache.curator.framework.CuratorFramework client, String path) Returns a facade of the client that uses the specified namespace, and ensures that all nodes in the path exist.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HA_STORAGE_SUBMITTED_JOBGRAPH_PREFIX

public static final String HA_STORAGE_SUBMITTED_JOBGRAPH_PREFIX

The prefix of the submitted job graph file.

See Also:

Constant Field Values

HA_STORAGE_COMPLETED_CHECKPOINT

public static final String HA_STORAGE_COMPLETED_CHECKPOINT

The prefix of the completed checkpoint file.

See Also:

Constant Field Values

Method Detail

startCuratorFramework

public static org.apache.curator.framework.CuratorFramework startCuratorFramework(Configuration configuration)

Starts a CuratorFramework instance and connects it to the given ZooKeeper quorum.

Parameters:

configuration - Configuration object containing the configuration values

Returns:

CuratorFramework instance

isZooKeeperRecoveryMode

public static boolean isZooKeeperRecoveryMode(Configuration flinkConf)

Returns whether HighAvailabilityMode.ZOOKEEPER is configured.

getZooKeeperEnsemble

public static String getZooKeeperEnsemble(Configuration flinkConf)
                                   throws IllegalConfigurationException

Returns the configured ZooKeeper quorum (and removes whitespace, because ZooKeeper does not tolerate it).

Throws:

IllegalConfigurationException

createLeaderRetrievalService

public static ZooKeeperLeaderRetrievalService createLeaderRetrievalService(org.apache.curator.framework.CuratorFramework client,
                                                                           Configuration configuration)

Creates a ZooKeeperLeaderRetrievalService instance.

Parameters:

client - The CuratorFramework ZooKeeper client to use

configuration - Configuration object containing the configuration values

Returns:

ZooKeeperLeaderRetrievalService instance.

createLeaderRetrievalService

public static ZooKeeperLeaderRetrievalService createLeaderRetrievalService(org.apache.curator.framework.CuratorFramework client,
                                                                           Configuration configuration,
                                                                           String pathSuffix)

Creates a ZooKeeperLeaderRetrievalService instance.

Parameters:

client - The CuratorFramework ZooKeeper client to use

configuration - Configuration object containing the configuration values

pathSuffix - The path suffix which we want to append

Returns:

ZooKeeperLeaderRetrievalService instance.

Throws:

Exception

createLeaderElectionService

public static ZooKeeperLeaderElectionService createLeaderElectionService(org.apache.curator.framework.CuratorFramework client,
                                                                         Configuration configuration)
                                                                  throws Exception

Creates a ZooKeeperLeaderElectionService instance.

Parameters:

client - The CuratorFramework ZooKeeper client to use

configuration - Configuration object containing the configuration values

Returns:

ZooKeeperLeaderElectionService instance.

Throws:

Exception

createLeaderElectionService

public static ZooKeeperLeaderElectionService createLeaderElectionService(org.apache.curator.framework.CuratorFramework client,
                                                                         Configuration configuration,
                                                                         String pathSuffix)

Creates a ZooKeeperLeaderElectionService instance.

Parameters:

client - The CuratorFramework ZooKeeper client to use

configuration - Configuration object containing the configuration values

pathSuffix - The path suffix which we want to append

Returns:

ZooKeeperLeaderElectionService instance.

createJobGraphs

public static ZooKeeperJobGraphStore createJobGraphs(org.apache.curator.framework.CuratorFramework client,
                                                     Configuration configuration)
                                              throws Exception

Creates a ZooKeeperJobGraphStore instance.

Parameters:

client - The CuratorFramework ZooKeeper client to use

configuration - Configuration object

Returns:

ZooKeeperJobGraphStore instance

Throws:

Exception - if the submitted job graph store cannot be created

createCompletedCheckpoints

public static CompletedCheckpointStore createCompletedCheckpoints(org.apache.curator.framework.CuratorFramework client,
                                                                  Configuration configuration,
                                                                  JobID jobId,
                                                                  int maxNumberOfCheckpointsToRetain,
                                                                  Executor executor)
                                                           throws Exception

Creates a ZooKeeperCompletedCheckpointStore instance.

Parameters:

client - The CuratorFramework ZooKeeper client to use

configuration - Configuration object

jobId - ID of job to create the instance for

maxNumberOfCheckpointsToRetain - The maximum number of checkpoints to retain

executor - to run ZooKeeper callbacks

Returns:

ZooKeeperCompletedCheckpointStore instance

Throws:

Exception - if the completed checkpoint store cannot be created

createZooKeeperStateHandleStore

public static <T extends Serializable> ZooKeeperStateHandleStore<T> createZooKeeperStateHandleStore(org.apache.curator.framework.CuratorFramework client,
                                                                                                    String path,
                                                                                                    RetrievableStateStorageHelper<T> stateStorage)
                                                                                             throws Exception

Creates an instance of ZooKeeperStateHandleStore.

Type Parameters:

T - Type of state

Parameters:

client - ZK client

path - Path to use for the client namespace

stateStorage - RetrievableStateStorageHelper that persist the actual state and whose returned state handle is then written to ZooKeeper

Returns:

ZooKeeperStateHandleStore instance

Throws:

Exception - ZK errors

createCheckpointIDCounter

public static ZooKeeperCheckpointIDCounter createCheckpointIDCounter(org.apache.curator.framework.CuratorFramework client,
                                                                     Configuration configuration,
                                                                     JobID jobId)

Creates a ZooKeeperCheckpointIDCounter instance.

Parameters:

client - The CuratorFramework ZooKeeper client to use

configuration - Configuration object

jobId - ID of job to create the instance for

Returns:

ZooKeeperCheckpointIDCounter instance

createFileSystemStateStorage

public static <T extends Serializable> FileSystemStateStorageHelper<T> createFileSystemStateStorage(Configuration configuration,
                                                                                                    String prefix)
                                                                                             throws IOException

Creates a FileSystemStateStorageHelper instance.

Type Parameters:

T - Type of the state objects

Parameters:

configuration - Configuration object

prefix - Prefix for the created files

Returns:

FileSystemStateStorageHelper instance

Throws:

IOException - if file system state storage cannot be created

generateZookeeperPath

public static String generateZookeeperPath(String root,
                                           String namespace)

useNamespaceAndEnsurePath

public static org.apache.curator.framework.CuratorFramework useNamespaceAndEnsurePath(org.apache.curator.framework.CuratorFramework client,
                                                                                      String path)
                                                                               throws Exception

Returns a facade of the client that uses the specified namespace, and ensures that all nodes in the path exist.

Parameters:

client - ZK client

path - the new namespace

Returns:

ZK Client that uses the new namespace

Throws:

Exception - ZK errors