org.apache.flink.kubernetes.highavailability

Class KubernetesMultipleComponentLeaderElectionHaServices

java.lang.Object

org.apache.flink.runtime.highavailability.AbstractHaServices

org.apache.flink.kubernetes.highavailability.KubernetesMultipleComponentLeaderElectionHaServices

All Implemented Interfaces:

AutoCloseable, GloballyCleanableResource, ClientHighAvailabilityServices, HighAvailabilityServices

public class KubernetesMultipleComponentLeaderElectionHaServices
extends AbstractHaServices

Kubernetes HA services that use a single leader election service per JobManager.

Field Summary

Fields inherited from class org.apache.flink.runtime.highavailability.AbstractHaServices

configuration, ioExecutor, logger

Fields inherited from interface org.apache.flink.runtime.highavailability.HighAvailabilityServices

DEFAULT_JOB_ID, DEFAULT_LEADER_ID

Method Summary

Modifier and Type	Method and Description
protected CheckpointRecoveryFactory	createCheckpointRecoveryFactory() Create the checkpoint recovery factory for the job manager.
protected JobGraphStore	createJobGraphStore() Create the submitted job graph store for the job manager.
protected LeaderElectionService	createLeaderElectionService(String leaderName) Create leader election service with specified leaderName.
protected LeaderRetrievalService	createLeaderRetrievalService(String componentId) Create leader retrieval service with specified leaderName.
protected String	getLeaderPathForDispatcher() Get the leader path for Dispatcher.
protected String	getLeaderPathForJobManager(JobID jobID) Get the leader path for specific JobManager.
protected String	getLeaderPathForResourceManager() Get the leader path for ResourceManager.
protected String	getLeaderPathForRestServer() Get the leader path for RestServer.
void	internalCleanup() Clean up the meta data in the distributed system(e.g.
void	internalCleanupJobData(JobID jobID) Clean up the meta data in the distributed system(e.g.
void	internalClose() Closes the components which is used for external operations(e.g.

Methods inherited from class org.apache.flink.runtime.highavailability.AbstractHaServices

close, closeAndCleanupAllData, createBlobStore, getCheckpointRecoveryFactory, getClusterRestEndpointLeaderElectionService, getClusterRestEndpointLeaderRetriever, getDispatcherLeaderElectionService, getDispatcherLeaderRetriever, getJobGraphStore, getJobManagerLeaderElectionService, getJobManagerLeaderRetriever, getJobManagerLeaderRetriever, getJobResultStore, getResourceManagerLeaderElectionService, getResourceManagerLeaderRetriever, globalCleanupAsync

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.runtime.highavailability.HighAvailabilityServices

getWebMonitorLeaderElectionService, getWebMonitorLeaderRetriever

Method Detail

createLeaderElectionService

protected LeaderElectionService createLeaderElectionService(String leaderName)

Description copied from class: AbstractHaServices

Create leader election service with specified leaderName.

Specified by:

createLeaderElectionService in class AbstractHaServices

Parameters:

leaderName - ConfigMap name in Kubernetes or child node path in Zookeeper.

Returns:

Return LeaderElectionService using Zookeeper or Kubernetes.

createLeaderRetrievalService

protected LeaderRetrievalService createLeaderRetrievalService(String componentId)

Description copied from class: AbstractHaServices

Create leader retrieval service with specified leaderName.

Specified by:

createLeaderRetrievalService in class AbstractHaServices

Parameters:

componentId - ConfigMap name in Kubernetes or child node path in Zookeeper.

Returns:

Return LeaderRetrievalService using Zookeeper or Kubernetes.

createCheckpointRecoveryFactory

protected CheckpointRecoveryFactory createCheckpointRecoveryFactory()

Description copied from class: AbstractHaServices

Create the checkpoint recovery factory for the job manager.

Specified by:

createCheckpointRecoveryFactory in class AbstractHaServices

Returns:

Checkpoint recovery factory

createJobGraphStore

protected JobGraphStore createJobGraphStore()
                                     throws Exception

Description copied from class: AbstractHaServices

Create the submitted job graph store for the job manager.

Specified by:

createJobGraphStore in class AbstractHaServices

Returns:

Submitted job graph store

Throws:

Exception - if the submitted job graph store could not be created

internalClose

public void internalClose()
                   throws Exception

Description copied from class: AbstractHaServices

Closes the components which is used for external operations(e.g. Zookeeper Client, Kubernetes Client).

Specified by:

internalClose in class AbstractHaServices

Throws:

Exception - if the close operation failed

internalCleanup

public void internalCleanup()
                     throws Exception

Description copied from class: AbstractHaServices

Clean up the meta data in the distributed system(e.g. Zookeeper, Kubernetes ConfigMap).

If an exception occurs during internal cleanup, we will continue the cleanup in AbstractHaServices.closeAndCleanupAllData() and report exceptions only after all cleanup steps have been attempted.

Specified by:

internalCleanup in class AbstractHaServices

Throws:

Exception - when do the cleanup operation on external storage.

internalCleanupJobData

public void internalCleanupJobData(JobID jobID)
                            throws Exception

Description copied from class: AbstractHaServices

Clean up the meta data in the distributed system(e.g. Zookeeper, Kubernetes ConfigMap) for the specified Job. Method implementations need to be thread-safe.

Specified by:

internalCleanupJobData in class AbstractHaServices

Parameters:

jobID - The identifier of the job to cleanup.

Throws:

Exception - when do the cleanup operation on external storage.

getLeaderPathForResourceManager

protected String getLeaderPathForResourceManager()

Description copied from class: AbstractHaServices

Get the leader path for ResourceManager.

Specified by:

getLeaderPathForResourceManager in class AbstractHaServices

Returns:

Return the ResourceManager leader name. It is ConfigMap name in Kubernetes or child node path in Zookeeper.

getLeaderPathForDispatcher

protected String getLeaderPathForDispatcher()

Description copied from class: AbstractHaServices

Get the leader path for Dispatcher.

Specified by:

getLeaderPathForDispatcher in class AbstractHaServices

Returns:

Return the Dispatcher leader name. It is ConfigMap name in Kubernetes or child node path in Zookeeper.

getLeaderPathForJobManager

protected String getLeaderPathForJobManager(JobID jobID)

Description copied from class: AbstractHaServices

Get the leader path for specific JobManager.

Specified by:

getLeaderPathForJobManager in class AbstractHaServices

Parameters:

jobID - job id

Returns:

Return the JobManager leader name for specified job id. It is ConfigMap name in Kubernetes or child node path in Zookeeper.

getLeaderPathForRestServer

protected String getLeaderPathForRestServer()

Description copied from class: AbstractHaServices

Get the leader path for RestServer.

Specified by:

getLeaderPathForRestServer in class AbstractHaServices

Returns:

Return the RestServer leader name. It is ConfigMap name in Kubernetes or child node path in Zookeeper.