org.apache.flink.runtime.highavailability

Interface HighAvailabilityServices

All Superinterfaces:

AutoCloseable, ClientHighAvailabilityServices

All Known Implementing Classes:

AbstractNonHaServices, EmbeddedHaServices, StandaloneHaServices, ZooKeeperHaServices

public interface HighAvailabilityServices
extends ClientHighAvailabilityServices

The HighAvailabilityServices give access to all services needed for a highly-available setup. In particular, the services provide access to highly available storage and registries, as well as distributed counters and leader election.

• ResourceManager leader election and leader retrieval
• JobManager leader election and leader retrieval
• Persistence for checkpoint metadata
• Registering the latest completed checkpoint(s)
• Persistence for the BLOB store
• Registry that marks a job's status
• Naming of RPC endpoints

Field Summary

Fields

Modifier and Type	Field and Description
static JobID	DEFAULT_JOB_ID This JobID should be used to identify the old JobManager when using the HighAvailabilityServices.
static UUID	DEFAULT_LEADER_ID This UUID should be used when no proper leader election happens, but a simple pre-configured leader is used.

Method Summary

Modifier and Type	Method and Description
void	close() Closes the high availability services, releasing all resources.
void	closeAndCleanupAllData() Closes the high availability services (releasing all resources) and deletes all data stored by these services in external stores.
BlobStore	createBlobStore() Creates the BLOB store in which BLOBs are stored in a highly-available fashion.
CheckpointRecoveryFactory	getCheckpointRecoveryFactory() Gets the checkpoint recovery factory for the job manager.
default LeaderElectionService	getClusterRestEndpointLeaderElectionService() Gets the leader election service for the cluster's rest endpoint.
default LeaderRetrievalService	getClusterRestEndpointLeaderRetriever() Get the leader retriever for the cluster's rest endpoint.
LeaderElectionService	getDispatcherLeaderElectionService() Gets the leader election service for the cluster's dispatcher.
LeaderRetrievalService	getDispatcherLeaderRetriever() Gets the leader retriever for the dispatcher.
JobGraphStore	getJobGraphStore() Gets the submitted job graph store for the job manager.
LeaderElectionService	getJobManagerLeaderElectionService(JobID jobID) Gets the leader election service for the given job.
LeaderRetrievalService	getJobManagerLeaderRetriever(JobID jobID) Deprecated. This method should only be used by the legacy code where the JobManager acts as the master.
LeaderRetrievalService	getJobManagerLeaderRetriever(JobID jobID, String defaultJobManagerAddress) Gets the leader retriever for the job JobMaster which is responsible for the given job.
LeaderElectionService	getResourceManagerLeaderElectionService() Gets the leader election service for the cluster's resource manager.
LeaderRetrievalService	getResourceManagerLeaderRetriever() Gets the leader retriever for the cluster's resource manager.
RunningJobsRegistry	getRunningJobsRegistry() Gets the registry that holds information about whether jobs are currently running.
default LeaderElectionService	getWebMonitorLeaderElectionService() Deprecated. Use getClusterRestEndpointLeaderElectionService() instead.
default LeaderRetrievalService	getWebMonitorLeaderRetriever() Deprecated. just use getClusterRestEndpointLeaderRetriever() instead of this method.

Field Detail

DEFAULT_LEADER_ID

static final UUID DEFAULT_LEADER_ID

This UUID should be used when no proper leader election happens, but a simple pre-configured leader is used. That is for example the case in non-highly-available standalone setups.

DEFAULT_JOB_ID

static final JobID DEFAULT_JOB_ID

This JobID should be used to identify the old JobManager when using the HighAvailabilityServices. With the new mode every JobMaster will have a distinct JobID assigned.

Method Detail

getResourceManagerLeaderRetriever

LeaderRetrievalService getResourceManagerLeaderRetriever()

Gets the leader retriever for the cluster's resource manager.

getDispatcherLeaderRetriever

LeaderRetrievalService getDispatcherLeaderRetriever()

Gets the leader retriever for the dispatcher. This leader retrieval service is not always accessible.

getJobManagerLeaderRetriever

@Deprecated
LeaderRetrievalService getJobManagerLeaderRetriever(JobID jobID)

Deprecated. This method should only be used by the legacy code where the JobManager acts as the master.

Gets the leader retriever for the job JobMaster which is responsible for the given job.

Parameters:

jobID - The identifier of the job.

Returns:

Leader retrieval service to retrieve the job manager for the given job

getJobManagerLeaderRetriever

LeaderRetrievalService getJobManagerLeaderRetriever(JobID jobID,
                                                    String defaultJobManagerAddress)

Gets the leader retriever for the job JobMaster which is responsible for the given job.

Parameters:

jobID - The identifier of the job.

defaultJobManagerAddress - JobManager address which will be returned by a static leader retrieval service.

Returns:

Leader retrieval service to retrieve the job manager for the given job

getWebMonitorLeaderRetriever

@Deprecated
default LeaderRetrievalService getWebMonitorLeaderRetriever()

Deprecated. just use getClusterRestEndpointLeaderRetriever() instead of this method.

This retriever should no longer be used on the cluster side. The web monitor retriever is only required on the client-side and we have a dedicated high-availability services for the client, named ClientHighAvailabilityServices. See also FLINK-13750.

Returns:

the leader retriever for web monitor

getResourceManagerLeaderElectionService

LeaderElectionService getResourceManagerLeaderElectionService()

Gets the leader election service for the cluster's resource manager.

Returns:

Leader election service for the resource manager leader election

getDispatcherLeaderElectionService

LeaderElectionService getDispatcherLeaderElectionService()

Gets the leader election service for the cluster's dispatcher.

Returns:

Leader election service for the dispatcher leader election

getJobManagerLeaderElectionService

LeaderElectionService getJobManagerLeaderElectionService(JobID jobID)

Gets the leader election service for the given job.

Parameters:

jobID - The identifier of the job running the election.

Returns:

Leader election service for the job manager leader election

getWebMonitorLeaderElectionService

@Deprecated
default LeaderElectionService getWebMonitorLeaderElectionService()

Deprecated. Use getClusterRestEndpointLeaderElectionService() instead.

Gets the leader election service for the cluster's rest endpoint.

Returns:

the leader election service used by the cluster's rest endpoint

getCheckpointRecoveryFactory

CheckpointRecoveryFactory getCheckpointRecoveryFactory()

Gets the checkpoint recovery factory for the job manager.

Returns:

Checkpoint recovery factory

getJobGraphStore

JobGraphStore getJobGraphStore()
                        throws Exception

Gets the submitted job graph store for the job manager.

Returns:

Submitted job graph store

Throws:

Exception - if the submitted job graph store could not be created

getRunningJobsRegistry

RunningJobsRegistry getRunningJobsRegistry()
                                    throws Exception

Gets the registry that holds information about whether jobs are currently running.

Returns:

Running job registry to retrieve running jobs

Throws:

Exception

createBlobStore

BlobStore createBlobStore()
                   throws IOException

Creates the BLOB store in which BLOBs are stored in a highly-available fashion.

Returns:

Blob store

Throws:

IOException - if the blob store could not be created

getClusterRestEndpointLeaderElectionService

default LeaderElectionService getClusterRestEndpointLeaderElectionService()

Gets the leader election service for the cluster's rest endpoint.

Returns:

the leader election service used by the cluster's rest endpoint

getClusterRestEndpointLeaderRetriever

default LeaderRetrievalService getClusterRestEndpointLeaderRetriever()

Description copied from interface: ClientHighAvailabilityServices

Get the leader retriever for the cluster's rest endpoint.

Specified by:

getClusterRestEndpointLeaderRetriever in interface ClientHighAvailabilityServices

Returns:

the leader retriever for cluster's rest endpoint.

close

void close()
    throws Exception

Closes the high availability services, releasing all resources.

This method does not delete or clean up any data stored in external stores (file systems, ZooKeeper, etc). Another instance of the high availability services will be able to recover the job.

If an exception occurs during closing services, this method will attempt to continue closing other services and report exceptions only after all services have been attempted to be closed.

Specified by:

close in interface AutoCloseable

Throws:

Exception - Thrown, if an exception occurred while closing these services.

closeAndCleanupAllData

void closeAndCleanupAllData()
                     throws Exception

Closes the high availability services (releasing all resources) and deletes all data stored by these services in external stores.

After this method was called, the any job or session that was managed by these high availability services will be unrecoverable.

If an exception occurs during cleanup, this method will attempt to continue the cleanup and report exceptions only after all cleanup steps have been attempted.

Throws:

Exception - Thrown, if an exception occurred while closing these services or cleaning up data stored by them.