org.apache.flink.runtime.jobmanager

Class DefaultJobGraphStore<R extends ResourceVersion<R>>

java.lang.Object

org.apache.flink.runtime.jobmanager.DefaultJobGraphStore<R>

All Implemented Interfaces:

GloballyCleanableResource, LocallyCleanableResource, JobGraphStore, JobGraphStore.JobGraphListener, JobGraphWriter

public class DefaultJobGraphStore<R extends ResourceVersion<R>>
extends Object
implements JobGraphStore, JobGraphStore.JobGraphListener

Default implementation for JobGraphStore. Combined with different StateHandleStore, we could persist the job graphs to various distributed storage. Also combined with different JobGraphStoreWatcher, we could get all the changes on the job graph store and do the response.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.flink.runtime.jobmanager.JobGraphStore

JobGraphStore.JobGraphListener

Constructor Summary

Constructors

Constructor and Description
DefaultJobGraphStore(StateHandleStore<JobGraph,R> stateHandleStore, JobGraphStoreWatcher jobGraphStoreWatcher, JobGraphStoreUtil jobGraphStoreUtil)

Method Summary

Modifier and Type	Method and Description
Collection<JobID>	getJobIds() Get all job ids of submitted job graphs to the submitted job graph store.
CompletableFuture<Void>	globalCleanupAsync(JobID jobId, Executor executor) globalCleanupAsync is expected to be called from the main thread.
CompletableFuture<Void>	localCleanupAsync(JobID jobId, Executor executor) Releases the locks on the specified JobGraph.
void	onAddedJobGraph(JobID jobId) Callback for JobGraph instances added by a different JobGraphStore instance.
void	onRemovedJobGraph(JobID jobId) Callback for JobGraph instances removed by a different JobGraphStore instance.
void	putJobGraph(JobGraph jobGraph) Adds the JobGraph instance.
void	putJobResourceRequirements(JobID jobId, JobResourceRequirements jobResourceRequirements) Persist job resource requirements for the given job.
JobGraph	recoverJobGraph(JobID jobId) Returns the JobGraph with the given JobID or null if no job was registered.
void	start(JobGraphStore.JobGraphListener jobGraphListener) Starts the JobGraphStore service.
void	stop() Stops the JobGraphStore service.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DefaultJobGraphStore

public DefaultJobGraphStore(StateHandleStore<JobGraph,R> stateHandleStore,
                            JobGraphStoreWatcher jobGraphStoreWatcher,
                            JobGraphStoreUtil jobGraphStoreUtil)

Method Detail

start

public void start(JobGraphStore.JobGraphListener jobGraphListener)
           throws Exception

Description copied from interface: JobGraphStore

Starts the JobGraphStore service.

Specified by:

start in interface JobGraphStore

Throws:

Exception

stop

public void stop()
          throws Exception

Description copied from interface: JobGraphStore

Stops the JobGraphStore service.

Specified by:

stop in interface JobGraphStore

Throws:

Exception

recoverJobGraph

@Nullable
public JobGraph recoverJobGraph(JobID jobId)
                                   throws Exception

Description copied from interface: JobGraphStore

Returns the JobGraph with the given JobID or null if no job was registered.

Specified by:

recoverJobGraph in interface JobGraphStore

Throws:

Exception

putJobGraph

public void putJobGraph(JobGraph jobGraph)
                 throws Exception

Description copied from interface: JobGraphWriter

Adds the JobGraph instance.

If a job graph with the same JobID exists, it is replaced.

Specified by:

putJobGraph in interface JobGraphWriter

Throws:

Exception

putJobResourceRequirements

public void putJobResourceRequirements(JobID jobId,
                                       JobResourceRequirements jobResourceRequirements)
                                throws Exception

Description copied from interface: JobGraphWriter

Persist job resource requirements for the given job.

Specified by:

putJobResourceRequirements in interface JobGraphWriter

Parameters:

jobId - job the given requirements belong to

jobResourceRequirements - requirements to persist

Throws:

Exception - in case we're not able to persist the requirements for some reason

globalCleanupAsync

public CompletableFuture<Void> globalCleanupAsync(JobID jobId,
                                                  Executor executor)

Description copied from interface: GloballyCleanableResource

globalCleanupAsync is expected to be called from the main thread. Heavy IO tasks should be outsourced into the passed cleanupExecutor. Thread-safety must be ensured.

Specified by:

globalCleanupAsync in interface GloballyCleanableResource

Specified by:

globalCleanupAsync in interface JobGraphWriter

Parameters:

jobId - The JobID of the job for which the local data should be cleaned up.

executor - The fallback executor for IO-heavy operations.

Returns:

The cleanup result future.

localCleanupAsync

public CompletableFuture<Void> localCleanupAsync(JobID jobId,
                                                 Executor executor)

Releases the locks on the specified JobGraph.

Releasing the locks allows that another instance can delete the job from the JobGraphStore.

Specified by:

localCleanupAsync in interface LocallyCleanableResource

Specified by:

localCleanupAsync in interface JobGraphWriter

Parameters:

jobId - specifying the job to release the locks for

executor - the executor being used for the asynchronous execution of the local cleanup.

Returns:

The cleanup result future.

getJobIds

public Collection<JobID> getJobIds()
                            throws Exception

Description copied from interface: JobGraphStore

Get all job ids of submitted job graphs to the submitted job graph store.

Specified by:

getJobIds in interface JobGraphStore

Returns:

Collection of submitted job ids

Throws:

Exception - if the operation fails

onAddedJobGraph

public void onAddedJobGraph(JobID jobId)

Description copied from interface: JobGraphStore.JobGraphListener

Callback for JobGraph instances added by a different JobGraphStore instance.

Important: It is possible to get false positives and be notified about a job graph, which was added by this instance.

Specified by:

onAddedJobGraph in interface JobGraphStore.JobGraphListener

Parameters:

jobId - The JobID of the added job graph

onRemovedJobGraph

public void onRemovedJobGraph(JobID jobId)

Description copied from interface: JobGraphStore.JobGraphListener

Callback for JobGraph instances removed by a different JobGraphStore instance.

Specified by:

onRemovedJobGraph in interface JobGraphStore.JobGraphListener

Parameters:

jobId - The JobID of the removed job graph