org.apache.flink.client.program

Class MiniClusterClient

java.lang.Object

org.apache.flink.client.program.MiniClusterClient

All Implemented Interfaces:

AutoCloseable, ClusterClient<MiniClusterClient.MiniClusterId>

public class MiniClusterClient
extends Object
implements ClusterClient<MiniClusterClient.MiniClusterId>

Client to interact with a MiniCluster.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MiniClusterClient.MiniClusterId The type of the Cluster ID for the local MiniCluster.

Constructor Summary

Constructors

Constructor and Description
MiniClusterClient(Configuration configuration, MiniCluster miniCluster)

Method Summary

Modifier and Type	Method and Description
CompletableFuture<Acknowledge>	cancel(JobID jobId) Cancels a job identified by the job id.
CompletableFuture<String>	cancelWithSavepoint(JobID jobId, String savepointDirectory, SavepointFormatType formatType) Cancels a job identified by the job id and triggers a savepoint.
void	close()
CompletableFuture<Acknowledge>	disposeSavepoint(String savepointPath) Dispose the savepoint under the given path.
CompletableFuture<Map<String,Object>>	getAccumulators(JobID jobID, ClassLoader loader) Requests and returns the accumulators for the given job identifier.
MiniClusterClient.MiniClusterId	getClusterId() Returns the cluster id identifying the cluster to which the client is connected.
Configuration	getFlinkConfiguration() Return the Flink configuration object.
CompletableFuture<JobStatus>	getJobStatus(JobID jobId) Requests the JobStatus of the job with the given JobID.
String	getWebInterfaceURL() Returns an URL (as a string) to the cluster web interface.
CompletableFuture<Void>	invalidateClusterDataset(AbstractID clusterDatasetId) Invalidate the cached intermediate dataset with the given id.
CompletableFuture<Set<AbstractID>>	listCompletedClusterDatasetIds() Return a set of ids of the completed cluster datasets.
CompletableFuture<Collection<JobStatusMessage>>	listJobs() Lists the currently running and finished jobs on the cluster.
CompletableFuture<Void>	reportHeartbeat(JobID jobId, long expiredTimestamp) The client reports the heartbeat to the dispatcher for aliveness.
CompletableFuture<JobResult>	requestJobResult(JobID jobId) Request the JobResult for the given JobID.
CompletableFuture<CoordinationResponse>	sendCoordinationRequest(JobID jobId, OperatorID operatorId, CoordinationRequest request) Sends out a request to a specified coordinator and return the response.
void	shutDownCluster() Shut down the cluster that this client communicate with.
CompletableFuture<String>	stopWithDetachedSavepoint(JobID jobId, boolean advanceToEndOfEventTime, String savepointDirectory, SavepointFormatType formatType) Stops a program on Flink cluster whose job-manager is configured in this client's configuration.
CompletableFuture<String>	stopWithSavepoint(JobID jobId, boolean advanceToEndOfEventTime, String savepointDirectory, SavepointFormatType formatType) Stops a program on Flink cluster whose job-manager is configured in this client's configuration.
CompletableFuture<JobID>	submitJob(JobGraph jobGraph) Submit the given JobGraph to the cluster.
CompletableFuture<Long>	triggerCheckpoint(JobID jobId, CheckpointType checkpointType) Triggers a checkpoint for the job identified by the job id.
CompletableFuture<String>	triggerDetachedSavepoint(JobID jobId, String savepointDirectory, SavepointFormatType formatType) Triggers a detached savepoint for the job identified by the job id.
CompletableFuture<String>	triggerSavepoint(JobID jobId, String savepointDirectory, SavepointFormatType formatType) Triggers a savepoint for the job identified by the job id.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.client.program.ClusterClient

getAccumulators

Constructor Detail

MiniClusterClient

public MiniClusterClient(@Nonnull
                         Configuration configuration,
                         @Nonnull
                         MiniCluster miniCluster)

Method Detail

getFlinkConfiguration

public Configuration getFlinkConfiguration()

Description copied from interface: ClusterClient

Return the Flink configuration object.

Specified by:

getFlinkConfiguration in interface ClusterClient<MiniClusterClient.MiniClusterId>

Returns:

The Flink configuration object

submitJob

public CompletableFuture<JobID> submitJob(@Nonnull
                                          JobGraph jobGraph)

Description copied from interface: ClusterClient

Submit the given JobGraph to the cluster.

Specified by:

submitJob in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobGraph - to submit

Returns:

JobID of the submitted job

requestJobResult

public CompletableFuture<JobResult> requestJobResult(@Nonnull
                                                     JobID jobId)

Description copied from interface: ClusterClient

Request the JobResult for the given JobID.

Specified by:

requestJobResult in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobId - for which to request the JobResult

Returns:

Future which is completed with the JobResult

cancel

public CompletableFuture<Acknowledge> cancel(JobID jobId)

Description copied from interface: ClusterClient

Cancels a job identified by the job id.

Specified by:

cancel in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobId - the job id

cancelWithSavepoint

public CompletableFuture<String> cancelWithSavepoint(JobID jobId,
                                                     @Nullable
                                                     String savepointDirectory,
                                                     SavepointFormatType formatType)

Description copied from interface: ClusterClient

Cancels a job identified by the job id and triggers a savepoint.

Specified by:

cancelWithSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobId - the job id

savepointDirectory - directory the savepoint should be written to

formatType - a binary format of the savepoint

Returns:

future of path where the savepoint is located

stopWithSavepoint

public CompletableFuture<String> stopWithSavepoint(JobID jobId,
                                                   boolean advanceToEndOfEventTime,
                                                   @Nullable
                                                   String savepointDirectory,
                                                   SavepointFormatType formatType)

Description copied from interface: ClusterClient

Stops a program on Flink cluster whose job-manager is configured in this client's configuration. Stopping works only for streaming programs. Be aware, that the program might continue to run for a while after sending the stop command, because after sources stopped to emit data all operators need to finish processing.

Specified by:

stopWithSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobId - the job ID of the streaming program to stop

advanceToEndOfEventTime - flag indicating if the source should inject a MAX_WATERMARK in the pipeline

savepointDirectory - directory the savepoint should be written to

formatType - a binary format of the savepoint

Returns:

a CompletableFuture containing the path where the savepoint is located

stopWithDetachedSavepoint

public CompletableFuture<String> stopWithDetachedSavepoint(JobID jobId,
                                                           boolean advanceToEndOfEventTime,
                                                           @Nullable
                                                           String savepointDirectory,
                                                           SavepointFormatType formatType)

Description copied from interface: ClusterClient

Stops a program on Flink cluster whose job-manager is configured in this client's configuration. Stopping works only for streaming programs. Be aware, that the program might continue to run for a while after sending the stop command, because after sources stopped to emit data all operators need to finish processing.

Specified by:

stopWithDetachedSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobId - the job ID of the streaming program to stop

advanceToEndOfEventTime - flag indicating if the source should inject a MAX_WATERMARK in the pipeline

savepointDirectory - directory the savepoint should be written to

formatType - a binary format of the savepoint

Returns:

the savepoint trigger id

triggerSavepoint

public CompletableFuture<String> triggerSavepoint(JobID jobId,
                                                  @Nullable
                                                  String savepointDirectory,
                                                  SavepointFormatType formatType)

Description copied from interface: ClusterClient

Triggers a savepoint for the job identified by the job id. The savepoint will be written to the given savepoint directory, or CheckpointingOptions.SAVEPOINT_DIRECTORY if it is null.

Specified by:

triggerSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobId - job id

savepointDirectory - directory the savepoint should be written to

formatType - a binary format of the savepoint

Returns:

path future where the savepoint is located

triggerCheckpoint

public CompletableFuture<Long> triggerCheckpoint(JobID jobId,
                                                 CheckpointType checkpointType)

Description copied from interface: ClusterClient

Triggers a checkpoint for the job identified by the job id. The checkpoint will be written to the checkpoint directory for the job.

Specified by:

triggerCheckpoint in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobId - job id

checkpointType - the checkpoint type (configured / full / incremental)

triggerDetachedSavepoint

public CompletableFuture<String> triggerDetachedSavepoint(JobID jobId,
                                                          @Nullable
                                                          String savepointDirectory,
                                                          SavepointFormatType formatType)

Description copied from interface: ClusterClient

Triggers a detached savepoint for the job identified by the job id. The savepoint will be written to the given savepoint directory, or CheckpointingOptions.SAVEPOINT_DIRECTORY if it is null. Notice that: the detached savepoint will return with a savepoint trigger id instead of the path future, that means the client will return very quickly.

Specified by:

triggerDetachedSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobId - job id

savepointDirectory - directory the savepoint should be written to

formatType - a binary format of the savepoint

Returns:

the savepoint trigger id

disposeSavepoint

public CompletableFuture<Acknowledge> disposeSavepoint(String savepointPath)

Description copied from interface: ClusterClient

Dispose the savepoint under the given path.

Specified by:

disposeSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

savepointPath - path to the savepoint to be disposed

Returns:

acknowledge future of the dispose action

listJobs

public CompletableFuture<Collection<JobStatusMessage>> listJobs()

Description copied from interface: ClusterClient

Lists the currently running and finished jobs on the cluster.

Specified by:

listJobs in interface ClusterClient<MiniClusterClient.MiniClusterId>

Returns:

future collection of running and finished jobs

getAccumulators

public CompletableFuture<Map<String,Object>> getAccumulators(JobID jobID,
                                                             ClassLoader loader)

Description copied from interface: ClusterClient

Requests and returns the accumulators for the given job identifier. Accumulators can be requested while a is running or after it has finished.

Specified by:

getAccumulators in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobID - The job identifier of a job.

loader - The class loader for deserializing the accumulator results.

Returns:

A Map containing the accumulator's name and its value.

getJobStatus

public CompletableFuture<JobStatus> getJobStatus(JobID jobId)

Description copied from interface: ClusterClient

Requests the JobStatus of the job with the given JobID.

Specified by:

getJobStatus in interface ClusterClient<MiniClusterClient.MiniClusterId>

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface ClusterClient<MiniClusterClient.MiniClusterId>

getClusterId

public MiniClusterClient.MiniClusterId getClusterId()

Description copied from interface: ClusterClient

Returns the cluster id identifying the cluster to which the client is connected.

Specified by:

getClusterId in interface ClusterClient<MiniClusterClient.MiniClusterId>

Returns:

cluster id of the connected cluster

shutDownCluster

public void shutDownCluster()

Description copied from interface: ClusterClient

Shut down the cluster that this client communicate with.

Specified by:

shutDownCluster in interface ClusterClient<MiniClusterClient.MiniClusterId>

getWebInterfaceURL

public String getWebInterfaceURL()

Description copied from interface: ClusterClient

Returns an URL (as a string) to the cluster web interface.

Specified by:

getWebInterfaceURL in interface ClusterClient<MiniClusterClient.MiniClusterId>

sendCoordinationRequest

public CompletableFuture<CoordinationResponse> sendCoordinationRequest(JobID jobId,
                                                                       OperatorID operatorId,
                                                                       CoordinationRequest request)

Description copied from interface: ClusterClient

Sends out a request to a specified coordinator and return the response.

Specified by:

sendCoordinationRequest in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobId - specifies the job which the coordinator belongs to

operatorId - specifies which coordinator to receive the request

request - the request to send

Returns:

the response from the coordinator

listCompletedClusterDatasetIds

public CompletableFuture<Set<AbstractID>> listCompletedClusterDatasetIds()

Description copied from interface: ClusterClient

Return a set of ids of the completed cluster datasets.

Specified by:

listCompletedClusterDatasetIds in interface ClusterClient<MiniClusterClient.MiniClusterId>

Returns:

A set of ids of the completely cached intermediate dataset.

invalidateClusterDataset

public CompletableFuture<Void> invalidateClusterDataset(AbstractID clusterDatasetId)

Description copied from interface: ClusterClient

Invalidate the cached intermediate dataset with the given id.

Specified by:

invalidateClusterDataset in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

clusterDatasetId - id of the cluster dataset to be invalidated.

Returns:

Future which will be completed when the cached dataset is invalidated.

reportHeartbeat

public CompletableFuture<Void> reportHeartbeat(JobID jobId,
                                               long expiredTimestamp)

Description copied from interface: ClusterClient

The client reports the heartbeat to the dispatcher for aliveness.

Specified by:

reportHeartbeat in interface ClusterClient<MiniClusterClient.MiniClusterId>

Parameters:

jobId - The jobId for the client and the job.

Returns: