Package org.apache.flink.client.deployment.executors

Class AbstractSessionClusterExecutor<ClusterID,ClientFactory extends ClusterClientFactory<ClusterID>>

java.lang.Object

org.apache.flink.client.deployment.executors.AbstractSessionClusterExecutor<ClusterID,ClientFactory>

Type Parameters:

ClusterID - the type of the id of the cluster.

ClientFactory - the type of the ClusterClientFactory used to create/retrieve a client to the target cluster.

All Implemented Interfaces:

CacheSupportedPipelineExecutor, PipelineExecutor

Direct Known Subclasses:

KubernetesSessionClusterExecutor, RemoteExecutor, YarnSessionClusterExecutor

@Internal
public class AbstractSessionClusterExecutor<ClusterID,ClientFactory extends ClusterClientFactory<ClusterID>>
extends Object
implements CacheSupportedPipelineExecutor

An abstract PipelineExecutor used to execute pipelines on an existing (session) cluster.

Constructor Summary

Constructors

Constructor	Description
AbstractSessionClusterExecutor(ClientFactory clusterClientFactory, Configuration configuration)

Method Summary

Modifier and Type	Method	Description
CompletableFuture<JobClient>	execute(Pipeline pipeline, Configuration configuration, ClassLoader userCodeClassloader)	Executes a Pipeline based on the provided configuration and returns a JobClient which allows to interact with the job being executed, e.g. cancel it or take a savepoint.
CompletableFuture<Void>	invalidateClusterDataset(AbstractID clusterDatasetId, Configuration configuration, ClassLoader userCodeClassloader)	Invalidate the cluster dataset with the given id.
CompletableFuture<Set<AbstractID>>	listCompletedClusterDatasetIds(Configuration configuration, ClassLoader userCodeClassloader)	Return a set of ids of the completed cluster dataset.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AbstractSessionClusterExecutor

public AbstractSessionClusterExecutor(@Nonnull
                                      ClientFactory clusterClientFactory,
                                      Configuration configuration)

Method Detail

execute

public CompletableFuture<JobClient> execute(@Nonnull
                                            Pipeline pipeline,
                                            @Nonnull
                                            Configuration configuration,
                                            @Nonnull
                                            ClassLoader userCodeClassloader)
                                     throws Exception

Description copied from interface: PipelineExecutor

Executes a Pipeline based on the provided configuration and returns a JobClient which allows to interact with the job being executed, e.g. cancel it or take a savepoint.

ATTENTION: The caller is responsible for managing the lifecycle of the returned JobClient. This means that e.g. close() should be called explicitly at the call-site.

Specified by:

execute in interface PipelineExecutor

Parameters:

pipeline - the Pipeline to execute

configuration - the Configuration with the required execution parameters

userCodeClassloader - the ClassLoader to deserialize usercode

Returns:

a CompletableFuture with the JobClient corresponding to the pipeline.

Throws:

Exception

listCompletedClusterDatasetIds

public CompletableFuture<Set<AbstractID>> listCompletedClusterDatasetIds(Configuration configuration,
                                                                         ClassLoader userCodeClassloader)
                                                                  throws Exception

Description copied from interface: CacheSupportedPipelineExecutor

Return a set of ids of the completed cluster dataset.

Specified by:

listCompletedClusterDatasetIds in interface CacheSupportedPipelineExecutor

Parameters:

configuration - the Configuration with the required parameters

userCodeClassloader - the ClassLoader to deserialize usercode

Returns:

A set of ids of the completely cached intermediate dataset.

Throws:

Exception

invalidateClusterDataset

public CompletableFuture<Void> invalidateClusterDataset(AbstractID clusterDatasetId,
                                                        Configuration configuration,
                                                        ClassLoader userCodeClassloader)
                                                 throws Exception

Description copied from interface: CacheSupportedPipelineExecutor

Invalidate the cluster dataset with the given id.

Specified by:

invalidateClusterDataset in interface CacheSupportedPipelineExecutor

Parameters:

clusterDatasetId - id of the cluster dataset to be invalidated.

configuration - the Configuration with the required parameters

userCodeClassloader - the ClassLoader to deserialize usercode

Returns:

Future which will be completed when the cached dataset is invalidated.

Throws:

Exception