Modifier and Type | Method and Description |
---|---|
static JobGraph |
FlinkPipelineTranslationUtil.getJobGraph(Pipeline pipeline,
Configuration optimizerConfiguration,
int defaultParallelism)
|
static JobGraph |
FlinkPipelineTranslationUtil.getJobGraphUnderUserClassLoader(ClassLoader userClassloader,
Pipeline pipeline,
Configuration configuration,
int defaultParallelism)
|
JobGraph |
StreamGraphTranslator.translateToJobGraph(Pipeline pipeline,
Configuration optimizerConfiguration,
int defaultParallelism) |
JobGraph |
FlinkPipelineTranslator.translateToJobGraph(Pipeline pipeline,
Configuration optimizerConfiguration,
int defaultParallelism)
|
JobGraph |
PlanTranslator.translateToJobGraph(Pipeline pipeline,
Configuration optimizerConfiguration,
int defaultParallelism) |
Modifier and Type | Method and Description |
---|---|
ClusterClientProvider<StandaloneClusterId> |
StandaloneClusterDescriptor.deployJobCluster(ClusterSpecification clusterSpecification,
JobGraph jobGraph,
boolean detached) |
ClusterClientProvider<T> |
ClusterDescriptor.deployJobCluster(ClusterSpecification clusterSpecification,
JobGraph jobGraph,
boolean detached)
Deploys a per-job cluster with the given job on the cluster.
|
Modifier and Type | Method and Description |
---|---|
AbstractDispatcherLeaderProcess.DispatcherGatewayService |
ApplicationDispatcherGatewayServiceFactory.create(DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
JobGraphWriter jobGraphWriter) |
Modifier and Type | Method and Description |
---|---|
static JobGraph |
PipelineExecutorUtils.getJobGraph(Pipeline pipeline,
Configuration configuration)
|
Modifier and Type | Method and Description |
---|---|
static JobGraph |
PackagedProgramUtils.createJobGraph(PackagedProgram packagedProgram,
Configuration configuration,
int defaultParallelism,
boolean suppressOutput)
|
static JobGraph |
PackagedProgramUtils.createJobGraph(PackagedProgram packagedProgram,
Configuration configuration,
int defaultParallelism,
JobID jobID,
boolean suppressOutput)
|
Modifier and Type | Method and Description |
---|---|
CompletableFuture<JobID> |
MiniClusterClient.submitJob(JobGraph jobGraph) |
CompletableFuture<JobID> |
ClusterClient.submitJob(JobGraph jobGraph)
Submit the given
JobGraph to the cluster. |
CompletableFuture<JobClient> |
PerJobMiniClusterFactory.submitJob(JobGraph jobGraph,
ClassLoader userCodeClassloader)
Starts a
MiniCluster and submits a job. |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<JobID> |
RestClusterClient.submitJob(JobGraph jobGraph) |
Modifier and Type | Method and Description |
---|---|
ClusterClientProvider<String> |
KubernetesClusterDescriptor.deployJobCluster(ClusterSpecification clusterSpecification,
JobGraph jobGraph,
boolean detached) |
Modifier and Type | Method and Description |
---|---|
static KubernetesStateHandleStore<JobGraph> |
KubernetesUtils.createJobGraphStateHandleStore(Configuration configuration,
FlinkKubeClient flinkKubeClient,
String configMapName,
String lockIdentity)
Create a
KubernetesStateHandleStore which storing JobGraph . |
Modifier and Type | Method and Description |
---|---|
JobGraph |
JobGraphGenerator.compileJobGraph(OptimizedPlan program)
Translates a
OptimizedPlan into a JobGraph . |
JobGraph |
JobGraphGenerator.compileJobGraph(OptimizedPlan program,
JobID jobId) |
Modifier and Type | Method and Description |
---|---|
static void |
ClientUtils.extractAndUploadJobGraphFiles(JobGraph jobGraph,
SupplierWithException<BlobClient,IOException> clientSupplier)
Extracts all files required for the execution from the given
JobGraph and uploads
them using the BlobClient from the given Supplier . |
static void |
ClientUtils.uploadJobGraphFiles(JobGraph jobGraph,
Collection<Path> userJars,
Collection<Tuple2<String,Path>> userArtifacts,
SupplierWithException<BlobClient,IOException> clientSupplier)
Uploads the given jars and artifacts required for the execution of the given
JobGraph
using the BlobClient from the given Supplier . |
Modifier and Type | Method and Description |
---|---|
JobGraph |
SingleJobJobGraphStore.recoverJobGraph(JobID jobId) |
Modifier and Type | Method and Description |
---|---|
JobManagerRunner |
DefaultJobManagerRunnerFactory.createJobManagerRunner(JobGraph jobGraph,
Configuration configuration,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
JobManagerSharedServices jobManagerServices,
JobManagerJobMetricGroupFactory jobManagerJobMetricGroupFactory,
FatalErrorHandler fatalErrorHandler,
long initializationTimestamp) |
JobManagerRunner |
JobManagerRunnerFactory.createJobManagerRunner(JobGraph jobGraph,
Configuration configuration,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
JobManagerSharedServices jobManagerServices,
JobManagerJobMetricGroupFactory jobManagerJobMetricGroupFactory,
FatalErrorHandler fatalErrorHandler,
long initializationTimestamp) |
void |
SingleJobJobGraphStore.putJobGraph(JobGraph jobGraph) |
CompletableFuture<Acknowledge> |
DispatcherGateway.submitJob(JobGraph jobGraph,
Time timeout)
Submit a job to the dispatcher.
|
CompletableFuture<Acknowledge> |
MiniDispatcher.submitJob(JobGraph jobGraph,
Time timeout) |
CompletableFuture<Acknowledge> |
Dispatcher.submitJob(JobGraph jobGraph,
Time timeout) |
Modifier and Type | Method and Description |
---|---|
Dispatcher |
DispatcherFactory.createDispatcher(RpcService rpcService,
DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
PartialDispatcherServicesWithJobGraphStore partialDispatcherServicesWithJobGraphStore)
Create a
Dispatcher . |
MiniDispatcher |
JobDispatcherFactory.createDispatcher(RpcService rpcService,
DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
PartialDispatcherServicesWithJobGraphStore partialDispatcherServicesWithJobGraphStore) |
StandaloneDispatcher |
SessionDispatcherFactory.createDispatcher(RpcService rpcService,
DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
PartialDispatcherServicesWithJobGraphStore partialDispatcherServicesWithJobGraphStore) |
Constructor and Description |
---|
MiniDispatcher(RpcService rpcService,
DispatcherId fencingToken,
DispatcherServices dispatcherServices,
JobGraph jobGraph,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
ClusterEntrypoint.ExecutionMode executionMode) |
SingleJobJobGraphStore(JobGraph jobGraph) |
Constructor and Description |
---|
Dispatcher(RpcService rpcService,
DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
DispatcherServices dispatcherServices) |
StandaloneDispatcher(RpcService rpcService,
DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
DispatcherBootstrapFactory dispatcherBootstrapFactory,
DispatcherServices dispatcherServices) |
Modifier and Type | Method and Description |
---|---|
AbstractDispatcherLeaderProcess.DispatcherGatewayService |
AbstractDispatcherLeaderProcess.DispatcherGatewayServiceFactory.create(DispatcherId fencingToken,
Collection<JobGraph> recoveredJobs,
JobGraphWriter jobGraphWriter) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
FileJobGraphRetriever.retrieveJobGraph(Configuration configuration) |
JobGraph |
JobGraphRetriever.retrieveJobGraph(Configuration configuration)
Retrieve the
JobGraph . |
Modifier and Type | Method and Description |
---|---|
static ExecutionGraph |
ExecutionGraphBuilder.buildGraph(ExecutionGraph prior,
JobGraph jobGraph,
Configuration jobManagerConfig,
ScheduledExecutorService futureExecutor,
Executor ioExecutor,
SlotProvider slotProvider,
ClassLoader classLoader,
CheckpointRecoveryFactory recoveryFactory,
Time rpcTimeout,
RestartStrategy restartStrategy,
MetricGroup metrics,
BlobWriter blobWriter,
Time allocationTimeout,
org.slf4j.Logger log,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
FailoverStrategy.Factory failoverStrategyFactory,
ExecutionDeploymentListener executionDeploymentListener,
ExecutionStateUpdateListener executionStateUpdateListener,
long initializationTimestamp) |
static ExecutionGraph |
ExecutionGraphBuilder.buildGraph(ExecutionGraph prior,
JobGraph jobGraph,
Configuration jobManagerConfig,
ScheduledExecutorService futureExecutor,
Executor ioExecutor,
SlotProvider slotProvider,
ClassLoader classLoader,
CheckpointRecoveryFactory recoveryFactory,
Time rpcTimeout,
RestartStrategy restartStrategy,
MetricGroup metrics,
BlobWriter blobWriter,
Time allocationTimeout,
org.slf4j.Logger log,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
long initializationTimestamp)
Builds the ExecutionGraph from the JobGraph.
|
Modifier and Type | Method and Description |
---|---|
static void |
JobGraphUtils.addUserArtifactEntries(Collection<Tuple2<String,DistributedCache.DistributedCacheEntry>> userArtifacts,
JobGraph jobGraph) |
Modifier and Type | Method and Description |
---|---|
static String |
JsonPlanGenerator.generatePlan(JobGraph jg) |
Constructor and Description |
---|
DefaultLogicalTopology(JobGraph jobGraph) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
JobGraphStore.recoverJobGraph(JobID jobId)
|
JobGraph |
StandaloneJobGraphStore.recoverJobGraph(JobID jobId) |
JobGraph |
DefaultJobGraphStore.recoverJobGraph(JobID jobId) |
Modifier and Type | Method and Description |
---|---|
void |
JobGraphWriter.putJobGraph(JobGraph jobGraph)
Adds the
JobGraph instance. |
void |
ThrowingJobGraphWriter.putJobGraph(JobGraph jobGraph) |
void |
StandaloneJobGraphStore.putJobGraph(JobGraph jobGraph) |
void |
DefaultJobGraphStore.putJobGraph(JobGraph jobGraph) |
Constructor and Description |
---|
DefaultJobGraphStore(StateHandleStore<JobGraph,R> stateHandleStore,
JobGraphStoreWatcher jobGraphStoreWatcher,
JobGraphStoreUtil jobGraphStoreUtil) |
Constructor and Description |
---|
JobManagerRunnerImpl(JobGraph jobGraph,
JobMasterServiceFactory jobMasterFactory,
HighAvailabilityServices haServices,
LibraryCacheManager.ClassLoaderLease classLoaderLease,
Executor executor,
FatalErrorHandler fatalErrorHandler,
long initializationTimestamp)
Exceptions that occur while creating the JobManager or JobManagerRunnerImpl are directly
thrown and not reported to the given
FatalErrorHandler . |
JobMaster(RpcService rpcService,
JobMasterConfiguration jobMasterConfiguration,
ResourceID resourceId,
JobGraph jobGraph,
HighAvailabilityServices highAvailabilityService,
SlotPoolFactory slotPoolFactory,
JobManagerSharedServices jobManagerSharedServices,
HeartbeatServices heartbeatServices,
JobManagerJobMetricGroupFactory jobMetricGroupFactory,
OnCompletionActions jobCompletionActions,
FatalErrorHandler fatalErrorHandler,
ClassLoader userCodeLoader,
SchedulerNGFactory schedulerNGFactory,
ShuffleMaster<?> shuffleMaster,
PartitionTrackerFactory partitionTrackerFactory,
ExecutionDeploymentTracker executionDeploymentTracker,
ExecutionDeploymentReconciler.Factory executionDeploymentReconcilerFactory,
long initializationTimestamp) |
Modifier and Type | Method and Description |
---|---|
JobManagerJobMetricGroup |
JobManagerJobMetricGroupFactory.create(JobGraph jobGraph)
Create a new
JobManagerJobMetricGroup . |
JobManagerJobMetricGroup |
DefaultJobManagerJobMetricGroupFactory.create(JobGraph jobGraph) |
JobManagerJobMetricGroup |
UnregisteredJobManagerJobMetricGroupFactory.create(JobGraph jobGraph) |
JobMaster |
DefaultJobMasterServiceFactory.createJobMasterService(JobGraph jobGraph,
OnCompletionActions jobCompletionActions,
ClassLoader userCodeClassloader,
long initializationTimestamp) |
JobMasterService |
JobMasterServiceFactory.createJobMasterService(JobGraph jobGraph,
OnCompletionActions jobCompletionActions,
ClassLoader userCodeClassloader,
long initializationTimestamp) |
Modifier and Type | Method and Description |
---|---|
JobManagerJobMetricGroup |
UnregisteredMetricGroups.UnregisteredJobManagerMetricGroup.addJob(JobGraph job) |
JobManagerJobMetricGroup |
JobManagerMetricGroup.addJob(JobGraph job) |
Modifier and Type | Method and Description |
---|---|
JobExecutionResult |
MiniCluster.executeJobBlocking(JobGraph job)
This method runs a job in blocking mode.
|
void |
MiniCluster.runDetached(JobGraph job)
This method executes a job in detached mode.
|
CompletableFuture<JobSubmissionResult> |
MiniCluster.submitJob(JobGraph jobGraph) |
Modifier and Type | Method and Description |
---|---|
protected JobGraph |
SchedulerBase.getJobGraph() |
Modifier and Type | Method and Description |
---|---|
SchedulerNG |
SchedulerNGFactory.createInstance(org.slf4j.Logger log,
JobGraph jobGraph,
BackPressureStatsTracker backPressureStatsTracker,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotPool slotPool,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
ExecutionDeploymentTracker executionDeploymentTracker,
long initializationTimestamp) |
SchedulerNG |
DefaultSchedulerFactory.createInstance(org.slf4j.Logger log,
JobGraph jobGraph,
BackPressureStatsTracker backPressureStatsTracker,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotPool slotPool,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
ExecutionDeploymentTracker executionDeploymentTracker,
long initializationTimestamp) |
Constructor and Description |
---|
SchedulerBase(org.slf4j.Logger log,
JobGraph jobGraph,
BackPressureStatsTracker backPressureStatsTracker,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotProvider slotProvider,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
RestartStrategyFactory restartStrategyFactory,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
ExecutionVertexVersioner executionVertexVersioner,
ExecutionDeploymentTracker executionDeploymentTracker,
boolean legacyScheduling,
long initializationTimestamp) |
Constructor and Description |
---|
JarPlanHandler(GatewayRetriever<? extends RestfulGateway> leaderRetriever,
Time timeout,
Map<String,String> responseHeaders,
MessageHeaders<JarPlanRequestBody,JobPlanInfo,org.apache.flink.runtime.webmonitor.handlers.JarPlanMessageParameters> messageHeaders,
Path jarDir,
Configuration configuration,
Executor executor,
java.util.function.Function<JobGraph,JobPlanInfo> planGenerator) |
Modifier and Type | Method and Description |
---|---|
JobGraph |
JarHandlerUtils.JarHandlerContext.toJobGraph(PackagedProgram packagedProgram,
Configuration configuration,
boolean suppressOutput) |
Modifier and Type | Method and Description |
---|---|
static JobGraph |
StreamingJobGraphGenerator.createJobGraph(StreamGraph streamGraph) |
static JobGraph |
StreamingJobGraphGenerator.createJobGraph(StreamGraph streamGraph,
JobID jobID) |
JobGraph |
StreamGraph.getJobGraph()
|
JobGraph |
StreamGraph.getJobGraph(JobID jobID)
|
Modifier and Type | Method and Description |
---|---|
ClusterClientProvider<org.apache.hadoop.yarn.api.records.ApplicationId> |
YarnClusterDescriptor.deployJobCluster(ClusterSpecification clusterSpecification,
JobGraph jobGraph,
boolean detached) |
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.