Modifier and Type | Method and Description |
---|---|
static DefaultExecutionGraph |
DefaultExecutionGraphBuilder.buildGraph(JobGraph jobGraph,
Configuration jobManagerConfig,
ScheduledExecutorService futureExecutor,
Executor ioExecutor,
ClassLoader classLoader,
CompletedCheckpointStore completedCheckpointStore,
CheckpointsCleaner checkpointsCleaner,
CheckpointIDCounter checkpointIdCounter,
Time rpcTimeout,
BlobWriter blobWriter,
org.slf4j.Logger log,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
TaskDeploymentDescriptorFactory.PartitionLocationConstraint partitionLocationConstraint,
ExecutionDeploymentListener executionDeploymentListener,
ExecutionStateUpdateListener executionStateUpdateListener,
long initializationTimestamp,
VertexAttemptNumberStore vertexAttemptNumberStore,
VertexParallelismStore vertexParallelismStore,
java.util.function.Supplier<CheckpointStatsTracker> checkpointStatsTrackerFactory,
boolean isDynamicGraph,
ExecutionJobVertex.Factory executionJobVertexFactory) |
Constructor and Description |
---|
DefaultExecutionGraph(JobInformation jobInformation,
ScheduledExecutorService futureExecutor,
Executor ioExecutor,
Time rpcTimeout,
int executionHistorySizeLimit,
ClassLoader userClassLoader,
BlobWriter blobWriter,
PartitionGroupReleaseStrategy.Factory partitionGroupReleaseStrategyFactory,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
TaskDeploymentDescriptorFactory.PartitionLocationConstraint partitionLocationConstraint,
ExecutionDeploymentListener executionDeploymentListener,
ExecutionStateUpdateListener executionStateUpdateListener,
long initializationTimestamp,
VertexAttemptNumberStore initialAttemptCounts,
VertexParallelismStore vertexParallelismStore,
boolean isDynamic,
ExecutionJobVertex.Factory executionJobVertexFactory,
List<JobStatusHook> jobStatusHooks) |
Modifier and Type | Interface and Description |
---|---|
interface |
MutableVertexParallelismStore
Mutability extension to the
VertexParallelismStore . |
Modifier and Type | Class and Description |
---|---|
class |
DefaultVertexParallelismStore
Maintains the configured parallelisms for vertices, which should be defined by a scheduler.
|
Modifier and Type | Method and Description |
---|---|
static VertexParallelismStore |
SchedulerBase.computeVertexParallelismStore(Iterable<JobVertex> vertices)
Compute the
VertexParallelismStore for all given vertices, which will set defaults
and ensure that the returned store contains valid parallelisms. |
static VertexParallelismStore |
SchedulerBase.computeVertexParallelismStore(Iterable<JobVertex> vertices,
java.util.function.Function<JobVertex,Integer> defaultMaxParallelismFunc) |
static VertexParallelismStore |
SchedulerBase.computeVertexParallelismStore(Iterable<JobVertex> vertices,
java.util.function.Function<JobVertex,Integer> defaultMaxParallelismFunc,
java.util.function.Function<Integer,Integer> normalizeParallelismFunc)
Compute the
VertexParallelismStore for all given vertices, which will set defaults
and ensure that the returned store contains valid parallelisms, with a custom function for
default max parallelism calculation and a custom function for normalizing vertex parallelism. |
static VertexParallelismStore |
SchedulerBase.computeVertexParallelismStore(JobGraph jobGraph)
Compute the
VertexParallelismStore for all vertices of a given job graph, which will
set defaults and ensure that the returned store contains valid parallelisms. |
Modifier and Type | Method and Description |
---|---|
ExecutionGraph |
DefaultExecutionGraphFactory.createAndRestoreExecutionGraph(JobGraph jobGraph,
CompletedCheckpointStore completedCheckpointStore,
CheckpointsCleaner checkpointsCleaner,
CheckpointIDCounter checkpointIdCounter,
TaskDeploymentDescriptorFactory.PartitionLocationConstraint partitionLocationConstraint,
long initializationTimestamp,
VertexAttemptNumberStore vertexAttemptNumberStore,
VertexParallelismStore vertexParallelismStore,
ExecutionStateUpdateListener executionStateUpdateListener,
org.slf4j.Logger log) |
ExecutionGraph |
ExecutionGraphFactory.createAndRestoreExecutionGraph(JobGraph jobGraph,
CompletedCheckpointStore completedCheckpointStore,
CheckpointsCleaner checkpointsCleaner,
CheckpointIDCounter checkpointIdCounter,
TaskDeploymentDescriptorFactory.PartitionLocationConstraint partitionLocationConstraint,
long initializationTimestamp,
VertexAttemptNumberStore vertexAttemptNumberStore,
VertexParallelismStore vertexParallelismStore,
ExecutionStateUpdateListener executionStateUpdateListener,
org.slf4j.Logger log)
Create and restore
ExecutionGraph from the given JobGraph and services. |
Constructor and Description |
---|
DefaultScheduler(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration jobMasterConfiguration,
java.util.function.Consumer<ComponentMainThreadExecutor> startUpAction,
ScheduledExecutor delayExecutor,
ClassLoader userCodeLoader,
CheckpointsCleaner checkpointsCleaner,
CheckpointRecoveryFactory checkpointRecoveryFactory,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
SchedulingStrategyFactory schedulingStrategyFactory,
FailoverStrategy.Factory failoverStrategyFactory,
RestartBackoffTimeStrategy restartBackoffTimeStrategy,
ExecutionOperations executionOperations,
ExecutionVertexVersioner executionVertexVersioner,
ExecutionSlotAllocatorFactory executionSlotAllocatorFactory,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
JobStatusListener jobStatusListener,
ExecutionGraphFactory executionGraphFactory,
ShuffleMaster<?> shuffleMaster,
Time rpcTimeout,
VertexParallelismStore vertexParallelismStore,
ExecutionDeployer.Factory executionDeployerFactory) |
SchedulerBase(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration jobMasterConfiguration,
CheckpointsCleaner checkpointsCleaner,
CheckpointRecoveryFactory checkpointRecoveryFactory,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
ExecutionVertexVersioner executionVertexVersioner,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
JobStatusListener jobStatusListener,
ExecutionGraphFactory executionGraphFactory,
VertexParallelismStore vertexParallelismStore) |
Constructor and Description |
---|
JobGraphJobInformation(JobGraph jobGraph,
VertexParallelismStore vertexParallelismStore) |
Modifier and Type | Method and Description |
---|---|
static VertexParallelismStore |
AdaptiveBatchScheduler.computeVertexParallelismStoreForDynamicGraph(Iterable<JobVertex> vertices,
int defaultMaxParallelism)
Compute the
VertexParallelismStore for all given vertices in a dynamic graph, which
will set defaults and ensure that the returned store contains valid parallelisms, with the
configured default max parallelism. |
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.