Modifier and Type | Method and Description |
---|---|
BlocklistHandler |
NoOpBlocklistHandler.Factory.create(BlocklistContext blocklistContext,
Function<ResourceID,String> taskManagerNodeIdRetriever,
ComponentMainThreadExecutor mainThreadExecutor,
org.slf4j.Logger log) |
BlocklistHandler |
DefaultBlocklistHandler.Factory.create(BlocklistContext blocklistContext,
Function<ResourceID,String> taskManagerNodeIdRetriever,
ComponentMainThreadExecutor mainThreadExecutor,
org.slf4j.Logger log) |
BlocklistHandler |
BlocklistHandler.Factory.create(BlocklistContext blocklistContext,
Function<ResourceID,String> taskManagerNodeIdRetriever,
ComponentMainThreadExecutor mainThreadExecutor,
org.slf4j.Logger log)
Instantiates a
BlocklistHandler . |
Modifier and Type | Class and Description |
---|---|
static class |
ComponentMainThreadExecutor.DummyComponentMainThreadExecutor
Dummy implementation of ComponentMainThreadExecutor.
|
Constructor and Description |
---|
OnMainThreadJobManagerRunnerRegistry(JobManagerRunnerRegistry delegate,
ComponentMainThreadExecutor mainThreadExecutor) |
Modifier and Type | Method and Description |
---|---|
ComponentMainThreadExecutor |
DefaultExecutionGraph.getJobMasterMainThreadExecutor() |
ComponentMainThreadExecutor |
ExecutionGraph.getJobMasterMainThreadExecutor() |
ComponentMainThreadExecutor |
InternalExecutionGraphAccessor.getJobMasterMainThreadExecutor() |
Modifier and Type | Method and Description |
---|---|
void |
DefaultExecutionGraph.start(ComponentMainThreadExecutor jobMasterMainThreadExecutor) |
void |
ExecutionGraph.start(ComponentMainThreadExecutor jobMasterMainThreadExecutor) |
Constructor and Description |
---|
ExecutionFailureHandler(Configuration jobMasterConfig,
SchedulingTopology schedulingTopology,
FailoverStrategy failoverStrategy,
RestartBackoffTimeStrategy restartBackoffTimeStrategy,
ComponentMainThreadExecutor mainThreadExecutor,
Collection<FailureEnricher> failureEnrichers,
FailureEnricher.Context taskFailureCtx,
FailureEnricher.Context globalFailureCtx,
MetricGroup metricGroup)
Creates the handler to deal with task failures.
|
Modifier and Type | Method and Description |
---|---|
SchedulerNG |
DefaultSlotPoolServiceSchedulerFactory.createScheduler(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration configuration,
SlotPoolService slotPoolService,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
ExecutionDeploymentTracker executionDeploymentTracker,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
FatalErrorHandler fatalErrorHandler,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
BlocklistOperations blocklistOperations) |
SchedulerNG |
SlotPoolServiceSchedulerFactory.createScheduler(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration configuration,
SlotPoolService slotPoolService,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
ExecutionDeploymentTracker executionDeploymentTracker,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
FatalErrorHandler fatalErrorHandler,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
BlocklistOperations blocklistOperations)
Creates a
SchedulerNG . |
Modifier and Type | Method and Description |
---|---|
protected void |
DeclarativeSlotPoolService.onStart(ComponentMainThreadExecutor componentMainThreadExecutor)
This method is called when the slot pool service is started.
|
protected void |
DeclarativeSlotPoolBridge.onStart(ComponentMainThreadExecutor componentMainThreadExecutor) |
void |
PhysicalSlotRequestBulkCheckerImpl.start(ComponentMainThreadExecutor mainThreadExecutor) |
void |
PhysicalSlotRequestBulkChecker.start(ComponentMainThreadExecutor mainThreadExecutor)
Starts the bulk checker by initializing the main thread executor.
|
void |
SlotPool.start(JobMasterId jobMasterId,
String newJobManagerAddress,
ComponentMainThreadExecutor jmMainThreadScheduledExecutor) |
void |
DeclarativeSlotPoolService.start(JobMasterId jobMasterId,
String address,
ComponentMainThreadExecutor mainThreadExecutor) |
void |
SlotPoolService.start(JobMasterId jobMasterId,
String address,
ComponentMainThreadExecutor mainThreadExecutor)
Start the encapsulated slot pool implementation.
|
Modifier and Type | Method and Description |
---|---|
void |
OperatorCoordinatorHolder.lazyInitialize(GlobalFailureHandler globalFailureHandler,
ComponentMainThreadExecutor mainThreadExecutor,
CheckpointCoordinator checkpointCoordinator) |
void |
OperatorCoordinatorHolder.lazyInitialize(GlobalFailureHandler globalFailureHandler,
ComponentMainThreadExecutor mainThreadExecutor,
CheckpointCoordinator checkpointCoordinator,
int operatorParallelism) |
Modifier and Type | Class and Description |
---|---|
protected static class |
RpcEndpoint.MainThreadExecutor
Executor which executes runnables in the main thread context.
|
Modifier and Type | Method and Description |
---|---|
protected ComponentMainThreadExecutor |
SchedulerBase.getMainThreadExecutor() |
Modifier and Type | Method and Description |
---|---|
DefaultExecutionDeployer |
DefaultExecutionDeployer.Factory.createInstance(org.slf4j.Logger log,
ExecutionSlotAllocator executionSlotAllocator,
ExecutionOperations executionOperations,
ExecutionVertexVersioner executionVertexVersioner,
Time partitionRegistrationTimeout,
BiConsumer<ExecutionVertexID,AllocationID> allocationReservationFunc,
ComponentMainThreadExecutor mainThreadExecutor) |
ExecutionDeployer |
ExecutionDeployer.Factory.createInstance(org.slf4j.Logger log,
ExecutionSlotAllocator executionSlotAllocator,
ExecutionOperations executionOperations,
ExecutionVertexVersioner executionVertexVersioner,
Time partitionRegistrationTimeout,
BiConsumer<ExecutionVertexID,AllocationID> allocationReservationFunc,
ComponentMainThreadExecutor mainThreadExecutor)
Instantiate an
ExecutionDeployer with the given params. |
SchedulerNG |
SchedulerNGFactory.createInstance(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotPoolService slotPoolService,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
ExecutionDeploymentTracker executionDeploymentTracker,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
FatalErrorHandler fatalErrorHandler,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
BlocklistOperations blocklistOperations) |
SchedulerNG |
DefaultSchedulerFactory.createInstance(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotPoolService slotPoolService,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
ExecutionDeploymentTracker executionDeploymentTracker,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
FatalErrorHandler fatalErrorHandler,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
BlocklistOperations blocklistOperations) |
void |
DefaultOperatorCoordinatorHandler.initializeOperatorCoordinators(ComponentMainThreadExecutor mainThreadExecutor) |
void |
OperatorCoordinatorHandler.initializeOperatorCoordinators(ComponentMainThreadExecutor mainThreadExecutor)
Initialize operator coordinators.
|
void |
DefaultOperatorCoordinatorHandler.registerAndStartNewCoordinators(Collection<OperatorCoordinatorHolder> coordinators,
ComponentMainThreadExecutor mainThreadExecutor,
int parallelism) |
void |
OperatorCoordinatorHandler.registerAndStartNewCoordinators(Collection<OperatorCoordinatorHolder> coordinators,
ComponentMainThreadExecutor mainThreadExecutor,
int parallelism)
Register and start new operator coordinators.
|
Constructor and Description |
---|
DefaultScheduler(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration jobMasterConfiguration,
Consumer<ComponentMainThreadExecutor> startUpAction,
ScheduledExecutor delayExecutor,
ClassLoader userCodeLoader,
CheckpointsCleaner checkpointsCleaner,
CheckpointRecoveryFactory checkpointRecoveryFactory,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
SchedulingStrategyFactory schedulingStrategyFactory,
FailoverStrategy.Factory failoverStrategyFactory,
RestartBackoffTimeStrategy restartBackoffTimeStrategy,
ExecutionOperations executionOperations,
ExecutionVertexVersioner executionVertexVersioner,
ExecutionSlotAllocatorFactory executionSlotAllocatorFactory,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
ExecutionGraphFactory executionGraphFactory,
ShuffleMaster<?> shuffleMaster,
Time rpcTimeout,
VertexParallelismStore vertexParallelismStore,
ExecutionDeployer.Factory executionDeployerFactory) |
ExecutionGraphHandler(ExecutionGraph executionGraph,
org.slf4j.Logger log,
Executor ioExecutor,
ComponentMainThreadExecutor mainThreadExecutor) |
SchedulerBase(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration jobMasterConfiguration,
CheckpointsCleaner checkpointsCleaner,
CheckpointRecoveryFactory checkpointRecoveryFactory,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
ExecutionVertexVersioner executionVertexVersioner,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
JobStatusListener jobStatusListener,
ExecutionGraphFactory executionGraphFactory,
VertexParallelismStore vertexParallelismStore) |
Constructor and Description |
---|
DefaultScheduler(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration jobMasterConfiguration,
Consumer<ComponentMainThreadExecutor> startUpAction,
ScheduledExecutor delayExecutor,
ClassLoader userCodeLoader,
CheckpointsCleaner checkpointsCleaner,
CheckpointRecoveryFactory checkpointRecoveryFactory,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
SchedulingStrategyFactory schedulingStrategyFactory,
FailoverStrategy.Factory failoverStrategyFactory,
RestartBackoffTimeStrategy restartBackoffTimeStrategy,
ExecutionOperations executionOperations,
ExecutionVertexVersioner executionVertexVersioner,
ExecutionSlotAllocatorFactory executionSlotAllocatorFactory,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
ExecutionGraphFactory executionGraphFactory,
ShuffleMaster<?> shuffleMaster,
Time rpcTimeout,
VertexParallelismStore vertexParallelismStore,
ExecutionDeployer.Factory executionDeployerFactory) |
Modifier and Type | Method and Description |
---|---|
ComponentMainThreadExecutor |
AdaptiveScheduler.getMainThreadExecutor() |
Modifier and Type | Method and Description |
---|---|
SchedulerNG |
AdaptiveSchedulerFactory.createInstance(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotPoolService slotPoolService,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
ExecutionDeploymentTracker executionDeploymentTracker,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
FatalErrorHandler fatalErrorHandler,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
BlocklistOperations blocklistOperations) |
Constructor and Description |
---|
AdaptiveScheduler(AdaptiveScheduler.Settings settings,
JobGraph jobGraph,
JobResourceRequirements jobResourceRequirements,
Configuration configuration,
DeclarativeSlotPool declarativeSlotPool,
SlotAllocator slotAllocator,
Executor ioExecutor,
ClassLoader userCodeClassLoader,
CheckpointsCleaner checkpointsCleaner,
CheckpointRecoveryFactory checkpointRecoveryFactory,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
RestartBackoffTimeStrategy restartBackoffTimeStrategy,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
FatalErrorHandler fatalErrorHandler,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
ExecutionGraphFactory executionGraphFactory) |
Modifier and Type | Method and Description |
---|---|
ComponentMainThreadExecutor |
BatchJobRecoveryContext.getMainThreadExecutor()
Provides the main thread executor.
|
Modifier and Type | Method and Description |
---|---|
SchedulerNG |
AdaptiveBatchSchedulerFactory.createInstance(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration jobMasterConfiguration,
SlotPoolService slotPoolService,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
Time slotRequestTimeout,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
ExecutionDeploymentTracker executionDeploymentTracker,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
FatalErrorHandler fatalErrorHandler,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
BlocklistOperations blocklistOperations) |
static AdaptiveBatchScheduler |
AdaptiveBatchSchedulerFactory.createScheduler(org.slf4j.Logger log,
JobGraph jobGraph,
ExecutionConfig executionConfig,
Executor ioExecutor,
Configuration jobMasterConfiguration,
ScheduledExecutorService futureExecutor,
ClassLoader userCodeLoader,
CheckpointRecoveryFactory checkpointRecoveryFactory,
Time rpcTimeout,
BlobWriter blobWriter,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
ShuffleMaster<?> shuffleMaster,
JobMasterPartitionTracker partitionTracker,
ExecutionDeploymentTracker executionDeploymentTracker,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
BlocklistOperations blocklistOperations,
ExecutionOperations executionOperations,
ExecutionSlotAllocatorFactory allocatorFactory,
RestartBackoffTimeStrategy restartBackoffTimeStrategy,
ScheduledExecutor delayExecutor,
VertexParallelismAndInputInfosDecider vertexParallelismAndInputInfosDecider,
BatchJobRecoveryHandler jobRecoveryHandler) |
void |
DummySpeculativeExecutionHandler.init(ExecutionGraph executionGraph,
ComponentMainThreadExecutor mainThreadExecutor,
MetricGroup metricGroup) |
void |
SpeculativeExecutionHandler.init(ExecutionGraph executionGraph,
ComponentMainThreadExecutor mainThreadExecutor,
MetricGroup metricGroup)
Initial speculative execution handler.
|
void |
DefaultSpeculativeExecutionHandler.init(ExecutionGraph executionGraph,
ComponentMainThreadExecutor mainThreadExecutor,
MetricGroup metricGroup) |
Constructor and Description |
---|
AdaptiveBatchScheduler(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration jobMasterConfiguration,
Consumer<ComponentMainThreadExecutor> startUpAction,
ScheduledExecutor delayExecutor,
ClassLoader userCodeLoader,
CheckpointsCleaner checkpointsCleaner,
CheckpointRecoveryFactory checkpointRecoveryFactory,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
SchedulingStrategyFactory schedulingStrategyFactory,
FailoverStrategy.Factory failoverStrategyFactory,
RestartBackoffTimeStrategy restartBackoffTimeStrategy,
ExecutionOperations executionOperations,
ExecutionVertexVersioner executionVertexVersioner,
ExecutionSlotAllocatorFactory executionSlotAllocatorFactory,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
ExecutionGraphFactory executionGraphFactory,
ShuffleMaster<?> shuffleMaster,
Time rpcTimeout,
VertexParallelismAndInputInfosDecider vertexParallelismAndInputInfosDecider,
int defaultMaxParallelism,
BlocklistOperations blocklistOperations,
JobManagerOptions.HybridPartitionDataConsumeConstraint hybridPartitionDataConsumeConstraint,
Map<JobVertexID,ForwardGroup> forwardGroupsByJobVertexId,
BatchJobRecoveryHandler jobRecoveryHandler) |
Constructor and Description |
---|
AdaptiveBatchScheduler(org.slf4j.Logger log,
JobGraph jobGraph,
Executor ioExecutor,
Configuration jobMasterConfiguration,
Consumer<ComponentMainThreadExecutor> startUpAction,
ScheduledExecutor delayExecutor,
ClassLoader userCodeLoader,
CheckpointsCleaner checkpointsCleaner,
CheckpointRecoveryFactory checkpointRecoveryFactory,
JobManagerJobMetricGroup jobManagerJobMetricGroup,
SchedulingStrategyFactory schedulingStrategyFactory,
FailoverStrategy.Factory failoverStrategyFactory,
RestartBackoffTimeStrategy restartBackoffTimeStrategy,
ExecutionOperations executionOperations,
ExecutionVertexVersioner executionVertexVersioner,
ExecutionSlotAllocatorFactory executionSlotAllocatorFactory,
long initializationTimestamp,
ComponentMainThreadExecutor mainThreadExecutor,
JobStatusListener jobStatusListener,
Collection<FailureEnricher> failureEnrichers,
ExecutionGraphFactory executionGraphFactory,
ShuffleMaster<?> shuffleMaster,
Time rpcTimeout,
VertexParallelismAndInputInfosDecider vertexParallelismAndInputInfosDecider,
int defaultMaxParallelism,
BlocklistOperations blocklistOperations,
JobManagerOptions.HybridPartitionDataConsumeConstraint hybridPartitionDataConsumeConstraint,
Map<JobVertexID,ForwardGroup> forwardGroupsByJobVertexId,
BatchJobRecoveryHandler jobRecoveryHandler) |
Modifier and Type | Method and Description |
---|---|
void |
SlowTaskDetector.start(ExecutionGraph executionGraph,
SlowTaskDetectorListener listener,
ComponentMainThreadExecutor mainThreadExecutor)
Start detecting slow tasks periodically.
|
void |
ExecutionTimeBasedSlowTaskDetector.start(ExecutionGraph executionGraph,
SlowTaskDetectorListener listener,
ComponentMainThreadExecutor mainThreadExecutor) |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<String> |
StopWithSavepointTerminationManager.stopWithSavepoint(CompletableFuture<CompletedCheckpoint> completedSavepointFuture,
CompletableFuture<Collection<ExecutionState>> terminatedExecutionStatesFuture,
ComponentMainThreadExecutor mainThreadExecutor)
Enforces the correct completion order of the passed
CompletableFuture instances in
accordance to the contract of StopWithSavepointTerminationHandler . |
Modifier and Type | Method and Description |
---|---|
void |
TaskSlotTable.start(SlotActions initialSlotActions,
ComponentMainThreadExecutor mainThreadExecutor)
Start the task slot table with the given slot actions.
|
void |
TaskSlotTableImpl.start(SlotActions initialSlotActions,
ComponentMainThreadExecutor mainThreadExecutor) |
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.