Modifier and Type | Method and Description |
---|---|
ResourceID |
KubernetesWorkerNode.getResourceID() |
Modifier and Type | Method and Description |
---|---|
boolean |
NoOpBlocklistHandler.isBlockedTaskManager(ResourceID taskManagerId) |
boolean |
BlockedTaskManagerChecker.isBlockedTaskManager(ResourceID resourceID)
Returns whether the given task manager is located on a blocked node.
|
boolean |
BlocklistHandler.isBlockedTaskManager(ResourceID taskManagerId)
Returns whether the given task manager is blocked (located on blocked nodes).
|
boolean |
DefaultBlocklistHandler.isBlockedTaskManager(ResourceID taskManagerId) |
Modifier and Type | Method and Description |
---|---|
BlocklistHandler |
NoOpBlocklistHandler.Factory.create(BlocklistContext blocklistContext,
java.util.function.Function<ResourceID,String> taskManagerNodeIdRetriever,
ComponentMainThreadExecutor mainThreadExecutor,
org.slf4j.Logger log) |
BlocklistHandler |
BlocklistHandler.Factory.create(BlocklistContext blocklistContext,
java.util.function.Function<ResourceID,String> taskManagerNodeIdRetriever,
ComponentMainThreadExecutor mainThreadExecutor,
org.slf4j.Logger log)
Instantiates a
BlocklistHandler . |
BlocklistHandler |
DefaultBlocklistHandler.Factory.create(BlocklistContext blocklistContext,
java.util.function.Function<ResourceID,String> taskManagerNodeIdRetriever,
ComponentMainThreadExecutor mainThreadExecutor,
org.slf4j.Logger log) |
Modifier and Type | Method and Description |
---|---|
static ResourceID |
ResourceID.generate()
Generate a random resource id.
|
ResourceID |
ResourceID.getResourceID()
A ResourceID can always retrieve a ResourceID.
|
ResourceID |
SlotID.getResourceID() |
ResourceID |
ResourceIDRetrievable.getResourceID()
Gets the ResourceID of the object.
|
Modifier and Type | Method and Description |
---|---|
static SlotID |
SlotID.getDynamicSlotID(ResourceID resourceID)
Get a SlotID without actual slot index for dynamic slot allocation.
|
Constructor and Description |
---|
SlotID(ResourceID resourceId,
int slotNumber) |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<Collection<Tuple2<ResourceID,String>>> |
Dispatcher.requestTaskManagerMetricQueryServiceAddresses(Time timeout) |
Modifier and Type | Method and Description |
---|---|
static File |
ClusterEntrypointUtils.generateJobManagerWorkingDirectoryFile(Configuration configuration,
ResourceID resourceId)
Generates the working directory
File for the JobManager process. |
static File |
ClusterEntrypointUtils.generateTaskManagerWorkingDirectoryFile(Configuration configuration,
ResourceID resourceId)
Generates the working directory
File for the TaskManager process. |
Modifier and Type | Method and Description |
---|---|
static DeterminismEnvelope<WorkingDirectory> |
ClusterEntrypointUtils.createJobManagerWorkingDirectory(Configuration configuration,
DeterminismEnvelope<ResourceID> envelopedResourceId)
Creates the working directory for the JobManager process.
|
static DeterminismEnvelope<WorkingDirectory> |
ClusterEntrypointUtils.createTaskManagerWorkingDirectory(Configuration configuration,
DeterminismEnvelope<ResourceID> envelopedResourceId)
Creates the working directory for the TaskManager process.
|
Modifier and Type | Method and Description |
---|---|
DispatcherResourceManagerComponent |
DefaultDispatcherResourceManagerComponentFactory.create(Configuration configuration,
ResourceID resourceId,
Executor ioExecutor,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
BlobServer blobServer,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
MetricRegistry metricRegistry,
ExecutionGraphInfoStore executionGraphInfoStore,
MetricQueryServiceRetriever metricQueryServiceRetriever,
FatalErrorHandler fatalErrorHandler) |
DispatcherResourceManagerComponent |
DispatcherResourceManagerComponentFactory.create(Configuration configuration,
ResourceID resourceId,
Executor ioExecutor,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
BlobServer blobServer,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
MetricRegistry metricRegistry,
ExecutionGraphInfoStore executionGraphInfoStore,
MetricQueryServiceRetriever metricQueryServiceRetriever,
FatalErrorHandler fatalErrorHandler) |
Modifier and Type | Method and Description |
---|---|
void |
ExecutionDeploymentListener.onStartedDeployment(ExecutionAttemptID execution,
ResourceID host) |
void |
NoOpExecutionDeploymentListener.onStartedDeployment(ExecutionAttemptID execution,
ResourceID host) |
Modifier and Type | Method and Description |
---|---|
ResourceID |
HeartbeatMonitor.getHeartbeatTargetId()
Gets heartbeat target id.
|
ResourceID |
HeartbeatMonitorImpl.getHeartbeatTargetId() |
Modifier and Type | Method and Description |
---|---|
<I,O> HeartbeatManager<I,O> |
HeartbeatServices.createHeartbeatManager(ResourceID resourceId,
HeartbeatListener<I,O> heartbeatListener,
ScheduledExecutor mainThreadExecutor,
org.slf4j.Logger log)
Creates a heartbeat manager which does not actively send heartbeats.
|
<I,O> HeartbeatManager<I,O> |
HeartbeatServices.createHeartbeatManagerSender(ResourceID resourceId,
HeartbeatListener<I,O> heartbeatListener,
ScheduledExecutor mainThreadExecutor,
org.slf4j.Logger log)
Creates a heartbeat manager which actively sends heartbeats to monitoring targets.
|
HeartbeatMonitor<O> |
HeartbeatMonitor.Factory.createHeartbeatMonitor(ResourceID resourceID,
HeartbeatTarget<O> heartbeatTarget,
ScheduledExecutor mainThreadExecutor,
HeartbeatListener<?,O> heartbeatListener,
long heartbeatTimeoutIntervalMs,
int failedRpcRequestsUntilUnreachable)
Create heartbeat monitor heartbeat monitor.
|
long |
NoOpHeartbeatManager.getLastHeartbeatFrom(ResourceID resourceId) |
long |
HeartbeatManager.getLastHeartbeatFrom(ResourceID resourceId)
Returns the last received heartbeat from the given target.
|
long |
HeartbeatManagerImpl.getLastHeartbeatFrom(ResourceID resourceId) |
protected java.util.function.BiConsumer<Void,Throwable> |
HeartbeatManagerImpl.handleHeartbeatRpc(ResourceID heartbeatTarget) |
void |
NoOpHeartbeatManager.monitorTarget(ResourceID resourceID,
HeartbeatTarget<O> heartbeatTarget) |
void |
HeartbeatManager.monitorTarget(ResourceID resourceID,
HeartbeatTarget<O> heartbeatTarget)
Start monitoring a
HeartbeatTarget . |
void |
HeartbeatManagerImpl.monitorTarget(ResourceID resourceID,
HeartbeatTarget<O> heartbeatTarget) |
void |
HeartbeatListener.notifyHeartbeatTimeout(ResourceID resourceID)
Callback which is called if a heartbeat for the machine identified by the given resource ID
times out.
|
void |
HeartbeatListener.notifyTargetUnreachable(ResourceID resourceID)
Callback which is called if a target specified by the given resource ID is no longer
reachable.
|
CompletableFuture<Void> |
HeartbeatTarget.receiveHeartbeat(ResourceID heartbeatOrigin,
I heartbeatPayload)
Sends a heartbeat response to the target.
|
CompletableFuture<Void> |
NoOpHeartbeatManager.receiveHeartbeat(ResourceID heartbeatOrigin,
I heartbeatPayload) |
CompletableFuture<Void> |
HeartbeatSender.receiveHeartbeat(ResourceID heartbeatOrigin,
I heartbeatPayload) |
CompletableFuture<Void> |
HeartbeatManagerImpl.receiveHeartbeat(ResourceID heartbeatOrigin,
I heartbeatPayload) |
void |
HeartbeatListener.reportPayload(ResourceID resourceID,
I payload)
Callback which is called whenever a heartbeat with an associated payload is received.
|
CompletableFuture<Void> |
HeartbeatTarget.requestHeartbeat(ResourceID requestOrigin,
I heartbeatPayload)
Requests a heartbeat from the target.
|
CompletableFuture<Void> |
NoOpHeartbeatManager.requestHeartbeat(ResourceID requestOrigin,
I heartbeatPayload) |
CompletableFuture<Void> |
HeartbeatReceiver.requestHeartbeat(ResourceID requestOrigin,
I heartbeatPayload) |
CompletableFuture<Void> |
HeartbeatManagerImpl.requestHeartbeat(ResourceID requestOrigin,
I heartbeatPayload) |
O |
HeartbeatListener.retrievePayload(ResourceID resourceID)
Retrieves the payload value for the next heartbeat message.
|
void |
NoOpHeartbeatManager.unmonitorTarget(ResourceID resourceID) |
void |
HeartbeatManager.unmonitorTarget(ResourceID resourceID)
Stops monitoring the heartbeat target with the associated resource ID.
|
void |
HeartbeatManagerImpl.unmonitorTarget(ResourceID resourceID) |
Constructor and Description |
---|
HeartbeatManagerImpl(long heartbeatTimeoutIntervalMs,
int failedRpcRequestsUntilUnreachable,
ResourceID ownResourceID,
HeartbeatListener<I,O> heartbeatListener,
ScheduledExecutor mainThreadExecutor,
org.slf4j.Logger log) |
HeartbeatManagerImpl(long heartbeatTimeoutIntervalMs,
int failedRpcRequestsUntilUnreachable,
ResourceID ownResourceID,
HeartbeatListener<I,O> heartbeatListener,
ScheduledExecutor mainThreadExecutor,
org.slf4j.Logger log,
HeartbeatMonitor.Factory<O> heartbeatMonitorFactory) |
Modifier and Type | Method and Description |
---|---|
ResourceID |
ConnectionID.getResourceID() |
Modifier and Type | Method and Description |
---|---|
static NettyShuffleEnvironment |
NettyShuffleServiceFactory.createNettyShuffleEnvironment(NettyShuffleEnvironmentConfiguration config,
ResourceID taskExecutorResourceId,
TaskEventPublisher taskEventPublisher,
ResultPartitionManager resultPartitionManager,
ConnectionManager connectionManager,
MetricGroup metricGroup,
Executor ioExecutor,
int numberOfSlots,
String[] tmpDirPaths) |
Constructor and Description |
---|
ConnectionID(ResourceID resourceID,
InetSocketAddress address,
int connectionIndex) |
Modifier and Type | Method and Description |
---|---|
Optional<TaskExecutorGateway> |
PartitionTrackerFactory.TaskExecutorGatewayLookup.lookup(ResourceID taskExecutorId)
Returns a
TaskExecutorGateway corresponding to the given ResourceID. |
void |
ResourceManagerPartitionTracker.processTaskExecutorClusterPartitionReport(ResourceID taskExecutorId,
ClusterPartitionReport clusterPartitionReport)
Processes
ClusterPartitionReport of a task executor. |
void |
ResourceManagerPartitionTrackerImpl.processTaskExecutorClusterPartitionReport(ResourceID taskExecutorId,
ClusterPartitionReport clusterPartitionReport) |
void |
ResourceManagerPartitionTracker.processTaskExecutorShutdown(ResourceID taskExecutorId)
Processes the shutdown of task executor.
|
void |
ResourceManagerPartitionTrackerImpl.processTaskExecutorShutdown(ResourceID taskExecutorId) |
void |
TaskExecutorClusterPartitionReleaser.releaseClusterPartitions(ResourceID taskExecutorId,
Set<IntermediateDataSetID> dataSetsToRelease) |
CompletableFuture<Void> |
ClusterPartitionManager.reportClusterPartitions(ResourceID taskExecutorId,
ClusterPartitionReport clusterPartitionReport)
Report the cluster partitions status in the task executor.
|
void |
JobMasterPartitionTrackerImpl.startTrackingPartition(ResourceID producingTaskExecutorId,
ResultPartitionDeploymentDescriptor resultPartitionDeploymentDescriptor) |
void |
JobMasterPartitionTracker.startTrackingPartition(ResourceID producingTaskExecutorId,
ResultPartitionDeploymentDescriptor resultPartitionDeploymentDescriptor)
Starts the tracking of the given partition for the given task executor ID.
|
Modifier and Type | Field and Description |
---|---|
protected ResourceID |
SingleInputGateFactory.taskExecutorResourceId |
Modifier and Type | Method and Description |
---|---|
void |
SingleInputGate.updateInputChannel(ResourceID localLocation,
NettyShuffleDescriptor shuffleDescriptor) |
Constructor and Description |
---|
SingleInputGateFactory(ResourceID taskExecutorResourceId,
NettyShuffleEnvironmentConfiguration networkConfig,
ConnectionManager connectionManager,
ResultPartitionManager partitionManager,
TaskEventPublisher taskEventPublisher,
NetworkBufferPool networkBufferPool) |
Modifier and Type | Method and Description |
---|---|
ResourceID |
JMTMRegistrationSuccess.getResourceID() |
ResourceID |
JobMasterRegistrationSuccess.getResourceManagerResourceId() |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<Acknowledge> |
JobMaster.disconnectTaskManager(ResourceID resourceID,
Exception cause) |
CompletableFuture<Acknowledge> |
JobMasterGateway.disconnectTaskManager(ResourceID resourceID,
Exception cause)
Disconnects the given
TaskExecutor from the
JobMaster . |
void |
JobMaster.failSlot(ResourceID taskManagerId,
AllocationID allocationId,
Exception cause) |
void |
JobMasterGateway.failSlot(ResourceID taskManagerId,
AllocationID allocationId,
Exception cause)
Fails the slot with the given allocation id and cause.
|
Map<ExecutionAttemptID,ExecutionDeploymentState> |
DefaultExecutionDeploymentTracker.getExecutionsOn(ResourceID host) |
Map<ExecutionAttemptID,ExecutionDeploymentState> |
ExecutionDeploymentTracker.getExecutionsOn(ResourceID host)
Returns all tracked executions for the given host.
|
CompletableFuture<Void> |
JobMaster.heartbeatFromResourceManager(ResourceID resourceID) |
CompletableFuture<Void> |
JobMasterGateway.heartbeatFromResourceManager(ResourceID resourceID)
Sends heartbeat request from the resource manager.
|
CompletableFuture<Void> |
JobMaster.heartbeatFromTaskManager(ResourceID resourceID,
TaskExecutorToJobManagerHeartbeatPayload payload) |
CompletableFuture<Void> |
JobMasterGateway.heartbeatFromTaskManager(ResourceID resourceID,
TaskExecutorToJobManagerHeartbeatPayload payload)
Sends the heartbeat to job manager from task manager.
|
CompletableFuture<Collection<SlotOffer>> |
JobMaster.offerSlots(ResourceID taskManagerId,
Collection<SlotOffer> slots,
Time timeout) |
CompletableFuture<Collection<SlotOffer>> |
JobMasterGateway.offerSlots(ResourceID taskManagerId,
Collection<SlotOffer> slots,
Time timeout)
Offers the given slots to the job manager.
|
void |
ExecutionDeploymentReconciliationHandler.onMissingDeploymentsOf(Collection<ExecutionAttemptID> executionAttemptIds,
ResourceID hostingTaskExecutor)
Called if some executions are expected to be hosted on a task executor, but aren't.
|
void |
ExecutionDeploymentTrackerDeploymentListenerAdapter.onStartedDeployment(ExecutionAttemptID execution,
ResourceID host) |
void |
ExecutionDeploymentReconciliationHandler.onUnknownDeploymentsOf(Collection<ExecutionAttemptID> executionAttemptIds,
ResourceID hostingTaskExecutor)
Called if some executions are hosted on a task executor, but we don't expect them.
|
void |
ExecutionDeploymentReconciler.reconcileExecutionDeployments(ResourceID taskExecutorHost,
ExecutionDeploymentReport executionDeploymentReport,
Map<ExecutionAttemptID,ExecutionDeploymentState> expectedDeployedExecutionIds)
Reconciles the deployment states between all reported/expected executions for the given task
executor.
|
void |
DefaultExecutionDeploymentReconciler.reconcileExecutionDeployments(ResourceID taskExecutorHost,
ExecutionDeploymentReport executionDeploymentReport,
Map<ExecutionAttemptID,ExecutionDeploymentState> expectedDeployedExecutions) |
void |
DefaultExecutionDeploymentTracker.startTrackingPendingDeploymentOf(ExecutionAttemptID executionAttemptId,
ResourceID host) |
void |
ExecutionDeploymentTracker.startTrackingPendingDeploymentOf(ExecutionAttemptID executionAttemptId,
ResourceID host)
Starts tracking the given execution that is being deployed on the given host.
|
Constructor and Description |
---|
JMTMRegistrationSuccess(ResourceID resourceID) |
JobMaster(RpcService rpcService,
JobMasterId jobMasterId,
JobMasterConfiguration jobMasterConfiguration,
ResourceID resourceId,
JobGraph jobGraph,
HighAvailabilityServices highAvailabilityService,
SlotPoolServiceSchedulerFactory slotPoolServiceSchedulerFactory,
JobManagerSharedServices jobManagerSharedServices,
HeartbeatServices heartbeatServices,
JobManagerJobMetricGroupFactory jobMetricGroupFactory,
OnCompletionActions jobCompletionActions,
FatalErrorHandler fatalErrorHandler,
ClassLoader userCodeLoader,
ShuffleMaster<?> shuffleMaster,
PartitionTrackerFactory partitionTrackerFactory,
ExecutionDeploymentTracker executionDeploymentTracker,
ExecutionDeploymentReconciler.Factory executionDeploymentReconcilerFactory,
BlocklistHandler.Factory blocklistHandlerFactory,
long initializationTimestamp) |
JobMasterRegistrationSuccess(ResourceManagerId resourceManagerId,
ResourceID resourceManagerResourceId) |
Modifier and Type | Method and Description |
---|---|
Optional<ResourceID> |
SlotPoolService.failAllocation(ResourceID taskManagerId,
AllocationID allocationId,
Exception cause)
Fails the allocation with the given allocationId.
|
Optional<ResourceID> |
DeclarativeSlotPoolService.failAllocation(ResourceID taskManagerId,
AllocationID allocationId,
Exception cause) |
Modifier and Type | Method and Description |
---|---|
boolean |
AllocatedSlotPool.containsSlots(ResourceID owner)
Checks whether the slot pool contains at least one slot belonging to the specified owner.
|
boolean |
DefaultDeclarativeSlotPool.containsSlots(ResourceID owner) |
boolean |
DefaultAllocatedSlotPool.containsSlots(ResourceID owner) |
boolean |
DeclarativeSlotPool.containsSlots(ResourceID owner)
Returns whether the slot pool has a slot registered which is owned by the given TaskExecutor.
|
AllocatedSlotReport |
SlotPool.createAllocatedSlotReport(ResourceID taskManagerId)
Create report about the allocated slots belonging to the specified task manager.
|
AllocatedSlotReport |
SlotPoolService.createAllocatedSlotReport(ResourceID taskManagerId)
Create report about the allocated slots belonging to the specified task manager.
|
AllocatedSlotReport |
DeclarativeSlotPoolService.createAllocatedSlotReport(ResourceID taskManagerId) |
Optional<ResourceID> |
SlotPoolService.failAllocation(ResourceID taskManagerId,
AllocationID allocationId,
Exception cause)
Fails the allocation with the given allocationId.
|
Optional<ResourceID> |
DeclarativeSlotPoolService.failAllocation(ResourceID taskManagerId,
AllocationID allocationId,
Exception cause) |
boolean |
SlotPool.registerTaskManager(ResourceID resourceID)
Registers a TaskExecutor with the given
ResourceID at SlotPool . |
boolean |
SlotPoolService.registerTaskManager(ResourceID taskManagerId)
Registers a TaskExecutor with the given
ResourceID at SlotPoolService . |
boolean |
DeclarativeSlotPoolService.registerTaskManager(ResourceID taskManagerId) |
void |
SlotPoolService.releaseFreeSlotsOnTaskManager(ResourceID taskManagerId,
Exception cause)
Releases all free slots belonging to the owning TaskExecutor if it has been registered.
|
void |
DeclarativeSlotPoolService.releaseFreeSlotsOnTaskManager(ResourceID taskManagerId,
Exception cause) |
ResourceCounter |
DefaultDeclarativeSlotPool.releaseSlots(ResourceID owner,
Exception cause) |
ResourceCounter |
DeclarativeSlotPool.releaseSlots(ResourceID owner,
Exception cause)
Releases all slots belonging to the owning TaskExecutor if it has been registered.
|
boolean |
SlotPool.releaseTaskManager(ResourceID resourceId,
Exception cause)
Releases a TaskExecutor with the given
ResourceID from the SlotPool . |
boolean |
SlotPoolService.releaseTaskManager(ResourceID taskManagerId,
Exception cause)
Releases a TaskExecutor with the given
ResourceID from the SlotPoolService . |
boolean |
DeclarativeSlotPoolService.releaseTaskManager(ResourceID taskManagerId,
Exception cause) |
AllocatedSlotPool.AllocatedSlotsAndReservationStatus |
AllocatedSlotPool.removeSlots(ResourceID owner)
Removes all slots belonging to the owning TaskExecutor identified by owner.
|
AllocatedSlotPool.AllocatedSlotsAndReservationStatus |
DefaultAllocatedSlotPool.removeSlots(ResourceID owner) |
Modifier and Type | Method and Description |
---|---|
void |
MetricRegistryImpl.startQueryService(RpcService rpcService,
ResourceID resourceID)
Initializes the MetricQueryService.
|
Modifier and Type | Method and Description |
---|---|
static MetricQueryService |
MetricQueryService.createMetricQueryService(RpcService rpcService,
ResourceID resourceID,
long maximumFrameSize)
Starts the MetricQueryService actor in the given actor system.
|
Modifier and Type | Method and Description |
---|---|
static TaskManagerMetricGroup |
TaskManagerMetricGroup.createTaskManagerMetricGroup(MetricRegistry metricRegistry,
String hostName,
ResourceID resourceID) |
Modifier and Type | Method and Description |
---|---|
static Tuple2<TaskManagerMetricGroup,MetricGroup> |
MetricUtils.instantiateTaskManagerMetricGroup(MetricRegistry metricRegistry,
String hostName,
ResourceID resourceID,
Optional<Time> systemResourceProbeInterval) |
Modifier and Type | Method and Description |
---|---|
ResourceID |
ResourceManagerProcessContext.getResourceId() |
ResourceID |
TaskExecutorRegistration.getResourceId() |
protected ResourceID |
StandaloneResourceManager.workerStarted(ResourceID resourceID) |
Modifier and Type | Method and Description |
---|---|
protected ResourceManager<ResourceID> |
StandaloneResourceManagerFactory.createResourceManager(Configuration configuration,
ResourceID resourceId,
RpcService rpcService,
UUID leaderSessionId,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
FatalErrorHandler fatalErrorHandler,
ClusterInformation clusterInformation,
String webInterfaceUrl,
ResourceManagerMetricGroup resourceManagerMetricGroup,
ResourceManagerRuntimeServices resourceManagerRuntimeServices,
Executor ioExecutor) |
CompletableFuture<Collection<Tuple2<ResourceID,String>>> |
ResourceManagerGateway.requestTaskManagerMetricQueryServiceAddresses(Time timeout)
Requests the paths for the TaskManager's
MetricQueryService to query. |
CompletableFuture<Collection<Tuple2<ResourceID,String>>> |
ResourceManager.requestTaskManagerMetricQueryServiceAddresses(Time timeout) |
Modifier and Type | Method and Description |
---|---|
protected Optional<WorkerType> |
ResourceManager.closeTaskManagerConnection(ResourceID resourceID,
Exception cause)
This method should be called by the framework once it detects that a currently registered
task executor has failed.
|
static ResourceManagerServiceImpl |
ResourceManagerServiceImpl.create(ResourceManagerFactory<?> resourceManagerFactory,
Configuration configuration,
ResourceID resourceId,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
FatalErrorHandler fatalErrorHandler,
ClusterInformation clusterInformation,
String webInterfaceUrl,
MetricRegistry metricRegistry,
String hostname,
Executor ioExecutor) |
protected ResourceManager<ResourceID> |
StandaloneResourceManagerFactory.createResourceManager(Configuration configuration,
ResourceID resourceId,
RpcService rpcService,
UUID leaderSessionId,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
FatalErrorHandler fatalErrorHandler,
ClusterInformation clusterInformation,
String webInterfaceUrl,
ResourceManagerMetricGroup resourceManagerMetricGroup,
ResourceManagerRuntimeServices resourceManagerRuntimeServices,
Executor ioExecutor) |
protected abstract ResourceManager<T> |
ResourceManagerFactory.createResourceManager(Configuration configuration,
ResourceID resourceId,
RpcService rpcService,
UUID leaderSessionId,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
FatalErrorHandler fatalErrorHandler,
ClusterInformation clusterInformation,
String webInterfaceUrl,
ResourceManagerMetricGroup resourceManagerMetricGroup,
ResourceManagerRuntimeServices resourceManagerRuntimeServices,
Executor ioExecutor) |
ResourceManagerProcessContext |
ResourceManagerFactory.createResourceManagerProcessContext(Configuration configuration,
ResourceID resourceId,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
FatalErrorHandler fatalErrorHandler,
ClusterInformation clusterInformation,
String webInterfaceUrl,
MetricRegistry metricRegistry,
String hostname,
Executor ioExecutor) |
void |
ResourceManagerGateway.disconnectTaskManager(ResourceID resourceID,
Exception cause)
Disconnects a TaskManager specified by the given resourceID from the
ResourceManager . |
void |
ResourceManager.disconnectTaskManager(ResourceID resourceId,
Exception cause) |
CompletableFuture<Void> |
ResourceManagerGateway.heartbeatFromJobManager(ResourceID heartbeatOrigin)
Sends the heartbeat to resource manager from job manager.
|
CompletableFuture<Void> |
ResourceManager.heartbeatFromJobManager(ResourceID resourceID) |
CompletableFuture<Void> |
ResourceManagerGateway.heartbeatFromTaskManager(ResourceID heartbeatOrigin,
TaskExecutorHeartbeatPayload heartbeatPayload)
Sends the heartbeat to resource manager from task manager.
|
CompletableFuture<Void> |
ResourceManager.heartbeatFromTaskManager(ResourceID resourceID,
TaskExecutorHeartbeatPayload heartbeatPayload) |
CompletableFuture<RegistrationResponse> |
ResourceManagerGateway.registerJobMaster(JobMasterId jobMasterId,
ResourceID jobMasterResourceId,
String jobMasterAddress,
JobID jobId,
Time timeout)
Register a
JobMaster at the resource manager. |
CompletableFuture<RegistrationResponse> |
ResourceManager.registerJobMaster(JobMasterId jobMasterId,
ResourceID jobManagerResourceId,
String jobManagerAddress,
JobID jobId,
Time timeout) |
CompletableFuture<Void> |
ResourceManager.reportClusterPartitions(ResourceID taskExecutorId,
ClusterPartitionReport clusterPartitionReport) |
CompletableFuture<TaskExecutorThreadInfoGateway> |
ResourceManagerGateway.requestTaskExecutorThreadInfoGateway(ResourceID taskManagerId,
Time timeout)
Requests the
TaskExecutorGateway . |
CompletableFuture<TaskExecutorThreadInfoGateway> |
ResourceManager.requestTaskExecutorThreadInfoGateway(ResourceID taskManagerId,
Time timeout) |
CompletableFuture<TaskManagerInfoWithSlots> |
ResourceManagerGateway.requestTaskManagerDetailsInfo(ResourceID taskManagerId,
Time timeout)
Requests detail information about the given
TaskExecutor . |
CompletableFuture<TaskManagerInfoWithSlots> |
ResourceManager.requestTaskManagerDetailsInfo(ResourceID resourceId,
Time timeout) |
CompletableFuture<TransientBlobKey> |
ResourceManagerGateway.requestTaskManagerFileUploadByName(ResourceID taskManagerId,
String fileName,
Time timeout)
Request the file upload from the given
TaskExecutor to the cluster's BlobServer . |
CompletableFuture<TransientBlobKey> |
ResourceManager.requestTaskManagerFileUploadByName(ResourceID taskManagerId,
String fileName,
Time timeout) |
CompletableFuture<TransientBlobKey> |
ResourceManagerGateway.requestTaskManagerFileUploadByType(ResourceID taskManagerId,
FileType fileType,
Time timeout)
Request the file upload from the given
TaskExecutor to the cluster's BlobServer . |
CompletableFuture<TransientBlobKey> |
ResourceManager.requestTaskManagerFileUploadByType(ResourceID taskManagerId,
FileType fileType,
Time timeout) |
CompletableFuture<Collection<LogInfo>> |
ResourceManagerGateway.requestTaskManagerLogList(ResourceID taskManagerId,
Time timeout)
Request log list from the given
TaskExecutor . |
CompletableFuture<Collection<LogInfo>> |
ResourceManager.requestTaskManagerLogList(ResourceID taskManagerId,
Time timeout) |
CompletableFuture<ThreadDumpInfo> |
ResourceManagerGateway.requestThreadDump(ResourceID taskManagerId,
Time timeout)
Requests the thread dump from the given
TaskExecutor . |
CompletableFuture<ThreadDumpInfo> |
ResourceManager.requestThreadDump(ResourceID taskManagerId,
Time timeout) |
CompletableFuture<Acknowledge> |
ResourceManagerGateway.sendSlotReport(ResourceID taskManagerResourceId,
InstanceID taskManagerRegistrationId,
SlotReport slotReport,
Time timeout)
Sends the given
SlotReport to the ResourceManager. |
CompletableFuture<Acknowledge> |
ResourceManager.sendSlotReport(ResourceID taskManagerResourceId,
InstanceID taskManagerRegistrationId,
SlotReport slotReport,
Time timeout) |
boolean |
StandaloneResourceManager.stopWorker(ResourceID resourceID) |
protected abstract WorkerType |
ResourceManager.workerStarted(ResourceID resourceID)
Callback when a worker was started.
|
protected ResourceID |
StandaloneResourceManager.workerStarted(ResourceID resourceID) |
Constructor and Description |
---|
ResourceManager(RpcService rpcService,
UUID leaderSessionId,
ResourceID resourceId,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
SlotManager slotManager,
ResourceManagerPartitionTrackerFactory clusterPartitionTrackerFactory,
BlocklistHandler.Factory blocklistHandlerFactory,
JobLeaderIdService jobLeaderIdService,
ClusterInformation clusterInformation,
FatalErrorHandler fatalErrorHandler,
ResourceManagerMetricGroup resourceManagerMetricGroup,
Time rpcTimeout,
Executor ioExecutor) |
ResourceManagerProcessContext(Configuration rmConfig,
ResourceID resourceId,
ResourceManagerRuntimeServicesConfiguration rmRuntimeServicesConfig,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
FatalErrorHandler fatalErrorHandler,
ClusterInformation clusterInformation,
String webInterfaceUrl,
MetricRegistry metricRegistry,
String hostname,
Executor ioExecutor) |
StandaloneResourceManager(RpcService rpcService,
UUID leaderSessionId,
ResourceID resourceId,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
SlotManager slotManager,
ResourceManagerPartitionTrackerFactory clusterPartitionTrackerFactory,
BlocklistHandler.Factory blocklistHandlerFactory,
JobLeaderIdService jobLeaderIdService,
ClusterInformation clusterInformation,
FatalErrorHandler fatalErrorHandler,
ResourceManagerMetricGroup resourceManagerMetricGroup,
Time startupPeriodTime,
Time rpcTimeout,
Executor ioExecutor) |
TaskExecutorRegistration(String taskExecutorAddress,
ResourceID resourceId,
int dataPort,
int jmxPort,
HardwareDescription hardwareDescription,
TaskExecutorMemoryConfiguration memoryConfiguration,
ResourceProfile defaultSlotResourceProfile,
ResourceProfile totalResourceProfile,
String nodeId) |
Modifier and Type | Method and Description |
---|---|
ResourceManager<WorkerType> |
ActiveResourceManagerFactory.createResourceManager(Configuration configuration,
ResourceID resourceId,
RpcService rpcService,
UUID leaderSessionId,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
FatalErrorHandler fatalErrorHandler,
ClusterInformation clusterInformation,
String webInterfaceUrl,
ResourceManagerMetricGroup resourceManagerMetricGroup,
ResourceManagerRuntimeServices resourceManagerRuntimeServices,
Executor ioExecutor) |
void |
ResourceEventHandler.onWorkerTerminated(ResourceID resourceId,
String diagnostics)
Notifies that the worker has been terminated.
|
void |
ActiveResourceManager.onWorkerTerminated(ResourceID resourceId,
String diagnostics) |
protected WorkerType |
ActiveResourceManager.workerStarted(ResourceID resourceID) |
Constructor and Description |
---|
ActiveResourceManager(ResourceManagerDriver<WorkerType> resourceManagerDriver,
Configuration flinkConfig,
RpcService rpcService,
UUID leaderSessionId,
ResourceID resourceId,
HeartbeatServices heartbeatServices,
DelegationTokenManager delegationTokenManager,
SlotManager slotManager,
ResourceManagerPartitionTrackerFactory clusterPartitionTrackerFactory,
BlocklistHandler.Factory blocklistHandlerFactory,
JobLeaderIdService jobLeaderIdService,
ClusterInformation clusterInformation,
FatalErrorHandler fatalErrorHandler,
ResourceManagerMetricGroup resourceManagerMetricGroup,
ThresholdMeter startWorkerFailureRater,
java.time.Duration retryInterval,
java.time.Duration workerRegistrationTimeout,
java.time.Duration previousWorkerRecoverTimeout,
Executor ioExecutor) |
Constructor and Description |
---|
UnknownTaskExecutorException(ResourceID taskExecutorId) |
Modifier and Type | Method and Description |
---|---|
ResourceID |
JobManagerRegistration.getJobManagerResourceID() |
ResourceID |
TaskExecutorConnection.getResourceID() |
Constructor and Description |
---|
JobManagerRegistration(JobID jobID,
ResourceID jobManagerResourceID,
JobMasterGateway jobManagerGateway) |
TaskExecutorConnection(ResourceID resourceID,
TaskExecutorGateway taskExecutorGateway) |
Modifier and Type | Method and Description |
---|---|
protected CompletableFuture<TransientBlobKey> |
TaskManagerCustomLogHandler.requestFileUpload(ResourceManagerGateway resourceManagerGateway,
Tuple2<ResourceID,String> taskManagerIdAndFileName) |
protected abstract CompletableFuture<TransientBlobKey> |
AbstractTaskManagerFileHandler.requestFileUpload(ResourceManagerGateway resourceManagerGateway,
Tuple2<ResourceID,String> taskManagerIdAndFileName) |
protected CompletableFuture<TransientBlobKey> |
TaskManagerStdoutFileHandler.requestFileUpload(ResourceManagerGateway resourceManagerGateway,
Tuple2<ResourceID,String> taskManagerIdAndFileName) |
protected CompletableFuture<TransientBlobKey> |
TaskManagerLogFileHandler.requestFileUpload(ResourceManagerGateway resourceManagerGateway,
Tuple2<ResourceID,String> taskManagerIdAndFileName) |
Modifier and Type | Method and Description |
---|---|
ResourceID |
TaskManagersFilterQueryParameter.convertStringToValue(String value) |
Modifier and Type | Method and Description |
---|---|
String |
TaskManagersFilterQueryParameter.convertValueToString(ResourceID value) |
Modifier and Type | Method and Description |
---|---|
ResourceID |
ResourceIDDeserializer.deserialize(org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonParser p,
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.DeserializationContext ctxt) |
Modifier and Type | Method and Description |
---|---|
void |
ResourceIDSerializer.serialize(ResourceID value,
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonGenerator gen,
org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.SerializerProvider provider) |
Modifier and Type | Method and Description |
---|---|
protected ResourceID |
TaskManagerIdPathParameter.convertFromString(String value) |
ResourceID |
TaskManagerInfo.getResourceId() |
Modifier and Type | Method and Description |
---|---|
protected String |
TaskManagerIdPathParameter.convertToString(ResourceID value) |
Constructor and Description |
---|
TaskManagerDetailsInfo(ResourceID resourceId,
String address,
int dataPort,
int jmxPort,
long lastHeartbeat,
int numberSlots,
int numberAvailableSlots,
ResourceProfileInfo totalResource,
ResourceProfileInfo freeResource,
HardwareDescription hardwareDescription,
TaskExecutorMemoryConfiguration memoryConfiguration,
Boolean blocked,
Collection<SlotInfo> allocatedSlots,
TaskManagerMetricsInfo taskManagerMetrics) |
TaskManagerInfo(ResourceID resourceId,
String address,
int dataPort,
int jmxPort,
long lastHeartbeat,
int numberSlots,
int numberAvailableSlots,
ResourceProfileInfo totalResource,
ResourceProfileInfo freeResource,
HardwareDescription hardwareDescription,
TaskExecutorMemoryConfiguration memoryConfiguration,
Boolean blocked) |
TaskManagerInfo(ResourceID resourceId,
String address,
int dataPort,
int jmxPort,
long lastHeartbeat,
int numberSlots,
int numberAvailableSlots,
ResourceProfile totalResource,
ResourceProfile freeResource,
HardwareDescription hardwareDescription,
TaskExecutorMemoryConfiguration memoryConfiguration,
Boolean blocked) |
Modifier and Type | Method and Description |
---|---|
ResourceID |
ExceptionHistoryEntry.ArchivedTaskManagerLocation.getResourceID() |
Modifier and Type | Method and Description |
---|---|
ResourceID |
ProducerDescriptor.getProducerLocation() |
ResourceID |
ShuffleEnvironmentContext.getTaskExecutorResourceId() |
Modifier and Type | Method and Description |
---|---|
Optional<ResourceID> |
NettyShuffleDescriptor.storesLocalResourcesOn() |
Optional<ResourceID> |
UnknownShuffleDescriptor.storesLocalResourcesOn() |
Optional<ResourceID> |
ShuffleDescriptor.storesLocalResourcesOn()
Returns the location of the producing task executor if the partition occupies local resources
there.
|
Modifier and Type | Method and Description |
---|---|
boolean |
NettyShuffleDescriptor.isLocalTo(ResourceID consumerLocation) |
Constructor and Description |
---|
NettyShuffleDescriptor(ResourceID producerLocation,
NettyShuffleDescriptor.PartitionConnectionInfo partitionConnectionInfo,
ResultPartitionID resultPartitionID) |
ProducerDescriptor(ResourceID producerLocation,
ExecutionAttemptID producerExecutionId,
InetAddress address,
int dataPort) |
ShuffleEnvironmentContext(Configuration configuration,
ResourceID taskExecutorResourceId,
MemorySize networkMemorySize,
boolean localCommunicationOnly,
InetAddress hostAddress,
int numberOfSlots,
String[] tmpDirPaths,
TaskEventPublisher eventPublisher,
MetricGroup parentMetricGroup,
Executor ioExecutor) |
Modifier and Type | Method and Description |
---|---|
ResourceID |
JobTable.Connection.getResourceId() |
ResourceID |
TaskManagerServicesConfiguration.getResourceID() |
ResourceID |
TaskExecutor.getResourceID() |
ResourceID |
TaskExecutorRegistrationSuccess.getResourceManagerId()
Gets the unique ID that identifies the ResourceManager.
|
Modifier and Type | Method and Description |
---|---|
JobTable.Connection |
JobTable.Job.connect(ResourceID resourceId,
JobMasterGateway jobMasterGateway,
TaskManagerActions taskManagerActions,
CheckpointResponder checkpointResponder,
GlobalAggregateManager aggregateManager,
PartitionProducerStateChecker partitionStateChecker)
Connects the job to a JobManager and associates the provided services with this
connection.
|
TaskManagerRunner.TaskExecutorService |
TaskManagerRunner.TaskExecutorServiceFactory.createTaskExecutor(Configuration configuration,
ResourceID resourceID,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
MetricRegistry metricRegistry,
BlobCacheService blobCacheService,
boolean localCommunicationOnly,
ExternalResourceInfoProvider externalResourceInfoProvider,
WorkingDirectory workingDirectory,
FatalErrorHandler fatalErrorHandler) |
static TaskManagerRunner.TaskExecutorService |
TaskManagerRunner.createTaskExecutorService(Configuration configuration,
ResourceID resourceID,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
MetricRegistry metricRegistry,
BlobCacheService blobCacheService,
boolean localCommunicationOnly,
ExternalResourceInfoProvider externalResourceInfoProvider,
WorkingDirectory workingDirectory,
FatalErrorHandler fatalErrorHandler) |
static TaskManagerServicesConfiguration |
TaskManagerServicesConfiguration.fromConfiguration(Configuration configuration,
ResourceID resourceID,
String externalAddress,
boolean localCommunicationOnly,
TaskExecutorResourceSpec taskExecutorResourceSpec,
WorkingDirectory workingDirectory)
Utility method to extract TaskManager config parameters from the configuration and to sanity
check them.
|
Optional<JobTable.Connection> |
JobTable.getConnection(ResourceID resourceId)
Gets the connection registered under resourceId.
|
Optional<JobTable.Connection> |
DefaultJobTable.getConnection(ResourceID resourceId) |
CompletableFuture<Void> |
TaskExecutorGatewayDecoratorBase.heartbeatFromJobManager(ResourceID heartbeatOrigin,
AllocatedSlotReport allocatedSlotReport) |
CompletableFuture<Void> |
TaskExecutorGateway.heartbeatFromJobManager(ResourceID heartbeatOrigin,
AllocatedSlotReport allocatedSlotReport)
Heartbeat request from the job manager.
|
CompletableFuture<Void> |
TaskExecutor.heartbeatFromJobManager(ResourceID resourceID,
AllocatedSlotReport allocatedSlotReport) |
CompletableFuture<Void> |
TaskExecutorGatewayDecoratorBase.heartbeatFromResourceManager(ResourceID heartbeatOrigin) |
CompletableFuture<Void> |
TaskExecutorGateway.heartbeatFromResourceManager(ResourceID heartbeatOrigin)
Heartbeat request from the resource manager.
|
CompletableFuture<Void> |
TaskExecutor.heartbeatFromResourceManager(ResourceID resourceID) |
static TaskExecutor |
TaskManagerRunner.startTaskManager(Configuration configuration,
ResourceID resourceID,
RpcService rpcService,
HighAvailabilityServices highAvailabilityServices,
HeartbeatServices heartbeatServices,
MetricRegistry metricRegistry,
TaskExecutorBlobService taskExecutorBlobService,
boolean localCommunicationOnly,
ExternalResourceInfoProvider externalResourceInfoProvider,
WorkingDirectory workingDirectory,
FatalErrorHandler fatalErrorHandler) |
Constructor and Description |
---|
TaskExecutorRegistrationSuccess(InstanceID registrationId,
ResourceID resourceManagerResourceId,
ClusterInformation clusterInformation)
Create a new
TaskExecutorRegistrationSuccess message. |
Modifier and Type | Method and Description |
---|---|
SlotReport |
TaskSlotTableImpl.createSlotReport(ResourceID resourceId) |
SlotReport |
TaskSlotTable.createSlotReport(ResourceID resourceId) |
Modifier and Type | Method and Description |
---|---|
ResourceID |
UnresolvedTaskManagerLocation.getResourceID() |
ResourceID |
TaskManagerLocation.getResourceID()
Gets the ID of the resource in which the TaskManager is started.
|
Constructor and Description |
---|
TaskManagerLocation(ResourceID resourceID,
InetAddress inetAddress,
int dataPort)
Constructs a new instance connection info object.
|
TaskManagerLocation(ResourceID resourceID,
InetAddress inetAddress,
int dataPort,
TaskManagerLocation.HostNameSupplier hostNameSupplier,
String nodeId)
Constructs a new instance connection info object.
|
UnresolvedTaskManagerLocation(ResourceID resourceID,
String externalAddress,
int dataPort,
String nodeId) |
Modifier and Type | Method and Description |
---|---|
CompletableFuture<Collection<Tuple2<ResourceID,String>>> |
RestfulGateway.requestTaskManagerMetricQueryServiceAddresses(Time timeout)
Requests the addresses for the TaskManagers'
MetricQueryService to query. |
CompletableFuture<Collection<Tuple2<ResourceID,String>>> |
NonLeaderRetrievalRestfulGateway.requestTaskManagerMetricQueryServiceAddresses(Time timeout) |
Modifier and Type | Method and Description |
---|---|
ResourceID |
YarnWorkerNode.getResourceID() |
Constructor and Description |
---|
YarnWorkerNode(org.apache.hadoop.yarn.api.records.Container container,
ResourceID resourceID) |
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.