org.apache.flink.runtime.resourcemanager.slotmanager

Class FineGrainedTaskManagerTracker

java.lang.Object

org.apache.flink.runtime.resourcemanager.slotmanager.FineGrainedTaskManagerTracker

public class FineGrainedTaskManagerTracker
extends Object

Implementation of TaskManagerTracker supporting fine-grained resource management.

Constructor Summary

Constructors

Constructor and Description
FineGrainedTaskManagerTracker()

Method Summary

Modifier and Type	Method and Description
void	addPendingTaskManager(PendingTaskManager pendingTaskManager) Add a new pending task manager.
void	addTaskManager(TaskExecutorConnection taskExecutorConnection, ResourceProfile totalResourceProfile, ResourceProfile defaultSlotResourceProfile) Register a new task manager.
void	addUnWantedTaskManager(InstanceID instanceId) Add an unwanted task manager.
void	clear() Removes all state from the tracker.
void	clearPendingAllocationsOfJob(JobID jobId) Clear all previous pending slot allocation records for the given job.
Optional<TaskManagerSlotInformation>	getAllocatedOrPendingSlot(AllocationID allocationId) Get the TaskManagerSlotInformation of the allocated slot with the given allocationId.
ResourceProfile	getFreeResource() Get profile of total free resources.
ResourceProfile	getFreeResourceOf(InstanceID instanceId) Get profile of free resources from the TaskManager with the given instance id.
int	getNumberFreeSlots() Get total number of free slots.
int	getNumberFreeSlotsOf(InstanceID instanceId) Get number of free slots from the TaskManager with the given instance id.
int	getNumberRegisteredSlots() Get total number of registered slots.
int	getNumberRegisteredSlotsOf(InstanceID instanceId) Get number of registered slots from the TaskManager with the given instance id.
Map<JobID,ResourceCounter>	getPendingAllocationsOfPendingTaskManager(PendingTaskManagerId pendingTaskManagerId) Get the pending allocations of the given pending task manager.
ResourceProfile	getPendingResource() Get profile of total pending resources.
Collection<PendingTaskManager>	getPendingTaskManagers() Get all pending task managers.
Collection<PendingTaskManager>	getPendingTaskManagersByTotalAndDefaultSlotResourceProfile(ResourceProfile totalResourceProfile, ResourceProfile defaultSlotResourceProfile) Get all pending task managers with given total and default slot profile.
ResourceProfile	getRegisteredResource() Get profile of total registered resources.
ResourceProfile	getRegisteredResourceOf(InstanceID instanceId) Get profile of registered resources from the TaskManager with the given instance id.
Optional<TaskManagerInfo>	getRegisteredTaskManager(InstanceID instanceId) Get the TaskManagerInfo of a registered task manager with the given instanceId
Collection<? extends TaskManagerInfo>	getRegisteredTaskManagers() Get the TaskManagerInfos of all registered task managers.
Map<InstanceID,WorkerResourceSpec>	getUnWantedTaskManager() Get unwanted task managers.
void	notifySlotStatus(AllocationID allocationId, JobID jobId, InstanceID instanceId, ResourceProfile resourceProfile, org.apache.flink.runtime.resourcemanager.slotmanager.SlotState slotState) Notifies the tracker about the slot status.
Map<JobID,ResourceCounter>	removePendingTaskManager(PendingTaskManagerId pendingTaskManagerId) Remove a pending task manager and it associated allocation records.
void	removeTaskManager(InstanceID instanceId) Unregister a task manager with the given instance id.
void	replaceAllPendingAllocations(Map<PendingTaskManagerId,Map<JobID,ResourceCounter>> pendingSlotAllocations) Clear all previous pending slot allocation records if any, and record new pending slot allocations.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FineGrainedTaskManagerTracker

public FineGrainedTaskManagerTracker()

Method Detail

replaceAllPendingAllocations

public void replaceAllPendingAllocations(Map<PendingTaskManagerId,Map<JobID,ResourceCounter>> pendingSlotAllocations)

Clear all previous pending slot allocation records if any, and record new pending slot allocations.

Parameters:

pendingSlotAllocations - new pending slot allocations be recorded

clearPendingAllocationsOfJob

public void clearPendingAllocationsOfJob(JobID jobId)

Clear all previous pending slot allocation records for the given job.

Parameters:

jobId - of the given job

addTaskManager

public void addTaskManager(TaskExecutorConnection taskExecutorConnection,
                           ResourceProfile totalResourceProfile,
                           ResourceProfile defaultSlotResourceProfile)

Register a new task manager.

Parameters:

taskExecutorConnection - of the new task manager

totalResourceProfile - of the new task manager

defaultSlotResourceProfile - of the new task manager

removeTaskManager

public void removeTaskManager(InstanceID instanceId)

Unregister a task manager with the given instance id.

Parameters:

instanceId - of the task manager

addUnWantedTaskManager

public void addUnWantedTaskManager(InstanceID instanceId)

Add an unwanted task manager.

Parameters:

instanceId - identifier of task manager.

getUnWantedTaskManager

public Map<InstanceID,WorkerResourceSpec> getUnWantedTaskManager()

Get unwanted task managers.

addPendingTaskManager

public void addPendingTaskManager(PendingTaskManager pendingTaskManager)

Add a new pending task manager.

Parameters:

pendingTaskManager - to be added

removePendingTaskManager

public Map<JobID,ResourceCounter> removePendingTaskManager(PendingTaskManagerId pendingTaskManagerId)

Remove a pending task manager and it associated allocation records.

Parameters:

pendingTaskManagerId - of the pending task manager

Returns:

the allocation records associated to the removed pending task manager

notifySlotStatus

public void notifySlotStatus(AllocationID allocationId,
                             JobID jobId,
                             InstanceID instanceId,
                             ResourceProfile resourceProfile,
                             org.apache.flink.runtime.resourcemanager.slotmanager.SlotState slotState)

Notifies the tracker about the slot status.

Parameters:

allocationId - of the slot

jobId - of the slot

instanceId - of the slot

resourceProfile - of the slot

slotState - of the slot

getPendingAllocationsOfPendingTaskManager

public Map<JobID,ResourceCounter> getPendingAllocationsOfPendingTaskManager(PendingTaskManagerId pendingTaskManagerId)

Get the pending allocations of the given pending task manager.

Parameters:

pendingTaskManagerId - of the pending task manager

Returns:

pending allocations, mapped by jobId

getRegisteredTaskManagers

public Collection<? extends TaskManagerInfo> getRegisteredTaskManagers()

Get the TaskManagerInfos of all registered task managers.

Returns:

a collection of TaskManagerInfos of all registered task managers.

getRegisteredTaskManager

public Optional<TaskManagerInfo> getRegisteredTaskManager(InstanceID instanceId)

Get the TaskManagerInfo of a registered task manager with the given instanceId

Parameters:

instanceId - of the task manager

Returns:

An Optional of TaskManagerInfo, if find, of the task manager

getAllocatedOrPendingSlot

public Optional<TaskManagerSlotInformation> getAllocatedOrPendingSlot(AllocationID allocationId)

Get the TaskManagerSlotInformation of the allocated slot with the given allocationId.

Parameters:

allocationId - of the slot

Returns:

An Optional of TaskManagerSlotInformation, if find, of the slot

getPendingTaskManagers

public Collection<PendingTaskManager> getPendingTaskManagers()

Get all pending task managers.

Returns:

a collection of PendingTaskManagers.

getPendingTaskManagersByTotalAndDefaultSlotResourceProfile

public Collection<PendingTaskManager> getPendingTaskManagersByTotalAndDefaultSlotResourceProfile(ResourceProfile totalResourceProfile,
                                                                                                 ResourceProfile defaultSlotResourceProfile)

Get all pending task managers with given total and default slot profile.

Parameters:

totalResourceProfile - of the pending task manager

defaultSlotResourceProfile - of the pending task manager

Returns:

Collection of all matching pending task managers

getNumberRegisteredSlots

public int getNumberRegisteredSlots()

Description copied from interface: ClusterResourceStatisticsProvider

Get total number of registered slots.

getNumberRegisteredSlotsOf

public int getNumberRegisteredSlotsOf(InstanceID instanceId)

Description copied from interface: ClusterResourceStatisticsProvider

Get number of registered slots from the TaskManager with the given instance id.

getNumberFreeSlots

public int getNumberFreeSlots()

Description copied from interface: ClusterResourceStatisticsProvider

Get total number of free slots.

getNumberFreeSlotsOf

public int getNumberFreeSlotsOf(InstanceID instanceId)

Description copied from interface: ClusterResourceStatisticsProvider

Get number of free slots from the TaskManager with the given instance id.

getRegisteredResource

public ResourceProfile getRegisteredResource()

Description copied from interface: ClusterResourceStatisticsProvider

Get profile of total registered resources.

getRegisteredResourceOf

public ResourceProfile getRegisteredResourceOf(InstanceID instanceId)

Description copied from interface: ClusterResourceStatisticsProvider

Get profile of registered resources from the TaskManager with the given instance id.

getFreeResource

public ResourceProfile getFreeResource()

Description copied from interface: ClusterResourceStatisticsProvider

Get profile of total free resources.

getFreeResourceOf

public ResourceProfile getFreeResourceOf(InstanceID instanceId)

Description copied from interface: ClusterResourceStatisticsProvider

Get profile of free resources from the TaskManager with the given instance id.

getPendingResource

public ResourceProfile getPendingResource()

Description copied from interface: ClusterResourceStatisticsProvider

Get profile of total pending resources.

clear

public void clear()

Removes all state from the tracker.