org.apache.flink.runtime.resourcemanager.slotmanager

Class SlotManager

java.lang.Object

org.apache.flink.runtime.resourcemanager.slotmanager.SlotManager

All Implemented Interfaces:

AutoCloseable

public class SlotManager
extends Object
implements AutoCloseable

The slot manager is responsible for maintaining a view on all registered task manager slots, their allocation and all pending slot requests. Whenever a new slot is registered or and allocated slot is freed, then it tries to fulfill another pending slot request. Whenever there are not enough slots available the slot manager will notify the resource manager about it via ResourceActions.allocateResource(ResourceProfile).

In order to free resources and avoid resource leaks, idling task managers (task managers whose slots are currently not used) and pending slot requests time out triggering their release and failure, respectively.

Constructor Summary

Constructors

Constructor and Description
SlotManager(ScheduledExecutor scheduledExecutor, Time taskManagerRequestTimeout, Time slotRequestTimeout, Time taskManagerTimeout)

Method Summary

Modifier and Type	Method and Description
void	close() Closes the slot manager.
protected PendingSlotRequest	findMatchingRequest(ResourceProfile slotResourceProfile) Finds a matching slot request for a given resource profile.
protected TaskManagerSlot	findMatchingSlot(ResourceProfile requestResourceProfile) Finds a matching slot for a given resource profile.
void	freeSlot(SlotID slotId, AllocationID allocationId) Free the given slot from the given allocation.
int	getNumberFreeSlots()
int	getNumberFreeSlotsOf(InstanceID instanceId)
int	getNumberPendingSlotRequests()
int	getNumberRegisteredSlots()
int	getNumberRegisteredSlotsOf(InstanceID instanceId)
boolean	registerSlotRequest(SlotRequest slotRequest) Requests a slot with the respective resource profile.
void	registerTaskManager(TaskExecutorConnection taskExecutorConnection, SlotReport initialSlotReport) Registers a new task manager at the slot manager.
boolean	reportSlotStatus(InstanceID instanceId, SlotReport slotReport) Reports the current slot allocations for a task manager identified by the given instance id.
void	start(ResourceManagerId newResourceManagerId, Executor newMainThreadExecutor, ResourceActions newResourceActions) Starts the slot manager with the given leader id and resource manager actions.
void	suspend() Suspends the component.
boolean	unregisterSlotRequest(AllocationID allocationId) Cancels and removes a pending slot request with the given allocation id.
boolean	unregisterTaskManager(InstanceID instanceId) Unregisters the task manager identified by the given instance id and its associated slots from the slot manager.
void	unregisterTaskManagersAndReleaseResources()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SlotManager

public SlotManager(ScheduledExecutor scheduledExecutor,
                   Time taskManagerRequestTimeout,
                   Time slotRequestTimeout,
                   Time taskManagerTimeout)

Method Detail

getNumberRegisteredSlots

public int getNumberRegisteredSlots()

getNumberRegisteredSlotsOf

public int getNumberRegisteredSlotsOf(InstanceID instanceId)

getNumberFreeSlots

public int getNumberFreeSlots()

getNumberFreeSlotsOf

public int getNumberFreeSlotsOf(InstanceID instanceId)

getNumberPendingSlotRequests

public int getNumberPendingSlotRequests()

start

public void start(ResourceManagerId newResourceManagerId,
                  Executor newMainThreadExecutor,
                  ResourceActions newResourceActions)

Starts the slot manager with the given leader id and resource manager actions.

Parameters:

newResourceManagerId - to use for communication with the task managers

newMainThreadExecutor - to use to run code in the ResourceManager's main thread

newResourceActions - to use for resource (de-)allocations

suspend

public void suspend()

Suspends the component. This clears the internal state of the slot manager.

close

public void close()
           throws Exception

Closes the slot manager.

Specified by:

close in interface AutoCloseable

Throws:

Exception - if the close operation fails

registerSlotRequest

public boolean registerSlotRequest(SlotRequest slotRequest)
                            throws SlotManagerException

Requests a slot with the respective resource profile.

Parameters:

slotRequest - specifying the requested slot specs

Returns:

true if the slot request was registered; false if the request is a duplicate

Throws:

SlotManagerException - if the slot request failed (e.g. not enough resources left)

unregisterSlotRequest

public boolean unregisterSlotRequest(AllocationID allocationId)

Cancels and removes a pending slot request with the given allocation id. If there is no such pending request, then nothing is done.

Parameters:

allocationId - identifying the pending slot request

Returns:

True if a pending slot request was found; otherwise false

registerTaskManager

public void registerTaskManager(TaskExecutorConnection taskExecutorConnection,
                                SlotReport initialSlotReport)

Registers a new task manager at the slot manager. This will make the task managers slots known and, thus, available for allocation.

Parameters:

taskExecutorConnection - for the new task manager

initialSlotReport - for the new task manager

unregisterTaskManager

public boolean unregisterTaskManager(InstanceID instanceId)

Unregisters the task manager identified by the given instance id and its associated slots from the slot manager.

Parameters:

instanceId - identifying the task manager to unregister

Returns:

True if there existed a registered task manager with the given instance id

reportSlotStatus

public boolean reportSlotStatus(InstanceID instanceId,
                                SlotReport slotReport)

Reports the current slot allocations for a task manager identified by the given instance id.

Parameters:

instanceId - identifying the task manager for which to report the slot status

slotReport - containing the status for all of its slots

Returns:

true if the slot status has been updated successfully, otherwise false

freeSlot

public void freeSlot(SlotID slotId,
                     AllocationID allocationId)

Free the given slot from the given allocation. If the slot is still allocated by the given allocation id, then the slot will be marked as free and will be subject to new slot requests.

Parameters:

slotId - identifying the slot to free

allocationId - with which the slot is presumably allocated

findMatchingRequest

protected PendingSlotRequest findMatchingRequest(ResourceProfile slotResourceProfile)

Finds a matching slot request for a given resource profile. If there is no such request, the method returns null.

Note: If you want to change the behaviour of the slot manager wrt slot allocation and request fulfillment, then you should override this method.

Parameters:

slotResourceProfile - defining the resources of an available slot

Returns:

A matching slot request which can be deployed in a slot with the given resource profile. Null if there is no such slot request pending.

findMatchingSlot

protected TaskManagerSlot findMatchingSlot(ResourceProfile requestResourceProfile)

Finds a matching slot for a given resource profile. A matching slot has at least as many resources available as the given resource profile. If there is no such slot available, then the method returns null.

Note: If you want to change the behaviour of the slot manager wrt slot allocation and request fulfillment, then you should override this method.

Parameters:

requestResourceProfile - specifying the resource requirements for the a slot request

Returns:

A matching slot which fulfills the given resource profile. Null if there is no such slot available.

unregisterTaskManagersAndReleaseResources

@VisibleForTesting
public void unregisterTaskManagersAndReleaseResources()