Package org.apache.flink.runtime.taskexecutor.slot

Class TaskSlotTableImpl<T extends TaskSlotPayload>

java.lang.Object

org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl<T>

All Implemented Interfaces:

AutoCloseable, TaskSlotTable<T>, TimeoutListener<AllocationID>, AutoCloseableAsync

public class TaskSlotTableImpl<T extends TaskSlotPayload>
extends Object
implements TaskSlotTable<T>

Default implementation of TaskSlotTable.

Constructor Summary

Constructors

Constructor	Description
TaskSlotTableImpl(int numberSlots, ResourceProfile totalAvailableResourceProfile, ResourceProfile defaultSlotResourceProfile, int memoryPageSize, TimerService<AllocationID> timerService, Executor memoryVerificationExecutor)

Method Summary

Modifier and Type	Method	Description
boolean	addTask(T task)	Add the given task to the slot identified by the task's allocation id.
boolean	allocateSlot(int index, JobID jobId, AllocationID allocationId, Duration slotTimeout)	Allocate the slot with the given index for the given job and allocation id.
boolean	allocateSlot(int requestedIndex, JobID jobId, AllocationID allocationId, ResourceProfile resourceProfile, Duration slotTimeout)	Allocate the slot with the given index for the given job and allocation id.
CompletableFuture<Void>	closeAsync()	Trigger the closing of the resource and return the corresponding close future.
SlotReport	createSlotReport(ResourceID resourceId)
int	freeSlot(AllocationID allocationId, Throwable cause)	Tries to free the slot.
Set<AllocationID>	getActiveTaskSlotAllocationIds()	Returns the AllocationID of any active task listed in this TaskSlotTable.
Set<AllocationID>	getActiveTaskSlotAllocationIdsPerJob(JobID jobId)	Returns the AllocationID of active TaskSlots attached to the job with the given JobID.
Iterator<TaskSlot<T>>	getAllocatedSlots(JobID jobId)	Return an iterator of allocated slots for the given job id.
Set<AllocationID>	getAllocationIdsPerJob(JobID jobId)	Returns the all AllocationID for the given job.
AllocationID	getCurrentAllocation(int index)	Get the current allocation for the task slot with the given index.
JobID	getOwningJob(AllocationID allocationId)	Returns the owning job of the TaskSlot identified by the given AllocationID.
T	getTask(ExecutionAttemptID executionAttemptID)	Get the task for the given execution attempt id.
MemoryManager	getTaskMemoryManager(AllocationID allocationID)	Get the memory manager of the slot allocated for the task.
Iterator<T>	getTasks(JobID jobId)	Return an iterator over all tasks for a given job.
boolean	hasAllocatedSlots(JobID jobId)	Check whether the job has allocated (not active) slots.
boolean	isAllocated(int index, JobID jobId, AllocationID allocationId)	Check whether the slot for the given index is allocated for the given job and allocation id.
boolean	isClosed()
boolean	isSlotFree(int index)	Check whether the task slot with the given index is free.
boolean	isValidTimeout(AllocationID allocationId, UUID ticket)	Check whether the timeout with ticket is valid for the given allocation id.
boolean	markSlotActive(AllocationID allocationId)	Marks the slot under the given allocation id as active.
boolean	markSlotInactive(AllocationID allocationId, Duration slotTimeout)	Marks the slot under the given allocation id as inactive.
void	notifyTimeout(AllocationID key, UUID ticket)	Notify the listener about the timeout for an event identified by key.
T	removeTask(ExecutionAttemptID executionAttemptID)	Remove the task with the given execution attempt id from its task slot.
void	start(SlotActions initialSlotActions, ComponentMainThreadExecutor mainThreadExecutor)	Start the task slot table with the given slot actions.
boolean	tryMarkSlotActive(JobID jobId, AllocationID allocationId)	Try to mark the specified slot as active if it has been allocated by the given job.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.util.AutoCloseableAsync

close

Methods inherited from interface org.apache.flink.runtime.taskexecutor.slot.TaskSlotTable

freeSlot

Constructor Detail

TaskSlotTableImpl

public TaskSlotTableImpl(int numberSlots,
                         ResourceProfile totalAvailableResourceProfile,
                         ResourceProfile defaultSlotResourceProfile,
                         int memoryPageSize,
                         TimerService<AllocationID> timerService,
                         Executor memoryVerificationExecutor)

Method Detail

start

public void start(SlotActions initialSlotActions,
                  ComponentMainThreadExecutor mainThreadExecutor)

Description copied from interface: TaskSlotTable

Start the task slot table with the given slot actions.

Specified by:

start in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

initialSlotActions - to use for slot actions

mainThreadExecutor - ComponentMainThreadExecutor to schedule internal calls to the main thread

closeAsync

public CompletableFuture<Void> closeAsync()

Description copied from interface: AutoCloseableAsync

Trigger the closing of the resource and return the corresponding close future.

Specified by:

closeAsync in interface AutoCloseableAsync

Returns:

Future which is completed once the resource has been closed

isClosed

@VisibleForTesting
public boolean isClosed()

getAllocationIdsPerJob

public Set<AllocationID> getAllocationIdsPerJob(JobID jobId)

Description copied from interface: TaskSlotTable

Returns the all AllocationID for the given job.

Specified by:

getAllocationIdsPerJob in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

jobId - for which to return the set of AllocationID.

Returns:

Set of AllocationID for the given job

getActiveTaskSlotAllocationIds

public Set<AllocationID> getActiveTaskSlotAllocationIds()

Description copied from interface: TaskSlotTable

Returns the AllocationID of any active task listed in this TaskSlotTable.

Specified by:

getActiveTaskSlotAllocationIds in interface TaskSlotTable<T extends TaskSlotPayload>

Returns:

The AllocationID of any active task.

getActiveTaskSlotAllocationIdsPerJob

public Set<AllocationID> getActiveTaskSlotAllocationIdsPerJob(JobID jobId)

Description copied from interface: TaskSlotTable

Returns the AllocationID of active TaskSlots attached to the job with the given JobID.

Specified by:

getActiveTaskSlotAllocationIdsPerJob in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

jobId - The JobID of the job for which the AllocationIDs of the attached active TaskSlots shall be returned.

Returns:

A set of AllocationIDs that belong to active TaskSlots having the passed JobID.

createSlotReport

public SlotReport createSlotReport(ResourceID resourceId)

Specified by:

createSlotReport in interface TaskSlotTable<T extends TaskSlotPayload>

allocateSlot

@VisibleForTesting
public boolean allocateSlot(int index,
                            JobID jobId,
                            AllocationID allocationId,
                            Duration slotTimeout)

Description copied from interface: TaskSlotTable

Allocate the slot with the given index for the given job and allocation id. If negative index is given, a new auto increasing index will be generated. Returns true if the slot could be allocated. Otherwise it returns false.

Specified by:

allocateSlot in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

index - of the task slot to allocate, use negative value for dynamic slot allocation

jobId - to allocate the task slot for

allocationId - identifying the allocation

slotTimeout - until the slot times out

Returns:

True if the task slot could be allocated; otherwise false

allocateSlot

public boolean allocateSlot(int requestedIndex,
                            JobID jobId,
                            AllocationID allocationId,
                            ResourceProfile resourceProfile,
                            Duration slotTimeout)

Description copied from interface: TaskSlotTable

Allocate the slot with the given index for the given job and allocation id. If negative index is given, a new auto increasing index will be generated. Returns true if the slot could be allocated. Otherwise it returns false.

Specified by:

allocateSlot in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

requestedIndex - of the task slot to allocate, use negative value for dynamic slot allocation

jobId - to allocate the task slot for

allocationId - identifying the allocation

resourceProfile - of the requested slot, used only for dynamic slot allocation and will be ignored otherwise

slotTimeout - until the slot times out

Returns:

True if the task slot could be allocated; otherwise false

markSlotActive

public boolean markSlotActive(AllocationID allocationId)
                       throws SlotNotFoundException

Description copied from interface: TaskSlotTable

Marks the slot under the given allocation id as active. If the slot could not be found, then a SlotNotFoundException is thrown.

Specified by:

markSlotActive in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

allocationId - to identify the task slot to mark as active

Returns:

True if the slot could be marked active; otherwise false

Throws:

SlotNotFoundException - if the slot could not be found for the given allocation id

markSlotInactive

public boolean markSlotInactive(AllocationID allocationId,
                                Duration slotTimeout)
                         throws SlotNotFoundException

Description copied from interface: TaskSlotTable

Marks the slot under the given allocation id as inactive. If the slot could not be found, then a SlotNotFoundException is thrown.

Specified by:

markSlotInactive in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

allocationId - to identify the task slot to mark as inactive

slotTimeout - until the slot times out

Returns:

True if the slot could be marked inactive

Throws:

SlotNotFoundException - if the slot could not be found for the given allocation id

freeSlot

public int freeSlot(AllocationID allocationId,
                    Throwable cause)
             throws SlotNotFoundException

Description copied from interface: TaskSlotTable

Tries to free the slot. If the slot is empty it will set the state of the task slot to free and return its index. If the slot is not empty, then it will set the state of the task slot to releasing, fail all tasks and return -1.

Specified by:

freeSlot in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

allocationId - identifying the task slot to be freed

cause - to fail the tasks with if slot is not empty

Returns:

Index of the freed slot if the slot could be freed; otherwise -1

Throws:

SlotNotFoundException - if there is not task slot for the given allocation id

isValidTimeout

public boolean isValidTimeout(AllocationID allocationId,
                              UUID ticket)

Description copied from interface: TaskSlotTable

Check whether the timeout with ticket is valid for the given allocation id.

Specified by:

isValidTimeout in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

allocationId - to check against

ticket - of the timeout

Returns:

True if the timeout is valid; otherwise false

isAllocated

public boolean isAllocated(int index,
                           JobID jobId,
                           AllocationID allocationId)

Description copied from interface: TaskSlotTable

Check whether the slot for the given index is allocated for the given job and allocation id.

Specified by:

isAllocated in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

index - of the task slot

jobId - for which the task slot should be allocated

allocationId - which should match the task slot's allocation id

Returns:

True if the given task slot is allocated for the given job and allocation id

tryMarkSlotActive

public boolean tryMarkSlotActive(JobID jobId,
                                 AllocationID allocationId)

Description copied from interface: TaskSlotTable

Try to mark the specified slot as active if it has been allocated by the given job.

Specified by:

tryMarkSlotActive in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

jobId - of the allocated slot

allocationId - identifying the allocation

Returns:

True if the task slot could be marked active.

isSlotFree

public boolean isSlotFree(int index)

Description copied from interface: TaskSlotTable

Check whether the task slot with the given index is free.

Specified by:

isSlotFree in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

index - of the task slot

Returns:

True if the task slot is free; otherwise false

hasAllocatedSlots

public boolean hasAllocatedSlots(JobID jobId)

Description copied from interface: TaskSlotTable

Check whether the job has allocated (not active) slots.

Specified by:

hasAllocatedSlots in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

jobId - for which to check for allocated slots

Returns:

True if there are allocated slots for the given job id.

getAllocatedSlots

public Iterator<TaskSlot<T>> getAllocatedSlots(JobID jobId)

Description copied from interface: TaskSlotTable

Return an iterator of allocated slots for the given job id.

Specified by:

getAllocatedSlots in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

jobId - for which to return the allocated slots

Returns:

Iterator of allocated slots.

getOwningJob

@Nullable
public JobID getOwningJob(AllocationID allocationId)

Description copied from interface: TaskSlotTable

Returns the owning job of the TaskSlot identified by the given AllocationID.

Specified by:

getOwningJob in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

allocationId - identifying the slot for which to retrieve the owning job

Returns:

Owning job of the specified TaskSlot or null if there is no slot for the given allocation id or if the slot has no owning job assigned

addTask

public boolean addTask(T task)
                throws SlotNotFoundException,
                       SlotNotActiveException

Description copied from interface: TaskSlotTable

Add the given task to the slot identified by the task's allocation id.

Specified by:

addTask in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

task - to add to the task slot with the respective allocation id

Returns:

True if the task could be added to the task slot; otherwise false

Throws:

SlotNotFoundException - if there was no slot for the given allocation id

SlotNotActiveException - if there was no slot active for task's job and allocation id

removeTask

public T removeTask(ExecutionAttemptID executionAttemptID)

Description copied from interface: TaskSlotTable

Remove the task with the given execution attempt id from its task slot. If the owning task slot is in state releasing and empty after removing the task, the slot is freed via the slot actions.

Specified by:

removeTask in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

executionAttemptID - identifying the task to remove

Returns:

The removed task if there is any for the given execution attempt id; otherwise null

getTask

public T getTask(ExecutionAttemptID executionAttemptID)

Description copied from interface: TaskSlotTable

Get the task for the given execution attempt id. If none could be found, then return null.

Specified by:

getTask in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

executionAttemptID - identifying the requested task

Returns:

The task for the given execution attempt id if it exist; otherwise null

getTasks

public Iterator<T> getTasks(JobID jobId)

Description copied from interface: TaskSlotTable

Return an iterator over all tasks for a given job.

Specified by:

getTasks in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

jobId - identifying the job of the requested tasks

Returns:

Iterator over all task for a given job

getCurrentAllocation

public AllocationID getCurrentAllocation(int index)

Description copied from interface: TaskSlotTable

Get the current allocation for the task slot with the given index.

Specified by:

getCurrentAllocation in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

index - identifying the slot for which the allocation id shall be retrieved

Returns:

Allocation id of the specified slot if allocated; otherwise null

getTaskMemoryManager

public MemoryManager getTaskMemoryManager(AllocationID allocationID)
                                   throws SlotNotFoundException

Description copied from interface: TaskSlotTable

Get the memory manager of the slot allocated for the task.

Specified by:

getTaskMemoryManager in interface TaskSlotTable<T extends TaskSlotPayload>

Parameters:

allocationID - allocation id of the slot allocated for the task

Returns:

the memory manager of the slot allocated for the task

Throws:

SlotNotFoundException

notifyTimeout

public void notifyTimeout(AllocationID key,
                          UUID ticket)

Description copied from interface: TimeoutListener

Notify the listener about the timeout for an event identified by key. Additionally the method is called with the timeout ticket which allows to identify outdated timeout events.

Specified by:

notifyTimeout in interface TimeoutListener<T extends TaskSlotPayload>

Parameters:

key - identifying the timed out event

ticket - used to check whether the timeout is still valid