public class TaskManagerServices extends Object
TaskExecutor
services such as the MemoryManager
, IOManager
,
ShuffleEnvironment
. All services are exclusive to a single TaskExecutor
.
Consequently, the respective TaskExecutor
is responsible for closing them.Modifier and Type | Field and Description |
---|---|
static String |
LOCAL_STATE_SUB_DIRECTORY_ROOT |
Modifier and Type | Method and Description |
---|---|
static long |
calculateHeapSizeMB(long totalJavaMemorySizeMB,
Configuration config)
Calculates the amount of heap memory to use (to set via -Xmx and -Xms)
based on the total memory to use and the given configuration parameters.
|
static ResourceProfile |
computeSlotResourceProfile(int numOfSlots,
long managedMemorySize) |
static TaskManagerServices |
fromConfiguration(TaskManagerServicesConfiguration taskManagerServicesConfiguration,
MetricGroup taskManagerMetricGroup,
Executor taskIOExecutor)
Creates and returns the task manager services.
|
BroadcastVariableManager |
getBroadcastVariableManager() |
IOManager |
getIOManager() |
JobLeaderService |
getJobLeaderService() |
JobManagerTable |
getJobManagerTable() |
KvStateService |
getKvStateService() |
static long |
getManagedMemoryFromHeapAndManaged(Configuration config,
long heapAndManagedMemory)
Gets the size of managed memory from the heap size after subtracting network buffer memory.
|
static long |
getManagedMemoryFromProcessMemory(Configuration config,
long totalProcessMemory)
Gets the size of managed memory from the JVM process size, which at that point includes
network buffer memory, managed memory, and non-flink-managed heap memory.
|
MemoryManager |
getMemoryManager() |
static long |
getReservedNetworkMemory(Configuration config,
long totalProcessMemory)
Gets the amount of memory reserved for networking, given the total JVM memory.
|
ShuffleEnvironment<?,?> |
getShuffleEnvironment() |
TaskEventDispatcher |
getTaskEventDispatcher() |
TaskManagerLocation |
getTaskManagerLocation() |
TaskExecutorLocalStateStoresManager |
getTaskManagerStateStore() |
TaskSlotTable |
getTaskSlotTable() |
void |
shutDown()
Shuts the
TaskExecutor services down. |
@VisibleForTesting public static final String LOCAL_STATE_SUB_DIRECTORY_ROOT
public MemoryManager getMemoryManager()
public IOManager getIOManager()
public ShuffleEnvironment<?,?> getShuffleEnvironment()
public KvStateService getKvStateService()
public TaskManagerLocation getTaskManagerLocation()
public BroadcastVariableManager getBroadcastVariableManager()
public TaskSlotTable getTaskSlotTable()
public JobManagerTable getJobManagerTable()
public JobLeaderService getJobLeaderService()
public TaskExecutorLocalStateStoresManager getTaskManagerStateStore()
public TaskEventDispatcher getTaskEventDispatcher()
public void shutDown() throws FlinkException
TaskExecutor
services down.FlinkException
public static TaskManagerServices fromConfiguration(TaskManagerServicesConfiguration taskManagerServicesConfiguration, MetricGroup taskManagerMetricGroup, Executor taskIOExecutor) throws Exception
taskManagerServicesConfiguration
- task manager configurationtaskManagerMetricGroup
- metric group of the task managertaskIOExecutor
- executor for async IO operationsException
public static long calculateHeapSizeMB(long totalJavaMemorySizeMB, Configuration config)
totalJavaMemorySizeMB
- overall available memory to use (heap and off-heap)config
- configuration objectpublic static long getManagedMemoryFromProcessMemory(Configuration config, long totalProcessMemory)
public static long getManagedMemoryFromHeapAndManaged(Configuration config, long heapAndManagedMemory)
public static long getReservedNetworkMemory(Configuration config, long totalProcessMemory)
public static ResourceProfile computeSlotResourceProfile(int numOfSlots, long managedMemorySize)
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.