org.apache.flink.runtime.util.config.memory

Class ProcessMemoryUtils<FM extends FlinkMemory>

java.lang.Object

org.apache.flink.runtime.util.config.memory.ProcessMemoryUtils<FM>

Type Parameters:

FM - the FLink memory component structure

public class ProcessMemoryUtils<FM extends FlinkMemory>
extends Object

Common utils to parse JVM process memory configuration for JM or TM.

The utility calculates all common process memory components from CommonProcessMemorySpec.

It is required to configure at least one subset of the following options and recommended to configure only one:

• ProcessMemoryOptions.getRequiredFineGrainedOptions()
• ProcessMemoryOptions.getTotalFlinkMemoryOption()
• ProcessMemoryOptions.getTotalProcessMemoryOption()

Otherwise the calculation fails.

The utility derives the Total Process Memory from the Total Flink Memory and JVM components and back. To perform the calculations, it uses the provided ProcessMemoryOptions which are different for different Flink processes: JM/TM.

The utility also calls the provided FlinkMemoryUtils to derive FlinkMemory components from ProcessMemoryOptions.getRequiredFineGrainedOptions() or from the Total Flink memory. The concrete FlinkMemoryUtils is implemented for the respective processes: JM/TM, according to the specific structure of their FlinkMemory.

Constructor Summary

Constructors

Constructor and Description
ProcessMemoryUtils(ProcessMemoryOptions options, FlinkMemoryUtils<FM> flinkMemoryUtils)

Method Summary

Modifier and Type	Method and Description
JvmMetaspaceAndOverhead	deriveJvmMetaspaceAndOverheadFromTotalFlinkMemory(Configuration config, MemorySize totalFlinkMemorySize)
static MemorySize	deriveWithFraction(String memoryDescription, MemorySize base, RangeFraction rangeFraction)
static MemorySize	deriveWithInverseFraction(String memoryDescription, MemorySize base, RangeFraction rangeFraction)
static String	generateJvmParametersStr(ProcessMemorySpec processSpec)
static String	generateJvmParametersStr(ProcessMemorySpec processSpec, boolean enableDirectMemoryLimit)
static MemorySize	getMemorySizeFromConfig(Configuration config, ConfigOption<MemorySize> option)
static RangeFraction	getRangeFraction(MemorySize minSize, MemorySize maxSize, ConfigOption<Float> fractionOption, Configuration config)
CommonProcessMemorySpec<FM>	memoryProcessSpecFromConfig(Configuration config)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ProcessMemoryUtils

public ProcessMemoryUtils(ProcessMemoryOptions options,
                          FlinkMemoryUtils<FM> flinkMemoryUtils)

Method Detail

memoryProcessSpecFromConfig

public CommonProcessMemorySpec<FM> memoryProcessSpecFromConfig(Configuration config)

deriveJvmMetaspaceAndOverheadFromTotalFlinkMemory

public JvmMetaspaceAndOverhead deriveJvmMetaspaceAndOverheadFromTotalFlinkMemory(Configuration config,
                                                                                 MemorySize totalFlinkMemorySize)

getMemorySizeFromConfig

public static MemorySize getMemorySizeFromConfig(Configuration config,
                                                 ConfigOption<MemorySize> option)

getRangeFraction

public static RangeFraction getRangeFraction(MemorySize minSize,
                                             MemorySize maxSize,
                                             ConfigOption<Float> fractionOption,
                                             Configuration config)

deriveWithFraction

public static MemorySize deriveWithFraction(String memoryDescription,
                                            MemorySize base,
                                            RangeFraction rangeFraction)

deriveWithInverseFraction

public static MemorySize deriveWithInverseFraction(String memoryDescription,
                                                   MemorySize base,
                                                   RangeFraction rangeFraction)

generateJvmParametersStr

public static String generateJvmParametersStr(ProcessMemorySpec processSpec)

generateJvmParametersStr

public static String generateJvmParametersStr(ProcessMemorySpec processSpec,
                                              boolean enableDirectMemoryLimit)