org.apache.flink.configuration

Class ResourceManagerOptions

java.lang.Object

org.apache.flink.configuration.ResourceManagerOptions

@PublicEvolving
public class ResourceManagerOptions
extends Object

The set of configuration options relating to the ResourceManager.

Field Summary

Fields

Modifier and Type	Field and Description
static String	CONTAINERIZED_MASTER_ENV_PREFIX Prefix for passing custom environment variables to Flink's master process.
static String	CONTAINERIZED_TASK_MANAGER_ENV_PREFIX Similar to the CONTAINERIZED_MASTER_ENV_PREFIX, this configuration prefix allows setting custom environment variables for the workers (TaskManagers).
static ConfigOption<Duration>	DECLARE_NEEDED_RESOURCE_DELAY
static ConfigOption<Integer>	IPC_PORT Defines the network port to connect to for communication with the resource manager.
static ConfigOption<String>	JOB_TIMEOUT Timeout for jobs which don't have a job manager as leader assigned.
static ConfigOption<Integer>	LOCAL_NUMBER_RESOURCE_MANAGER Deprecated.
static ConfigOption<Integer>	MAX_SLOT_NUM
static ConfigOption<Double>	MAX_TOTAL_CPU
static ConfigOption<MemorySize>	MAX_TOTAL_MEM
static ConfigOption<Integer>	MIN_SLOT_NUM
static ConfigOption<Double>	MIN_TOTAL_CPU
static ConfigOption<MemorySize>	MIN_TOTAL_MEM
static ConfigOption<Integer>	REDUNDANT_TASK_MANAGER_NUM The number of redundant task managers.
static ConfigOption<Duration>	REQUIREMENTS_CHECK_DELAY
static ConfigOption<Duration>	RESOURCE_MANAGER_PREVIOUS_WORKER_RECOVERY_TIMEOUT Timeout for ResourceManager to recover all the previous attempts workers.
static ConfigOption<Long>	SLOT_MANAGER_TASK_MANAGER_TIMEOUT Deprecated. Use TASK_MANAGER_TIMEOUT.
static ConfigOption<Long>	STANDALONE_CLUSTER_STARTUP_PERIOD_TIME Time in milliseconds of the start-up period of a standalone cluster.
static ConfigOption<Double>	START_WORKER_MAX_FAILURE_RATE The maximum number of start worker failures (Native Kubernetes / Yarn) per minute before pausing requesting new workers.
static ConfigOption<Duration>	START_WORKER_RETRY_INTERVAL The time to wait before requesting new workers (Native Kubernetes / Yarn) once the max failure rate of starting workers (START_WORKER_MAX_FAILURE_RATE) is reached.
static ConfigOption<Duration>	TASK_MANAGER_REGISTRATION_TIMEOUT Timeout for TaskManagers to register at the active resource managers.
static ConfigOption<Boolean>	TASK_MANAGER_RELEASE_WHEN_RESULT_CONSUMED Deprecated. The default value should be reasonable enough in all cases, this option is to fallback to older behaviour which will be removed or refactored in future.
static ConfigOption<Long>	TASK_MANAGER_TIMEOUT The timeout for an idle task manager to be released, in milliseconds.

Method Summary

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

JOB_TIMEOUT

public static final ConfigOption<String> JOB_TIMEOUT

Timeout for jobs which don't have a job manager as leader assigned.

LOCAL_NUMBER_RESOURCE_MANAGER

@Deprecated
public static final ConfigOption<Integer> LOCAL_NUMBER_RESOURCE_MANAGER

Deprecated.

This option is not used any more.

IPC_PORT

public static final ConfigOption<Integer> IPC_PORT

Defines the network port to connect to for communication with the resource manager. By default, the port of the JobManager, because the same ActorSystem is used. Its not possible to use this configuration key to define port ranges.

MIN_SLOT_NUM

public static final ConfigOption<Integer> MIN_SLOT_NUM

MAX_SLOT_NUM

public static final ConfigOption<Integer> MAX_SLOT_NUM

MIN_TOTAL_CPU

public static final ConfigOption<Double> MIN_TOTAL_CPU

MAX_TOTAL_CPU

public static final ConfigOption<Double> MAX_TOTAL_CPU

MIN_TOTAL_MEM

public static final ConfigOption<MemorySize> MIN_TOTAL_MEM

MAX_TOTAL_MEM

public static final ConfigOption<MemorySize> MAX_TOTAL_MEM

REDUNDANT_TASK_MANAGER_NUM

public static final ConfigOption<Integer> REDUNDANT_TASK_MANAGER_NUM

The number of redundant task managers. Redundant task managers are extra task managers started by Flink, in order to speed up job recovery in case of failures due to task manager lost. Note that this feature is available only to the active deployments (native K8s, Yarn). For fine-grained resource requirement, Redundant resources will be reserved, but it is possible that we have many small pieces of free resources form multiple TMs, which added up larger than the desired redundant resources, but each piece is too small to match the resource requirement of tasks from the failed worker.

START_WORKER_MAX_FAILURE_RATE

public static final ConfigOption<Double> START_WORKER_MAX_FAILURE_RATE

The maximum number of start worker failures (Native Kubernetes / Yarn) per minute before pausing requesting new workers. Once the threshold is reached, subsequent worker requests will be postponed to after a configured retry interval (START_WORKER_RETRY_INTERVAL).

START_WORKER_RETRY_INTERVAL

public static final ConfigOption<Duration> START_WORKER_RETRY_INTERVAL

The time to wait before requesting new workers (Native Kubernetes / Yarn) once the max failure rate of starting workers (START_WORKER_MAX_FAILURE_RATE) is reached.

REQUIREMENTS_CHECK_DELAY

public static final ConfigOption<Duration> REQUIREMENTS_CHECK_DELAY

DECLARE_NEEDED_RESOURCE_DELAY

public static final ConfigOption<Duration> DECLARE_NEEDED_RESOURCE_DELAY

STANDALONE_CLUSTER_STARTUP_PERIOD_TIME

public static final ConfigOption<Long> STANDALONE_CLUSTER_STARTUP_PERIOD_TIME

Time in milliseconds of the start-up period of a standalone cluster. During this time, resource manager of the standalone cluster expects new task executors to be registered, and will not fail slot requests that can not be satisfied by any current registered slots. After this time, it will fail pending and new coming requests immediately that can not be satisfied by registered slots. If not set, JobManagerOptions.SLOT_REQUEST_TIMEOUT will be used by default.

SLOT_MANAGER_TASK_MANAGER_TIMEOUT

@Deprecated
public static final ConfigOption<Long> SLOT_MANAGER_TASK_MANAGER_TIMEOUT

Deprecated. Use TASK_MANAGER_TIMEOUT.

The timeout for an idle task manager to be released, in milliseconds.

TASK_MANAGER_TIMEOUT

public static final ConfigOption<Long> TASK_MANAGER_TIMEOUT

The timeout for an idle task manager to be released, in milliseconds.

TASK_MANAGER_RELEASE_WHEN_RESULT_CONSUMED

@Deprecated
public static final ConfigOption<Boolean> TASK_MANAGER_RELEASE_WHEN_RESULT_CONSUMED

Deprecated. The default value should be reasonable enough in all cases, this option is to fallback to older behaviour which will be removed or refactored in future.

Release task executor only when each produced result partition is either consumed or failed.

Currently, produced result partition is released when it fails or consumer sends close request to confirm successful end of consumption and to close the communication channel.

CONTAINERIZED_MASTER_ENV_PREFIX

public static final String CONTAINERIZED_MASTER_ENV_PREFIX

Prefix for passing custom environment variables to Flink's master process. For example for passing LD_LIBRARY_PATH as an env variable to the AppMaster, set: containerized.master.env.LD_LIBRARY_PATH: "/usr/lib/native" in the config.yaml.

See Also:

Constant Field Values

CONTAINERIZED_TASK_MANAGER_ENV_PREFIX

public static final String CONTAINERIZED_TASK_MANAGER_ENV_PREFIX

Similar to the CONTAINERIZED_MASTER_ENV_PREFIX, this configuration prefix allows setting custom environment variables for the workers (TaskManagers).

See Also:

Constant Field Values

TASK_MANAGER_REGISTRATION_TIMEOUT

public static final ConfigOption<Duration> TASK_MANAGER_REGISTRATION_TIMEOUT

Timeout for TaskManagers to register at the active resource managers.

RESOURCE_MANAGER_PREVIOUS_WORKER_RECOVERY_TIMEOUT

public static final ConfigOption<Duration> RESOURCE_MANAGER_PREVIOUS_WORKER_RECOVERY_TIMEOUT

Timeout for ResourceManager to recover all the previous attempts workers.