Package org.apache.flink.yarn.configuration

Class YarnConfigOptions

  • @PublicEvolving
public class YarnConfigOptions
extends Object
    This class holds configuration constants used by Flink's YARN runners.

    These options are not expected to be ever configured by users explicitly.

    • Field Detail

      • APP_MASTER_VCORES

        public static final ConfigOption<Integer> APP_MASTER_VCORES
        The vcores used by YARN application master.

      • CLASSPATH_INCLUDE_USER_JAR

        public static final ConfigOption<YarnConfigOptions.UserJarInclusion> CLASSPATH_INCLUDE_USER_JAR
        Defines whether user-jars are included in the system class path as well as their positioning in the path. They can be positioned at the beginning (FIRST), at the end (LAST), or be positioned based on their name (ORDER). DISABLED means the user-jars are excluded from the system class path and as a result these jars will be loaded by user classloader.

      • APPLICATION_ATTEMPTS

        public static final ConfigOption<Integer> APPLICATION_ATTEMPTS
        Set the number of retries for failed YARN ApplicationMasters/JobManagers in high availability mode. This value is usually limited by YARN. By default, it's 1 in the standalone case and 2 in the high availability case.

        >Note: This option returns a String since Integer options must have a static default value.

      • APPLICATION_ATTEMPT_FAILURE_VALIDITY_INTERVAL

        public static final ConfigOption<Long> APPLICATION_ATTEMPT_FAILURE_VALIDITY_INTERVAL
        The config parameter defining the attemptFailuresValidityInterval of Yarn application.

      • HEARTBEAT_DELAY_SECONDS

        public static final ConfigOption<Integer> HEARTBEAT_DELAY_SECONDS
        The heartbeat interval between the Application Master and the YARN Resource Manager.

      • CONTAINER_REQUEST_HEARTBEAT_INTERVAL_MILLISECONDS

        public static final ConfigOption<Duration> CONTAINER_REQUEST_HEARTBEAT_INTERVAL_MILLISECONDS
        The heartbeat interval between the Application Master and the YARN Resource Manager if Flink is requesting containers.

      • PROPERTIES_FILE_LOCATION

        public static final ConfigOption<String> PROPERTIES_FILE_LOCATION
        When a Flink job is submitted to YARN, the JobManager's host and the number of available processing slots is written into a properties file, so that the Flink client is able to pick those details up. This configuration parameter allows changing the default location of that file (for example for environments sharing a Flink installation between users)

      • APPLICATION_MASTER_PORT

        public static final ConfigOption<String> APPLICATION_MASTER_PORT
        The config parameter defining the Pekko actor system port for the ApplicationMaster and JobManager. The port can either be a port, such as "9123", a range of ports: "50100-50200" or a list of ranges and or points: "50100-50200,50300-50400,51234". Setting the port to 0 will let the OS choose an available port.

      • APPLICATION_PRIORITY

        public static final ConfigOption<Integer> APPLICATION_PRIORITY
        A non-negative integer indicating the priority for submitting a Flink YARN application. It will only take effect if YARN priority scheduling setting is enabled. Larger integer corresponds with higher priority. If priority is negative or set to '-1'(default), Flink will unset yarn priority setting and use cluster default priority.
        See Also:
        YARN Capacity Scheduling Doc

      • FILE_REPLICATION

        public static final ConfigOption<Integer> FILE_REPLICATION
        Yarn session client uploads flink jar and user libs to file system (hdfs/s3) as local resource for yarn application context. The replication number changes the how many replica of each of these files in hdfs/s3. It is useful to accelerate this container bootstrap, when a Flink application needs more than one hundred of containers. If it is not configured, Flink will use the default replication value in hadoop configuration.

      • APPLICATION_TAGS

        public static final ConfigOption<String> APPLICATION_TAGS
        A comma-separated list of strings to use as YARN application tags.

      • APPLICATION_VIEW_ACLS

        public static final ConfigOption<String> APPLICATION_VIEW_ACLS
        Users and groups to give VIEW access. https://www.cloudera.com/documentation/enterprise/latest/topics/cm_mc_yarn_acl.html

      • APPLICATION_MODIFY_ACLS

        public static final ConfigOption<String> APPLICATION_MODIFY_ACLS
        Users and groups to give MODIFY access.

      • TASK_MANAGER_NODE_LABEL

        public static final ConfigOption<String> TASK_MANAGER_NODE_LABEL

      • PROVIDED_USRLIB_DIR

        public static final ConfigOption<String> PROVIDED_USRLIB_DIR
        Allows users to directly utilize usrlib directory in HDFS for YARN application mode. The classloader for loading jars under the usrlib will be controlled by CLASSPATH_INCLUDE_USER_JAR.

      • YARN_CONTAINER_START_COMMAND_TEMPLATE

        public static final ConfigOption<String> YARN_CONTAINER_START_COMMAND_TEMPLATE

      • EXTERNAL_RESOURCE_YARN_CONFIG_KEY_SUFFIX

        public static final String EXTERNAL_RESOURCE_YARN_CONFIG_KEY_SUFFIX
        Defines the configuration key of that external resource in Yarn. This is used as a suffix in an actual config.
        See Also:
        Constant Field Values

      • EXTERNAL_RESOURCE_YARN_CONFIG_KEY

        public static final ConfigOption<String> EXTERNAL_RESOURCE_YARN_CONFIG_KEY
        If configured, Flink will add this key to the resource profile of container request to Yarn. The value will be set to ExternalResourceOptions.EXTERNAL_RESOURCE_AMOUNT.

        It is intentionally included into user docs while unused.