org.apache.flink.runtime.clusterframework

Class BootstrapTools

java.lang.Object

org.apache.flink.runtime.clusterframework.BootstrapTools

public class BootstrapTools
extends Object

Tools for starting JobManager and TaskManager processes.

Method Summary

Modifier and Type	Method and Description
static Configuration	cloneConfiguration(Configuration configuration) Clones the given configuration and resets instance specific config options.
static String	escapeForDifferentOS(String value) Escape all the dynamic property values.
static String	escapeWithDoubleQuote(String value)
static String	escapeWithSingleQuote(String value)
static String	getDynamicPropertiesAsString(Configuration baseConfig, Configuration targetConfig) Get dynamic properties based on two Flink configurations.
static org.apache.commons.cli.Option	newDynamicPropertiesOption() Get an instance of the dynamic properties option.
static Configuration	parseDynamicProperties(org.apache.commons.cli.CommandLine cmd) Parse the dynamic properties (passed on the command line).
static void	substituteDeprecatedConfigKey(Configuration config, String deprecated, String designated) Sets the value of a new config key to the value of a deprecated config key.
static void	substituteDeprecatedConfigPrefix(Configuration config, String deprecatedPrefix, String designatedPrefix) Sets the value of a new config key to the value of a deprecated config key.
static void	updateTmpDirectoriesInConfiguration(Configuration configuration, String defaultDirs) Set temporary configuration directories if necessary.
static void	writeConfiguration(Configuration cfg, File file) Writes a Flink YAML config file from a Flink Configuration object.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

writeConfiguration

public static void writeConfiguration(Configuration cfg,
                                      File file)
                               throws IOException

Writes a Flink YAML config file from a Flink Configuration object.

Parameters:

cfg - The Flink config

file - The File to write to

Throws:

IOException

substituteDeprecatedConfigKey

public static void substituteDeprecatedConfigKey(Configuration config,
                                                 String deprecated,
                                                 String designated)

Sets the value of a new config key to the value of a deprecated config key.

Parameters:

config - Config to write

deprecated - The old config key

designated - The new config key

substituteDeprecatedConfigPrefix

public static void substituteDeprecatedConfigPrefix(Configuration config,
                                                    String deprecatedPrefix,
                                                    String designatedPrefix)

Sets the value of a new config key to the value of a deprecated config key. Taking into account the changed prefix.

Parameters:

config - Config to write

deprecatedPrefix - Old prefix of key

designatedPrefix - New prefix of key

newDynamicPropertiesOption

public static org.apache.commons.cli.Option newDynamicPropertiesOption()

Get an instance of the dynamic properties option.

Dynamic properties allow the user to specify additional configuration values with -D, such as -Dfs.overwrite-files=true -Dtaskmanager.memory.network.min=536346624

parseDynamicProperties

public static Configuration parseDynamicProperties(org.apache.commons.cli.CommandLine cmd)

Parse the dynamic properties (passed on the command line).

updateTmpDirectoriesInConfiguration

public static void updateTmpDirectoriesInConfiguration(Configuration configuration,
                                                       @Nullable
                                                       String defaultDirs)

Set temporary configuration directories if necessary.

Parameters:

configuration - flink config to patch

defaultDirs - in case no tmp directories is set, next directories will be applied

cloneConfiguration

public static Configuration cloneConfiguration(Configuration configuration)

Clones the given configuration and resets instance specific config options.

Parameters:

configuration - to clone

Returns:

Cloned configuration with reset instance specific config options

getDynamicPropertiesAsString

public static String getDynamicPropertiesAsString(Configuration baseConfig,
                                                  Configuration targetConfig)

Get dynamic properties based on two Flink configurations. If base config does not contain and target config contains the key or the value is different, it should be added to results. Otherwise, if the base config contains and target config does not contain the key, it will be ignored.

Parameters:

baseConfig - The base configuration.

targetConfig - The target configuration.

Returns:

Dynamic properties as string, separated by whitespace.

escapeForDifferentOS

public static String escapeForDifferentOS(String value)

Escape all the dynamic property values. For Unix-like OS(Linux, MacOS, FREE_BSD, etc.), each value will be surrounded with single quotes. This works for all chars except single quote itself. To escape the single quote, close the quoting before it, insert the escaped single quote, and then re-open the quoting. For example, the value is foo'bar and the escaped value is 'foo'\''bar'. See https://stackoverflow.com/questions/15783701/which-characters-need-to-be-escaped-when-using-bash for more information about Unix escaping.

For Windows OS, each value will be surrounded with double quotes. The double quote itself needs to be escaped with back slash. Also the caret symbol need to be escaped with double carets since Windows uses it to escape characters. See https://en.wikibooks.org/wiki/Windows_Batch_Scripting for more information about Windows escaping.

Parameters:

value - value to be escaped

Returns:

escaped value

escapeWithSingleQuote

public static String escapeWithSingleQuote(String value)

escapeWithDoubleQuote

public static String escapeWithDoubleQuote(String value)