org.apache.flink.runtime.yarn

Class AbstractFlinkYarnClient

java.lang.Object

org.apache.flink.runtime.yarn.AbstractFlinkYarnClient

Direct Known Subclasses:

FlinkYarnClientBase

public abstract class AbstractFlinkYarnClient
extends Object

Abstract interface for an implementation of a Flink on YARN client to deploy. The Client describes the properties of the YARN application to create.

Constructor Summary

Constructors

Constructor and Description
AbstractFlinkYarnClient()

Method Summary

Modifier and Type	Method and Description
abstract AbstractFlinkYarnCluster	deploy() Trigger the deployment to YARN.
abstract String	getClusterDescription() Returns a String containing details about the cluster (NodeManagers, available memory, ...)
abstract String	getDynamicPropertiesEncoded()
abstract Configuration	getFlinkConfiguration()
abstract org.apache.hadoop.fs.Path	getFlinkLoggingConfigurationPath()
abstract String	getSessionFilesDir()
abstract int	getTaskManagerCount()
abstract int	getTaskManagerSlots()
abstract boolean	isDetached()
abstract void	setConfigurationDirectory(String confDirPath)
abstract void	setConfigurationFilePath(org.apache.hadoop.fs.Path confPath)
abstract void	setDetachedMode(boolean detachedMode)
abstract void	setDynamicPropertiesEncoded(String dynamicPropertiesEncoded)
abstract void	setFlinkConfiguration(Configuration conf) Flink configuration
abstract void	setFlinkLoggingConfigurationPath(org.apache.hadoop.fs.Path logConfPath)
abstract void	setJobManagerMemory(int memoryMB)
abstract void	setLocalJarPath(org.apache.hadoop.fs.Path localJarPath)
abstract void	setName(String name) Set a name for the YARN application
abstract void	setQueue(String queue)
abstract void	setShipFiles(List<File> shipFiles) List of files to transfer to the YARN containers.
abstract void	setTaskManagerCount(int tmCount)
abstract void	setTaskManagerMemory(int memoryMB)
abstract void	setTaskManagerSlots(int slots)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AbstractFlinkYarnClient

public AbstractFlinkYarnClient()

Method Detail

setJobManagerMemory

public abstract void setJobManagerMemory(int memoryMB)

Parameters:

memoryMB - The amount of memory for the JobManager (in MB)

setTaskManagerMemory

public abstract void setTaskManagerMemory(int memoryMB)

Parameters:

memoryMB - The memory per TaskManager (in MB)

setFlinkConfiguration

public abstract void setFlinkConfiguration(Configuration conf)

Flink configuration

getFlinkConfiguration

public abstract Configuration getFlinkConfiguration()

setTaskManagerSlots

public abstract void setTaskManagerSlots(int slots)

Parameters:

slots - The number of TaskManager slots per TaskManager.

getTaskManagerSlots

public abstract int getTaskManagerSlots()

Returns:

the number of TaskManager processing slots per TaskManager.

setQueue

public abstract void setQueue(String queue)

Parameters:

queue - Name of the YARN queue

setLocalJarPath

public abstract void setLocalJarPath(org.apache.hadoop.fs.Path localJarPath)

Parameters:

localJarPath - Local Path to the Flink uberjar

setConfigurationFilePath

public abstract void setConfigurationFilePath(org.apache.hadoop.fs.Path confPath)

Parameters:

confPath - local path to the Flink configuration file

setFlinkLoggingConfigurationPath

public abstract void setFlinkLoggingConfigurationPath(org.apache.hadoop.fs.Path logConfPath)

Parameters:

logConfPath - local path to the flink logging configuration

getFlinkLoggingConfigurationPath

public abstract org.apache.hadoop.fs.Path getFlinkLoggingConfigurationPath()

setTaskManagerCount

public abstract void setTaskManagerCount(int tmCount)

Parameters:

tmCount - number of TaskManagers to start

getTaskManagerCount

public abstract int getTaskManagerCount()

setConfigurationDirectory

public abstract void setConfigurationDirectory(String confDirPath)

Parameters:

confDirPath - Path to config directory.

setShipFiles

public abstract void setShipFiles(List<File> shipFiles)

List of files to transfer to the YARN containers.

setDynamicPropertiesEncoded

public abstract void setDynamicPropertiesEncoded(String dynamicPropertiesEncoded)

Parameters:

dynamicPropertiesEncoded - Encoded String of the dynamic properties (-D configuration values of the Flink configuration)

getDynamicPropertiesEncoded

public abstract String getDynamicPropertiesEncoded()

getClusterDescription

public abstract String getClusterDescription()
                                      throws Exception

Returns a String containing details about the cluster (NodeManagers, available memory, ...)

Throws:

Exception

deploy

public abstract AbstractFlinkYarnCluster deploy()
                                         throws Exception

Trigger the deployment to YARN.

Throws:

Exception

setDetachedMode

public abstract void setDetachedMode(boolean detachedMode)

Parameters:

detachedMode - If true, the Flink YARN client is non-blocking. That means it returns once Flink has been started successfully on YARN.

isDetached

public abstract boolean isDetached()

getSessionFilesDir

public abstract String getSessionFilesDir()

Returns:

The string representation of the Path to the YARN session files. This is a temporary directory in HDFS that contains the jar files and configuration which is shipped to all the containers.

setName

public abstract void setName(String name)

Set a name for the YARN application

Parameters:

name -