org.apache.flink.api.java.hadoop.mapred.utils

Class HadoopUtils

java.lang.Object

org.apache.flink.api.java.hadoop.mapred.utils.HadoopUtils

@Internal
public final class HadoopUtils
extends Object

Utility class to work with Apache Hadoop MapRed classes.

Method Summary

Modifier and Type	Method and Description
static org.apache.hadoop.conf.Configuration	getHadoopConfiguration(Configuration flinkConfiguration) Returns a new Hadoop Configuration object using the path to the hadoop conf configured in the main configuration (flink-conf.yaml).
static void	mergeHadoopConf(org.apache.hadoop.mapred.JobConf jobConf) Merge HadoopConfiguration into JobConf.
static String[]	possibleHadoopConfPaths(Configuration flinkConfiguration) Get possible Hadoop conf dir paths, based on environment variables and flink configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

mergeHadoopConf

public static void mergeHadoopConf(org.apache.hadoop.mapred.JobConf jobConf)

Merge HadoopConfiguration into JobConf. This is necessary for the HDFS configuration.

getHadoopConfiguration

public static org.apache.hadoop.conf.Configuration getHadoopConfiguration(Configuration flinkConfiguration)

Returns a new Hadoop Configuration object using the path to the hadoop conf configured in the main configuration (flink-conf.yaml). This method is public because its being used in the HadoopDataSource.

Parameters:

flinkConfiguration - Flink configuration object

Returns:

A Hadoop configuration instance

possibleHadoopConfPaths

public static String[] possibleHadoopConfPaths(Configuration flinkConfiguration)

Get possible Hadoop conf dir paths, based on environment variables and flink configuration.

Parameters:

flinkConfiguration - The flink configuration that may contain the path to Hadoop conf dir.

Returns:

an array of possible paths