Constructor and Description |
---|
HBaseRowInputFormat(Configuration conf,
String tableName,
HBaseTableSchema schema) |
HBaseTableSource(Configuration conf,
String tableName)
The HBase configuration and the name of the table to read.
|
Modifier and Type | Method and Description |
---|---|
static Configuration |
HadoopUtils.getHadoopConfiguration(Configuration flinkConfiguration)
Returns a new Hadoop Configuration object using the path to the hadoop conf configured
in the main configuration (flink-conf.yaml).
|
Modifier and Type | Field and Description |
---|---|
protected Configuration |
HadoopOutputFormatBase.configuration |
Modifier and Type | Method and Description |
---|---|
Configuration |
HadoopOutputFormatBase.getConfiguration() |
Configuration |
HadoopInputFormatBase.getConfiguration() |
Modifier and Type | Method and Description |
---|---|
static void |
HadoopUtils.mergeHadoopConf(Configuration hadoopConfig)
Merge HadoopConfiguration into Configuration.
|
Constructor and Description |
---|
SequenceFileWriterFactory(Configuration hadoopConf,
Class<K> keyClass,
Class<V> valueClass)
Creates a new SequenceFileWriterFactory using the given builder to assemble the
SequenceFileWriter.
|
SequenceFileWriterFactory(Configuration hadoopConf,
Class<K> keyClass,
Class<V> valueClass,
String compressionCodecName)
Creates a new SequenceFileWriterFactory using the given builder to assemble the
SequenceFileWriter.
|
SequenceFileWriterFactory(Configuration hadoopConf,
Class<K> keyClass,
Class<V> valueClass,
String compressionCodecName,
org.apache.hadoop.io.SequenceFile.CompressionType compressionType)
Creates a new SequenceFileWriterFactory using the given builder to assemble the
SequenceFileWriter.
|
Modifier and Type | Method and Description |
---|---|
Configuration |
HadoopConfigLoader.getOrLoadHadoopConfig()
get the loaded Hadoop config (or fall back to one loaded from the classpath).
|
Modifier and Type | Method and Description |
---|---|
protected abstract URI |
AbstractS3FileSystemFactory.getInitURI(URI fsUri,
Configuration hadoopConfig) |
Modifier and Type | Method and Description |
---|---|
protected URI |
S3FileSystemFactory.getInitURI(URI fsUri,
Configuration hadoopConfig) |
Constructor and Description |
---|
HadoopS3AccessHelper(org.apache.hadoop.fs.s3a.S3AFileSystem s3a,
Configuration conf) |
Modifier and Type | Method and Description |
---|---|
protected URI |
S3FileSystemFactory.getInitURI(URI fsUri,
Configuration hadoopConfig) |
Modifier and Type | Method and Description |
---|---|
Configuration |
HCatInputFormatBase.getConfiguration()
Returns the
Configuration of the HCatInputFormat. |
Constructor and Description |
---|
HCatInputFormatBase(String database,
String table,
Configuration config)
Creates a HCatInputFormat for the given database, table, and
Configuration . |
Constructor and Description |
---|
HCatInputFormat(String database,
String table,
Configuration config) |
Modifier and Type | Method and Description |
---|---|
OrcTableSource.Builder |
OrcTableSource.Builder.withConfiguration(Configuration config)
Sets a Hadoop
Configuration for the ORC reader. |
Constructor and Description |
---|
OrcRowInputFormat(String path,
String schemaString,
Configuration orcConfig)
Creates an OrcRowInputFormat.
|
OrcRowInputFormat(String path,
String schemaString,
Configuration orcConfig,
int batchSize)
Creates an OrcRowInputFormat.
|
OrcRowInputFormat(String path,
org.apache.orc.TypeDescription orcSchema,
Configuration orcConfig,
int batchSize)
Creates an OrcRowInputFormat.
|
Constructor and Description |
---|
HadoopModule(SecurityConfiguration securityConfiguration,
Configuration hadoopConfiguration) |
Modifier and Type | Method and Description |
---|---|
static Configuration |
HadoopUtils.getHadoopConfiguration(Configuration flinkConfiguration) |
Modifier and Type | Method and Description |
---|---|
RollingSink<T> |
RollingSink.setFSConfig(Configuration config)
Deprecated.
Specify a custom
Configuration that will be used when creating
the FileSystem for writing. |
Modifier and Type | Method and Description |
---|---|
BucketingSink<T> |
BucketingSink.setFSConfig(Configuration config)
Specify a custom
Configuration that will be used when creating
the FileSystem for writing. |
Modifier and Type | Method and Description |
---|---|
static void |
Utils.setTokensFor(org.apache.hadoop.yarn.api.records.ContainerLaunchContext amContainer,
List<org.apache.hadoop.fs.Path> paths,
Configuration conf) |
static void |
Utils.setupYarnClassPath(Configuration conf,
Map<String,String> appMasterEnv) |
Modifier and Type | Method and Description |
---|---|
static YarnHighAvailabilityServices |
YarnHighAvailabilityServices.forSingleJobAppMaster(Configuration flinkConfig,
Configuration hadoopConfig)
Creates the high-availability services for a single-job Flink YARN application, to be
used in the Application Master that runs both ResourceManager and JobManager.
|
static YarnHighAvailabilityServices |
YarnHighAvailabilityServices.forYarnTaskManager(Configuration flinkConfig,
Configuration hadoopConfig)
Creates the high-availability services for the TaskManagers participating in
a Flink YARN application.
|
Constructor and Description |
---|
AbstractYarnNonHaServices(Configuration config,
Configuration hadoopConf)
Creates new YARN high-availability services, configuring the file system and recovery
data directory based on the working directory in the given Hadoop configuration.
|
YarnHighAvailabilityServices(Configuration config,
Configuration hadoopConf)
Creates new YARN high-availability services, configuring the file system and recovery
data directory based on the working directory in the given Hadoop configuration.
|
YarnIntraNonHaMasterServices(Configuration config,
Configuration hadoopConf)
Creates new YarnIntraNonHaMasterServices for the given Flink and YARN configuration.
|
YarnPreConfiguredMasterNonHaServices(Configuration config,
Configuration hadoopConf,
HighAvailabilityServicesUtils.AddressResolution addressResolution)
Creates new YarnPreConfiguredMasterHaServices for the given Flink and YARN configuration.
|
Modifier and Type | Method and Description |
---|---|
void |
Configuration.addResource(Configuration conf)
Add a configuration resource.
|
static void |
Configuration.dumpConfiguration(Configuration config,
String propertyName,
Writer out)
Writes properties and their attributes (final and resource)
to the given
Writer . |
static void |
Configuration.dumpConfiguration(Configuration config,
Writer out)
Writes out all properties and their attributes (final and resource) to
the given
Writer , the format of the output would be, |
Constructor and Description |
---|
Configuration(Configuration other)
A new configuration with the same settings cloned from another.
|
Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.