org.apache.flink.formats.hadoop.bulk

Interface HadoopFileCommitterFactory

All Superinterfaces:

Serializable

All Known Implementing Classes:

DefaultHadoopFileCommitterFactory

@Internal
public interface HadoopFileCommitterFactory
extends Serializable

The factory to create the HadoopFileCommitter.

Method Summary

Modifier and Type	Method and Description
HadoopFileCommitter	create(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path targetFilePath) Creates a new Hadoop file committer for writing.
HadoopFileCommitter	recoverForCommit(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path targetFilePath, org.apache.hadoop.fs.Path inProgressPath) Creates a Hadoop file committer for commit the pending file.

Method Detail

create

HadoopFileCommitter create(org.apache.hadoop.conf.Configuration configuration,
                           org.apache.hadoop.fs.Path targetFilePath)
                    throws IOException

Creates a new Hadoop file committer for writing.

Parameters:

configuration - The hadoop configuration.

targetFilePath - The target path to commit to.

Returns:

The corresponding Hadoop file committer.

Throws:

IOException

recoverForCommit

HadoopFileCommitter recoverForCommit(org.apache.hadoop.conf.Configuration configuration,
                                     org.apache.hadoop.fs.Path targetFilePath,
                                     org.apache.hadoop.fs.Path inProgressPath)
                              throws IOException

Creates a Hadoop file committer for commit the pending file.

Parameters:

configuration - The hadoop configuration.

targetFilePath - The target path to commit to.

inProgressPath - The path of the remaining pending file.

Returns:

The corresponding Hadoop file committer.

Throws:

IOException