@Public public abstract class FileOutputFormat<IT> extends RichOutputFormat<IT> implements InitializeOnMaster, CleanupWhenUnsuccessful
Modifier and Type | Class and Description |
---|---|
static class |
FileOutputFormat.OutputDirectoryMode
Behavior for creating output directories.
|
OutputFormat.InitializationContext
Modifier and Type | Field and Description |
---|---|
static String |
FILE_PARAMETER_KEY
The key under which the name of the target path is stored in the configuration.
|
protected Path |
outputFilePath
The path of the file to be written.
|
protected FSDataOutputStream |
stream
The stream to which the data is written;
|
Constructor and Description |
---|
FileOutputFormat() |
FileOutputFormat(Path outputPath) |
Modifier and Type | Method and Description |
---|---|
void |
close()
Method that marks the end of the life-cycle of parallel output instance.
|
void |
configure(Configuration parameters)
Configures this output format.
|
protected String |
getDirectoryFileName(int taskNumber) |
FileOutputFormat.OutputDirectoryMode |
getOutputDirectoryMode() |
Path |
getOutputFilePath() |
FileSystem.WriteMode |
getWriteMode() |
static void |
initDefaultsFromConfiguration(Configuration configuration)
Initialize defaults for output format.
|
void |
initializeGlobal(int parallelism)
Initialization of the distributed file system if it is used.
|
void |
open(int taskNumber,
int numTasks)
Opens a parallel instance of the output format to store the result of its parallel instance.
|
void |
setOutputDirectoryMode(FileOutputFormat.OutputDirectoryMode mode) |
void |
setOutputFilePath(Path path) |
void |
setWriteMode(FileSystem.WriteMode mode) |
void |
tryCleanupOnError()
Hook that is called upon an unsuccessful execution.
|
getRuntimeContext, setRuntimeContext
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
open, writeRecord
public static final String FILE_PARAMETER_KEY
protected Path outputFilePath
protected transient FSDataOutputStream stream
public FileOutputFormat()
public FileOutputFormat(Path outputPath)
public static void initDefaultsFromConfiguration(Configuration configuration)
configuration
- The configuration to load defaults frompublic void setOutputFilePath(Path path)
public Path getOutputFilePath()
public void setWriteMode(FileSystem.WriteMode mode)
public FileSystem.WriteMode getWriteMode()
public void setOutputDirectoryMode(FileOutputFormat.OutputDirectoryMode mode)
public FileOutputFormat.OutputDirectoryMode getOutputDirectoryMode()
public void configure(Configuration parameters)
OutputFormat
This method is always called first on a newly instantiated output format.
configure
in interface OutputFormat<IT>
parameters
- The configuration with all parameters.public void open(int taskNumber, int numTasks) throws IOException
OutputFormat
When this method is called, the output format it guaranteed to be configured.
open
in interface OutputFormat<IT>
taskNumber
- The number of the parallel instance.numTasks
- The number of parallel tasks.IOException
- Thrown, if the output could not be opened due to an I/O problem.protected String getDirectoryFileName(int taskNumber)
public void close() throws IOException
OutputFormat
When this method is called, the output format it guaranteed to be opened.
close
in interface OutputFormat<IT>
IOException
- Thrown, if the input could not be closed properly.public void initializeGlobal(int parallelism) throws IOException
initializeGlobal
in interface InitializeOnMaster
parallelism
- The task parallelism.IOException
- The initialization may throw exceptions, which may cause the job to
abort.public void tryCleanupOnError()
CleanupWhenUnsuccessful
tryCleanupOnError
in interface CleanupWhenUnsuccessful
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.