org.apache.flink.api.java.io

Class CsvOutputFormat<T extends Tuple>

java.lang.Object

org.apache.flink.api.common.io.RichOutputFormat<IT>

org.apache.flink.api.common.io.FileOutputFormat<T>

org.apache.flink.api.java.io.CsvOutputFormat<T>

All Implemented Interfaces:

Serializable, CleanupWhenUnsuccessful, InitializeOnMaster, OutputFormat<T>, InputTypeConfigurable

@PublicEvolving
public class CsvOutputFormat<T extends Tuple>
extends FileOutputFormat<T>
implements InputTypeConfigurable

This is an OutputFormat to serialize Tuples to text. The output is structured by record delimiters and field delimiters as common in CSV files. Record delimiter separate records from each other ('\n' is common). Field delimiters separate fields within a record.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileOutputFormat

FileOutputFormat.OutputDirectoryMode

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_FIELD_DELIMITER
static String	DEFAULT_LINE_DELIMITER

Fields inherited from class org.apache.flink.api.common.io.FileOutputFormat

FILE_PARAMETER_KEY, outputFilePath, stream

Constructor Summary

Constructors

Constructor and Description
CsvOutputFormat(Path outputPath) Creates an instance of CsvOutputFormat.
CsvOutputFormat(Path outputPath, String fieldDelimiter) Creates an instance of CsvOutputFormat.
CsvOutputFormat(Path outputPath, String recordDelimiter, String fieldDelimiter) Creates an instance of CsvOutputFormat.

Method Summary

Modifier and Type	Method and Description
void	close() Method that marks the end of the life-cycle of parallel output instance.
void	open(int taskNumber, int numTasks) Opens a parallel instance of the output format to store the result of its parallel instance.
void	setAllowNullValues(boolean allowNulls) Configures the format to either allow null values (writing an empty field), or to throw an exception when encountering a null field.
void	setCharsetName(String charsetName) Sets the charset with which the CSV strings are written to the file.
void	setInputType(TypeInformation<?> type, ExecutionConfig executionConfig) The purpose of this method is solely to check whether the data type to be processed is in fact a tuple type.
void	setQuoteStrings(boolean quoteStrings) Configures whether the output format should quote string values.
String	toString()
void	writeRecord(T element) Adds a record to the output.

Methods inherited from class org.apache.flink.api.common.io.FileOutputFormat

configure, getDirectoryFileName, getOutputDirectoryMode, getOutputFilePath, getWriteMode, initDefaultsFromConfiguration, initializeGlobal, setOutputDirectoryMode, setOutputFilePath, setWriteMode, tryCleanupOnError

Methods inherited from class org.apache.flink.api.common.io.RichOutputFormat

getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_LINE_DELIMITER

public static final String DEFAULT_LINE_DELIMITER

See Also:

Constant Field Values

DEFAULT_FIELD_DELIMITER

public static final String DEFAULT_FIELD_DELIMITER

Constructor Detail

CsvOutputFormat

public CsvOutputFormat(Path outputPath)

Creates an instance of CsvOutputFormat. Lines are separated by the newline character '\n', fields are separated by ','.

Parameters:

outputPath - The path where the CSV file is written.

CsvOutputFormat

public CsvOutputFormat(Path outputPath,
                       String fieldDelimiter)

Creates an instance of CsvOutputFormat. Lines are separated by the newline character '\n', fields by the given field delimiter.

Parameters:

outputPath - The path where the CSV file is written.

fieldDelimiter - The delimiter that is used to separate fields in a tuple.

CsvOutputFormat

public CsvOutputFormat(Path outputPath,
                       String recordDelimiter,
                       String fieldDelimiter)

Creates an instance of CsvOutputFormat.

Parameters:

outputPath - The path where the CSV file is written.

recordDelimiter - The delimiter that is used to separate the tuples.

fieldDelimiter - The delimiter that is used to separate fields in a tuple.

Method Detail

setAllowNullValues

public void setAllowNullValues(boolean allowNulls)

Configures the format to either allow null values (writing an empty field), or to throw an exception when encountering a null field.

by default, null values are disallowed.

Parameters:

allowNulls - Flag to indicate whether the output format should accept null values.

setCharsetName

public void setCharsetName(String charsetName)

Sets the charset with which the CSV strings are written to the file. If not specified, the output format uses the systems default character encoding.

Parameters:

charsetName - The name of charset to use for encoding the output.

setQuoteStrings

public void setQuoteStrings(boolean quoteStrings)

Configures whether the output format should quote string values. String values are fields of type String and StringValue, as well as all subclasses of the latter.

By default, strings are not quoted.

Parameters:

quoteStrings - Flag indicating whether string fields should be quoted.

open

public void open(int taskNumber,
                 int numTasks)
          throws IOException

Description copied from interface: OutputFormat

Opens a parallel instance of the output format to store the result of its parallel instance.

When this method is called, the output format it guaranteed to be configured.

Specified by:

open in interface OutputFormat<T extends Tuple>

Overrides:

open in class FileOutputFormat<T extends Tuple>

Parameters:

taskNumber - The number of the parallel instance.

numTasks - The number of parallel tasks.

Throws:

IOException - Thrown, if the output could not be opened due to an I/O problem.

close

public void close()
           throws IOException

Description copied from interface: OutputFormat

Method that marks the end of the life-cycle of parallel output instance. Should be used to close channels and streams and release resources. After this method returns without an error, the output is assumed to be correct.

When this method is called, the output format it guaranteed to be opened.

Specified by:

close in interface OutputFormat<T extends Tuple>

Overrides:

close in class FileOutputFormat<T extends Tuple>

Throws:

IOException - Thrown, if the input could not be closed properly.

writeRecord

public void writeRecord(T element)
                 throws IOException

Description copied from interface: OutputFormat

Adds a record to the output.

When this method is called, the output format it guaranteed to be opened.

Specified by:

writeRecord in interface OutputFormat<T extends Tuple>

Parameters:

element - The records to add to the output.

Throws:

IOException - Thrown, if the records could not be added to to an I/O problem.

toString

public String toString()

Overrides:

toString in class Object

setInputType

public void setInputType(TypeInformation<?> type,
                         ExecutionConfig executionConfig)

The purpose of this method is solely to check whether the data type to be processed is in fact a tuple type.

Specified by:

setInputType in interface InputTypeConfigurable

Parameters:

type - The data type of the input.

executionConfig - The execution config for this parallel execution.