org.apache.flink.formats.avro

Class AvroOutputFormat<E>

java.lang.Object

org.apache.flink.api.common.io.RichOutputFormat<IT>

org.apache.flink.api.common.io.FileOutputFormat<E>

org.apache.flink.formats.avro.AvroOutputFormat<E>

Type Parameters:

E -

All Implemented Interfaces:

Serializable, CleanupWhenUnsuccessful, InitializeOnMaster, OutputFormat<E>

public class AvroOutputFormat<E>
extends FileOutputFormat<E>
implements Serializable

FileOutputFormat for Avro records.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AvroOutputFormat.Codec Wrapper which encapsulates the supported codec and a related serialization byte.

Nested classes/interfaces inherited from class org.apache.flink.api.common.io.FileOutputFormat

FileOutputFormat.OutputDirectoryMode

Field Summary

Fields inherited from class org.apache.flink.api.common.io.FileOutputFormat

FILE_PARAMETER_KEY, outputFilePath, stream

Constructor Summary

Constructors

Constructor and Description
AvroOutputFormat(Class<E> type)
AvroOutputFormat(Path filePath, Class<E> type)

Method Summary

Modifier and Type	Method and Description
void	close() Method that marks the end of the life-cycle of parallel output instance.
protected String	getDirectoryFileName(int taskNumber)
void	open(int taskNumber, int numTasks) Opens a parallel instance of the output format to store the result of its parallel instance.
void	setCodec(AvroOutputFormat.Codec codec) Set avro codec for compression.
void	setSchema(org.apache.avro.Schema schema)
void	writeRecord(E record) Adds a record to the output.

Methods inherited from class org.apache.flink.api.common.io.FileOutputFormat

configure, getOutputDirectoryMode, getOutputFilePath, getWriteMode, initDefaultsFromConfiguration, initializeGlobal, setOutputDirectoryMode, setOutputFilePath, setWriteMode, tryCleanupOnError

Methods inherited from class org.apache.flink.api.common.io.RichOutputFormat

getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroOutputFormat

public AvroOutputFormat(Path filePath,
                        Class<E> type)

AvroOutputFormat

public AvroOutputFormat(Class<E> type)

Method Detail

getDirectoryFileName

protected String getDirectoryFileName(int taskNumber)

Overrides:

getDirectoryFileName in class FileOutputFormat<E>

setSchema

public void setSchema(org.apache.avro.Schema schema)

setCodec

public void setCodec(AvroOutputFormat.Codec codec)

Set avro codec for compression.

Parameters:

codec - avro codec.

writeRecord

public void writeRecord(E record)
                 throws IOException

Description copied from interface: OutputFormat

Adds a record to the output.

When this method is called, the output format it guaranteed to be opened.

Specified by:

writeRecord in interface OutputFormat<E>

Parameters:

record - The records to add to the output.

Throws:

IOException - Thrown, if the records could not be added to to an I/O problem.

open

public void open(int taskNumber,
                 int numTasks)
          throws IOException

Description copied from interface: OutputFormat

Opens a parallel instance of the output format to store the result of its parallel instance.

When this method is called, the output format it guaranteed to be configured.

Specified by:

open in interface OutputFormat<E>

Overrides:

open in class FileOutputFormat<E>

Parameters:

taskNumber - The number of the parallel instance.

numTasks - The number of parallel tasks.

Throws:

IOException - Thrown, if the output could not be opened due to an I/O problem.

close

public void close()
           throws IOException

Description copied from interface: OutputFormat

Method that marks the end of the life-cycle of parallel output instance. Should be used to close channels and streams and release resources. After this method returns without an error, the output is assumed to be correct.

When this method is called, the output format it guaranteed to be opened.

Specified by:

close in interface OutputFormat<E>

Overrides:

close in class FileOutputFormat<E>

Throws:

IOException - Thrown, if the input could not be closed properly.