org.apache.flink.formats.compress

Class CompressWriterFactory<IN>

java.lang.Object

org.apache.flink.formats.compress.CompressWriterFactory<IN>

Type Parameters:

IN - The type of element to write.

All Implemented Interfaces:

Serializable, BulkWriter.Factory<IN>

@PublicEvolving
public class CompressWriterFactory<IN>
extends Object
implements BulkWriter.Factory<IN>

A factory that creates for bulk writers that, when provided with a CompressionCodec, they compress the data they write. If no codec is provided, the data is written in bulk but uncompressed.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
CompressWriterFactory(Extractor<IN> extractor) Creates a new CompressWriterFactory using the given Extractor to assemble either HadoopCompressionBulkWriter or NoCompressionBulkWriter based on whether a Hadoop CompressionCodec name is specified.

Method Summary

Modifier and Type	Method and Description
BulkWriter<IN>	create(FSDataOutputStream out) Creates a writer that writes to the given stream.
String	getExtension()
CompressWriterFactory<IN>	withHadoopCompression(String codecName) Compresses the data using the provided Hadoop CompressionCodec.
CompressWriterFactory<IN>	withHadoopCompression(String codecName, Configuration hadoopConfig) Compresses the data using the provided Hadoop CompressionCodec and Configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CompressWriterFactory

public CompressWriterFactory(Extractor<IN> extractor)

Creates a new CompressWriterFactory using the given Extractor to assemble either HadoopCompressionBulkWriter or NoCompressionBulkWriter based on whether a Hadoop CompressionCodec name is specified.

Parameters:

extractor - Extractor to extract the element

Method Detail

withHadoopCompression

public CompressWriterFactory<IN> withHadoopCompression(String codecName)
                                                throws IOException

Compresses the data using the provided Hadoop CompressionCodec.

Parameters:

codecName - Simple/complete name or alias of the CompressionCodec

Returns:

the instance of CompressionWriterFactory

Throws:

IOException

withHadoopCompression

public CompressWriterFactory<IN> withHadoopCompression(String codecName,
                                                       Configuration hadoopConfig)
                                                throws IOException

Compresses the data using the provided Hadoop CompressionCodec and Configuration.

Parameters:

codecName - Simple/complete name or alias of the CompressionCodec

hadoopConfig - Hadoop Configuration

Returns:

the instance of CompressionWriterFactory

Throws:

IOException

create

public BulkWriter<IN> create(FSDataOutputStream out)
                      throws IOException

Description copied from interface: BulkWriter.Factory

Creates a writer that writes to the given stream.

Specified by:

create in interface BulkWriter.Factory<IN>

Parameters:

out - The output stream to write the encoded data to.

Throws:

IOException - Thrown if the writer cannot be opened, or if the output stream throws an exception.

getExtension

public String getExtension()