Interface BulkWriter<T>
-
- Type Parameters:
T
- The type of the elements encoded through this encoder.
- All Known Subinterfaces:
HadoopPathBasedBulkWriter<T>
- All Known Implementing Classes:
AvroBulkWriter
,HadoopCompressionBulkWriter
,NoCompressionBulkWriter
,OrcBulkWriter
,ParquetBulkWriter
,SequenceFileWriter
@PublicEvolving public interface BulkWriter<T>
An encoder that encodes data in a bulk fashion, encoding many records together at a time.Examples for bulk encoding are most compressed formats, including formats like Parquet and ORC which encode batches of records into blocks of column vectors.
The bulk encoder may be stateful and is bound to a single stream during its lifetime.
-
-
Nested Class Summary
Nested Classes Modifier and Type Interface Description static interface
BulkWriter.Factory<T>
A factory that creates aBulkWriter
.
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description void
addElement(T element)
Adds an element to the encoder.void
finish()
Finishes the writing.void
flush()
Flushes all intermediate buffered data to the output stream.
-
-
-
Method Detail
-
addElement
void addElement(T element) throws IOException
Adds an element to the encoder. The encoder may temporarily buffer the element, or immediately write it to the stream.It may be that adding this element fills up an internal buffer and causes the encoding and flushing of a batch of internally buffered elements.
- Parameters:
element
- The element to add.- Throws:
IOException
- Thrown, if the element cannot be added to the encoder, or if the output stream throws an exception.
-
flush
void flush() throws IOException
Flushes all intermediate buffered data to the output stream. It is expected that flushing often may reduce the efficiency of the encoding.- Throws:
IOException
- Thrown if the encoder cannot be flushed, or if the output stream throws an exception.
-
finish
void finish() throws IOException
Finishes the writing. This must flush all internal buffer, finish encoding, and write footers.The writer is not expected to handle any more records via
addElement(Object)
after this method is called.Important: This method MUST NOT close the stream that the writer writes to. Closing the stream is expected to happen through the invoker of this method afterwards.
- Throws:
IOException
- Thrown if the finalization fails.
-
-