IN
- The type of input elements.BucketID
- The type of ids for the buckets, as returned by the BucketAssigner
.@Internal public class BulkBucketWriter<IN,BucketID> extends Object
BulkPartWriters
.BucketWriter.PendingFile
Constructor and Description |
---|
BulkBucketWriter(RecoverableWriter recoverableWriter,
BulkWriter.Factory<IN> writerFactory) |
public BulkBucketWriter(RecoverableWriter recoverableWriter, BulkWriter.Factory<IN> writerFactory) throws IOException
IOException
public InProgressFileWriter<IN,BucketID> resumeFrom(BucketID bucketId, RecoverableFsDataOutputStream stream, Path path, RecoverableWriter.ResumeRecoverable resumable, long creationTime) throws IOException
IOException
public InProgressFileWriter<IN,BucketID> openNew(BucketID bucketId, RecoverableFsDataOutputStream stream, Path path, long creationTime) throws IOException
IOException
public InProgressFileWriter<IN,BucketID> openNewInProgressFile(BucketID bucketID, Path path, long creationTime) throws IOException
BucketWriter
InProgressFileWriter
.openNewInProgressFile
in interface BucketWriter<IN,BucketID>
bucketID
- the id of the bucket this writer is writing to.path
- the path this writer will write to.creationTime
- the creation time of the file.InProgressFileWriter
IOException
- Thrown if creating a writer fails.public CompactingFileWriter openNewCompactingFile(CompactingFileWriter.Type type, BucketID bucketID, Path path, long creationTime) throws IOException
BucketWriter
CompactingFileWriter
of the requesting type. Requesting a writer
of an unsupported type will result in UnsupportedOperationException. By default, only
RECORD_WISE type is supported, for which a InProgressFileWriter
will be created.openNewCompactingFile
in interface BucketWriter<IN,BucketID>
type
- the type of this writer.bucketID
- the id of the bucket this writer is writing to.path
- the path this writer will write to.creationTime
- the creation time of the file.InProgressFileWriter
IOException
- Thrown if creating a writer fails.public InProgressFileWriter<IN,BucketID> resumeInProgressFileFrom(BucketID bucketID, InProgressFileWriter.InProgressFileRecoverable inProgressFileRecoverable, long creationTime) throws IOException
BucketWriter
InProgressFileWriter
from a InProgressFileWriter.InProgressFileRecoverable
.resumeInProgressFileFrom
in interface BucketWriter<IN,BucketID>
bucketID
- the id of the bucket this writer is writing to.inProgressFileRecoverable
- the state of the part file.creationTime
- the creation time of the file.InProgressFileWriter
IOException
- Thrown if resuming a writer fails.public BucketWriter.PendingFile recoverPendingFile(InProgressFileWriter.PendingFileRecoverable pendingFileRecoverable) throws IOException
BucketWriter
recoverPendingFile
in interface BucketWriter<IN,BucketID>
pendingFileRecoverable
- The handle with the recovery information.IOException
- Thrown if recovering a pending file fails.public boolean cleanupInProgressFileRecoverable(InProgressFileWriter.InProgressFileRecoverable inProgressFileRecoverable) throws IOException
BucketWriter
NOTE: This operation should not throw an exception, but return false if the cleanup did not happen for any reason.
cleanupInProgressFileRecoverable
in interface BucketWriter<IN,BucketID>
inProgressFileRecoverable
- the InProgressFileWriter.InProgressFileRecoverable
whose state we want to clean-up.true
if the resources were successfully freed, false
otherwise (e.g.
the file to be deleted was not there for any reason - already deleted or never created).IOException
- if an I/O error occurspublic WriterProperties getProperties()
getProperties
in interface BucketWriter<IN,BucketID>
BucketWriter
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.