@Internal public abstract class BaseHadoopFsRecoverableFsDataOutputStream extends CommitterFromPersistRecoverableFsDataOutputStream<HadoopFsRecoverable>
RecoverableFsDataOutputStream.Committer
Modifier and Type | Field and Description |
---|---|
protected org.apache.hadoop.fs.FileSystem |
fs |
protected long |
initialFileSize |
protected org.apache.hadoop.fs.FSDataOutputStream |
out |
protected org.apache.hadoop.fs.Path |
targetFile |
protected org.apache.hadoop.fs.Path |
tempFile |
Constructor and Description |
---|
BaseHadoopFsRecoverableFsDataOutputStream() |
Modifier and Type | Method and Description |
---|---|
void |
close()
Closes this stream.
|
HadoopFsRecoverable |
createHadoopFsRecoverable(long pos) |
void |
flush()
Flushes the stream, writing any data currently buffered in stream implementation to the
proper output stream.
|
long |
getPos()
Gets the position of the stream (non-negative), defined as the number of bytes from the
beginning of the file to the current writing position.
|
HadoopFsRecoverable |
persist()
Ensures all data so far is persistent (similar to
FSDataOutputStream.sync() ) and returns a handle to
recover the stream at the current position. |
void |
sync()
Flushes the data all the way to the persistent non-volatile storage (for example disks).
|
void |
write(byte[] b,
int off,
int len) |
void |
write(int b) |
closeForCommit, createCommitterFromResumeRecoverable
write
protected org.apache.hadoop.fs.FileSystem fs
protected org.apache.hadoop.fs.Path targetFile
protected org.apache.hadoop.fs.Path tempFile
protected org.apache.hadoop.fs.FSDataOutputStream out
protected long initialFileSize
public BaseHadoopFsRecoverableFsDataOutputStream()
public long getPos() throws IOException
FSDataOutputStream
This method must report accurately report the current position of the stream. Various components of the high-availability and recovery logic rely on the accurate
getPos
in class FSDataOutputStream
IOException
- Thrown if an I/O error occurs while obtaining the position from the
stream implementation.public void write(int b) throws IOException
write
in class OutputStream
IOException
public void write(byte[] b, int off, int len) throws IOException
write
in class OutputStream
IOException
public void flush() throws IOException
FSDataOutputStream
A completed flush does not mean that the data is necessarily persistent. Data persistence
can is only assumed after calls to FSDataOutputStream.close()
or FSDataOutputStream.sync()
.
Implementation note: This overrides the method defined in OutputStream
as abstract
to force implementations of the FSDataOutputStream
to implement this method directly.
flush
in interface Flushable
flush
in class FSDataOutputStream
IOException
- Thrown if an I/O error occurs while flushing the stream.public void sync() throws IOException
FSDataOutputStream
sync
in class FSDataOutputStream
IOException
- Thrown if an I/O error occurspublic HadoopFsRecoverable persist() throws IOException
RecoverableFsDataOutputStream
FSDataOutputStream.sync()
) and returns a handle to
recover the stream at the current position.persist
in class CommitterFromPersistRecoverableFsDataOutputStream<HadoopFsRecoverable>
IOException
RecoverableFsDataOutputStream.persist()
public HadoopFsRecoverable createHadoopFsRecoverable(long pos)
public void close() throws IOException
RecoverableFsDataOutputStream
In order to persist all previously written data, one needs to call the RecoverableFsDataOutputStream.closeForCommit()
method and call RecoverableFsDataOutputStream.Committer.commit()
on the returned committer
object.
close
in interface Closeable
close
in interface AutoCloseable
close
in class RecoverableFsDataOutputStream
IOException
- Thrown if an error occurred during closing.Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.