Package org.apache.flink.runtime.state

Class DuplicatingCheckpointOutputStream

  • All Implemented Interfaces:
    Closeable, Flushable, AutoCloseable
    public class DuplicatingCheckpointOutputStream
extends CheckpointStateOutputStream
    A CheckpointStateOutputStream that wraps a primary and a secondary CheckpointStateOutputStream and duplicates all writes into both streams. This stream applies buffering to reduce the amount of dual-method calling. Furthermore, exceptions that happen in interactions with the secondary stream are not exposed, until the user calls closeAndGetSecondaryHandle(). In contrast to that, exceptions from interactions with the primary stream are immediately returned to the user. This class is used to write state for local recovery as a local (secondary) copy of the (primary) state snapshot that is written to a (slower but highly-available) remote filesystem.

    • Method Detail

      • getPos

        public long getPos()
            throws IOException
        Description copied from class: FSDataOutputStream
        Gets the position of the stream (non-negative), defined as the number of bytes from the beginning of the file to the current writing position. The position corresponds to the zero-based index of the next byte that will be written.

        This method must report accurately report the current position of the stream. Various components of the high-availability and recovery logic rely on the accurate

        Specified by:
        getPos in class FSDataOutputStream
        Returns:
        The current position in the stream, defined as the number of bytes from the beginning of the file to the current writing position.
        Throws:
        IOException - Thrown if an I/O error occurs while obtaining the position from the stream implementation.

      • flush

        public void flush()
           throws IOException
        Description copied from class: FSDataOutputStream
        Flushes the stream, writing any data currently buffered in stream implementation to the proper output stream. After this method has been called, the stream implementation must not hold onto any buffered data any more.

        A completed flush does not mean that the data is necessarily persistent. Data persistence can is only assumed after calls to FSDataOutputStream.close() or FSDataOutputStream.sync().

        Implementation note: This overrides the method defined in OutputStream as abstract to force implementations of the FSDataOutputStream to implement this method directly.

        Specified by:
        flush in interface Flushable
        Specified by:
        flush in class FSDataOutputStream
        Throws:
        IOException - Thrown if an I/O error occurs while flushing the stream.

      • sync

        public void sync()
          throws IOException
        Description copied from class: FSDataOutputStream
        Flushes the data all the way to the persistent non-volatile storage (for example disks). The method behaves similar to the fsync function, forcing all data to be persistent on the devices.
        Specified by:
        sync in class FSDataOutputStream
        Throws:
        IOException - Thrown if an I/O error occurs

      • closeAndGetHandle

        @Nullable
public StreamStateHandle closeAndGetHandle()
                                    throws IOException
        Description copied from class: CheckpointStateOutputStream
        Closes the stream and gets a state handle that can create an input stream producing the data written to this stream.

        This closing must be called (also when the caller is not interested in the handle) to successfully close the stream and retain the produced resource. In contrast, the CheckpointStateOutputStream.close() method removes the target resource when called.

        Specified by:
        closeAndGetHandle in class CheckpointStateOutputStream
        Returns:
        A state handle that can create an input stream producing the data written to this stream.
        Throws:
        IOException - Thrown, if the stream cannot be closed.

      • getSecondaryStreamException

        public Exception getSecondaryStreamException()