Interface CheckpointableInputFormat<S extends InputSplit,​T extends Serializable>

    • Method Detail

      • getCurrentState

        T getCurrentState()
                   throws IOException
        Returns the split currently being read, along with its current state. This will be used to restore the state of the reading channel when recovering from a task failure. In the case of a simple text file, the state can correspond to the last read offset in the split.
        Returns:
        The state of the channel.
        Throws:
        IOException - Thrown if the creation of the state object failed.
      • reopen

        void reopen​(S split,
                    T state)
             throws IOException
        Restores the state of a parallel instance reading from an InputFormat. This is necessary when recovering from a task failure. When this method is called, the input format it guaranteed to be configured.

        NOTE: The caller has to make sure that the provided split is the one to whom the state belongs.

        Parameters:
        split - The split to be opened.
        state - The state from which to start from. This can contain the offset, but also other data, depending on the input format.
        Throws:
        IOException