Class RecordAndPosition<E>

  • Direct Known Subclasses:
    MutableRecordAndPosition

    @PublicEvolving
    public class RecordAndPosition<E>
    extends Object
    A record, together with the reader position to be stored in the checkpoint.

    The position defines the point in the reader AFTER the record. Record processing and updating checkpointed state happens atomically. The position points to where the reader should resume after this record is processed.

    For example, the very first record in a file split could have an offset of zero and a recordSkipCount of one.

    This class is immutable for safety. Use MutableRecordAndPosition if you need a mutable version for efficiency.

    Note on this design: Why do we not make the position point to the current record and always skip one record after recovery (the just processed record)? We need to be able to support formats where skipping records (even one) is not an option. For example formats that execute (pushed down) filters may want to avoid a skip-record-count all together, so that they don't skip the wrong records when the filter gets updated around a checkpoint/savepoint.

    • Field Detail

      • NO_OFFSET

        public static final long NO_OFFSET
        Constant for the offset, reflecting that the position does not contain any offset information. It is used in positions that are defined only by a number of records to skip.
        See Also:
        Constant Field Values
    • Constructor Detail

      • RecordAndPosition

        public RecordAndPosition​(E record,
                                 long offset,
                                 long recordSkipCount)
        Creates a new RecordAndPosition with the given record and position info.
    • Method Detail

      • getRecord

        public E getRecord()
      • getOffset

        public long getOffset()
      • getRecordSkipCount

        public long getRecordSkipCount()