Class RecordAndPosition<E>
- java.lang.Object
-
- org.apache.flink.connector.file.src.util.RecordAndPosition<E>
-
- Direct Known Subclasses:
MutableRecordAndPosition
@PublicEvolving public class RecordAndPosition<E> extends Object
A record, together with the reader position to be stored in the checkpoint.The position defines the point in the reader AFTER the record. Record processing and updating checkpointed state happens atomically. The position points to where the reader should resume after this record is processed.
For example, the very first record in a file split could have an
offset
of zero and arecordSkipCount
of one.This class is immutable for safety. Use
MutableRecordAndPosition
if you need a mutable version for efficiency.Note on this design: Why do we not make the position point to the current record and always skip one record after recovery (the just processed record)? We need to be able to support formats where skipping records (even one) is not an option. For example formats that execute (pushed down) filters may want to avoid a skip-record-count all together, so that they don't skip the wrong records when the filter gets updated around a checkpoint/savepoint.
-
-
Field Summary
Fields Modifier and Type Field Description static long
NO_OFFSET
Constant for the offset, reflecting that the position does not contain any offset information.
-
Constructor Summary
Constructors Constructor Description RecordAndPosition(E record, long offset, long recordSkipCount)
Creates a newRecordAndPosition
with the given record and position info.
-
-
-
Field Detail
-
NO_OFFSET
public static final long NO_OFFSET
Constant for the offset, reflecting that the position does not contain any offset information. It is used in positions that are defined only by a number of records to skip.- See Also:
- Constant Field Values
-
-
Constructor Detail
-
RecordAndPosition
public RecordAndPosition(E record, long offset, long recordSkipCount)
Creates a newRecordAndPosition
with the given record and position info.
-
-