org.apache.flink.api.common.io

Interface CheckpointableInputFormat<S extends InputSplit,T extends Serializable>

Type Parameters:

S - The type of input split.

T - The type of the channel state to be checkpointed / included in the snapshot.

All Known Implementing Classes:

AvroInputFormat, BinaryInputFormat, CsvInputFormat, DelimitedInputFormat, GenericCsvInputFormat, PojoCsvInputFormat, PrimitiveInputFormat, RowCsvInputFormat, SerializedInputFormat, TextInputFormat, TextValueInputFormat, TupleCsvInputFormat, TypeSerializerInputFormat

@PublicEvolving
public interface CheckpointableInputFormat<S extends InputSplit,T extends Serializable>

An interface that describes InputFormats that allow checkpointing/restoring their state.

Method Summary

Modifier and Type	Method and Description
T	getCurrentState() Returns the split currently being read, along with its current state.
void	reopen(S split, T state) Restores the state of a parallel instance reading from an InputFormat.

Method Detail

getCurrentState

T getCurrentState()
           throws IOException

Returns the split currently being read, along with its current state. This will be used to restore the state of the reading channel when recovering from a task failure. In the case of a simple text file, the state can correspond to the last read offset in the split.

Returns:

The state of the channel.

Throws:

IOException - Thrown if the creation of the state object failed.

reopen

void reopen(S split,
            T state)
     throws IOException

Restores the state of a parallel instance reading from an InputFormat. This is necessary when recovering from a task failure. When this method is called, the input format it guaranteed to be configured.

NOTE: The caller has to make sure that the provided split is the one to whom the state belongs.

Parameters:

split - The split to be opened.

state - The state from which to start from. This can contain the offset, but also other data, depending on the input format.

Throws:

IOException