org.apache.flink.streaming.api.functions.source

Class TimestampedFileInputSplit

java.lang.Object

org.apache.flink.core.io.LocatableInputSplit

org.apache.flink.core.fs.FileInputSplit

org.apache.flink.streaming.api.functions.source.TimestampedFileInputSplit

All Implemented Interfaces:

Serializable, Comparable<TimestampedFileInputSplit>, InputSplit

public class TimestampedFileInputSplit
extends FileInputSplit
implements Comparable<TimestampedFileInputSplit>

An extended FileInputSplit that also includes information about:

• The modification time of the file this split belongs to.
• When checkpointing, the state of the split at the moment of the checkpoint.

This class is used by the ContinuousFileMonitoringFunction and the ContinuousFileReaderOperator to perform continuous file processing.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
TimestampedFileInputSplit(long modificationTime, int num, Path file, long start, long length, String[] hosts) Creates a TimestampedFileInputSplit based on the file modification time and the rest of the information of the FileInputSplit, as returned by the underlying filesystem.

Method Summary

Modifier and Type	Method and Description
int	compareTo(TimestampedFileInputSplit o)
boolean	equals(Object o)
long	getModificationTime()
Serializable	getSplitState()
int	hashCode()
void	resetSplitState() Sets the state of the split to null.
void	setSplitState(Serializable state) Sets the state of the split.
String	toString()

Methods inherited from class org.apache.flink.core.fs.FileInputSplit

getLength, getPath, getStart

Methods inherited from class org.apache.flink.core.io.LocatableInputSplit

getHostnames, getSplitNumber

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

TimestampedFileInputSplit

public TimestampedFileInputSplit(long modificationTime,
                                 int num,
                                 Path file,
                                 long start,
                                 long length,
                                 String[] hosts)

Creates a TimestampedFileInputSplit based on the file modification time and the rest of the information of the FileInputSplit, as returned by the underlying filesystem.

Parameters:

modificationTime - the modification file of the file this split belongs to

num - the number of this input split

file - the file name

start - the position of the first byte in the file to process

length - the number of bytes in the file to process (-1 is flag for "read whole file")

hosts - the list of hosts containing the block, possibly null

Method Detail

setSplitState

public void setSplitState(Serializable state)

Sets the state of the split. This information is used when restoring from a checkpoint and allows to resume reading the underlying file from the point we left off.

* This is applicable to FileInputFormats that implement the CheckpointableInputFormat interface.

resetSplitState

public void resetSplitState()

Sets the state of the split to null.

getSplitState

public Serializable getSplitState()

Returns:

the state of the split.

getModificationTime

public long getModificationTime()

Returns:

The modification time of the file this split belongs to.

compareTo

public int compareTo(TimestampedFileInputSplit o)

Specified by:

compareTo in interface Comparable<TimestampedFileInputSplit>

equals

public boolean equals(Object o)

Overrides:

equals in class FileInputSplit

hashCode

public int hashCode()

Overrides:

hashCode in class FileInputSplit

toString

public String toString()

Overrides:

toString in class FileInputSplit