org.apache.flink.api.connector.source

Interface Source<T,SplitT extends SourceSplit,EnumChkT>

Type Parameters:

T - The type of records produced by the source.

SplitT - The type of splits handled by the source.

EnumChkT - The type of the enumerator checkpoints.

All Superinterfaces:

Serializable, SourceReaderFactory<T,SplitT>

All Known Implementing Classes:

AbstractFileSource, DataGeneratorSource, FileSource, FromElementsSource, HiveSource, HybridSource, KafkaSource, NumberSequenceSource

@Public
public interface Source<T,SplitT extends SourceSplit,EnumChkT>
extends SourceReaderFactory<T,SplitT>

The interface for Source. It acts like a factory class that helps construct the SplitEnumerator and SourceReader and corresponding serializers.

Method Summary

Modifier and Type	Method and Description
SplitEnumerator<SplitT,EnumChkT>	createEnumerator(SplitEnumeratorContext<SplitT> enumContext) Creates a new SplitEnumerator for this source, starting a new input.
Boundedness	getBoundedness() Get the boundedness of this source.
SimpleVersionedSerializer<EnumChkT>	getEnumeratorCheckpointSerializer() Creates the serializer for the SplitEnumerator checkpoint.
SimpleVersionedSerializer<SplitT>	getSplitSerializer() Creates a serializer for the source splits.
SplitEnumerator<SplitT,EnumChkT>	restoreEnumerator(SplitEnumeratorContext<SplitT> enumContext, EnumChkT checkpoint) Restores an enumerator from a checkpoint.

Methods inherited from interface org.apache.flink.api.connector.source.SourceReaderFactory

createReader

Method Detail

getBoundedness

Boundedness getBoundedness()

Get the boundedness of this source.

Returns:

the boundedness of this source.

createEnumerator

SplitEnumerator<SplitT,EnumChkT> createEnumerator(SplitEnumeratorContext<SplitT> enumContext)
                                           throws Exception

Creates a new SplitEnumerator for this source, starting a new input.

Parameters:

enumContext - The context for the split enumerator.

Returns:

A new SplitEnumerator.

Throws:

Exception - The implementor is free to forward all exceptions directly. Exceptions thrown from this method cause JobManager failure/recovery.

restoreEnumerator

SplitEnumerator<SplitT,EnumChkT> restoreEnumerator(SplitEnumeratorContext<SplitT> enumContext,
                                                   EnumChkT checkpoint)
                                            throws Exception

Restores an enumerator from a checkpoint.

Parameters:

enumContext - The context for the restored split enumerator.

checkpoint - The checkpoint to restore the SplitEnumerator from.

Returns:

A SplitEnumerator restored from the given checkpoint.

Throws:

Exception - The implementor is free to forward all exceptions directly. Exceptions thrown from this method cause JobManager failure/recovery.

getSplitSerializer

SimpleVersionedSerializer<SplitT> getSplitSerializer()

Creates a serializer for the source splits. Splits are serialized when sending them from enumerator to reader, and when checkpointing the reader's current state.

Returns:

The serializer for the split type.

getEnumeratorCheckpointSerializer

SimpleVersionedSerializer<EnumChkT> getEnumeratorCheckpointSerializer()

Creates the serializer for the SplitEnumerator checkpoint. The serializer is used for the result of the SplitEnumerator.snapshotState(long) method.

Returns:

The serializer for the SplitEnumerator checkpoint.