T
- The type of records produced by the source.SplitT
- The type of splits handled by the source.EnumChkT
- The type of the enumerator checkpoints.@PublicEvolving public interface Source<T,SplitT extends SourceSplit,EnumChkT> extends Serializable
SplitEnumerator
and SourceReader
and corresponding serializers.Modifier and Type | Method and Description |
---|---|
SplitEnumerator<SplitT,EnumChkT> |
createEnumerator(SplitEnumeratorContext<SplitT> enumContext)
Creates a new SplitEnumerator for this source, starting a new input.
|
SourceReader<T,SplitT> |
createReader(SourceReaderContext readerContext)
Creates a new reader to read data from the splits it gets assigned.
|
Boundedness |
getBoundedness()
Get the boundedness of this source.
|
SimpleVersionedSerializer<EnumChkT> |
getEnumeratorCheckpointSerializer()
Creates the serializer for the
SplitEnumerator checkpoint. |
SimpleVersionedSerializer<SplitT> |
getSplitSerializer()
Creates a serializer for the source splits.
|
SplitEnumerator<SplitT,EnumChkT> |
restoreEnumerator(SplitEnumeratorContext<SplitT> enumContext,
EnumChkT checkpoint)
Restores an enumerator from a checkpoint.
|
Boundedness getBoundedness()
SourceReader<T,SplitT> createReader(SourceReaderContext readerContext) throws Exception
SplitEnumerator<SplitT,EnumChkT> createEnumerator(SplitEnumeratorContext<SplitT> enumContext) throws Exception
SplitEnumerator<SplitT,EnumChkT> restoreEnumerator(SplitEnumeratorContext<SplitT> enumContext, EnumChkT checkpoint) throws Exception
enumContext
- The context
for the restored split
enumerator.checkpoint
- The checkpoint to restore the SplitEnumerator from.Exception
- The implementor is free to forward all exceptions directly. * Exceptions
thrown from this method cause JobManager failure/recovery.SimpleVersionedSerializer<SplitT> getSplitSerializer()
SimpleVersionedSerializer<EnumChkT> getEnumeratorCheckpointSerializer()
SplitEnumerator
checkpoint. The serializer is used for
the result of the SplitEnumerator#snapshotState()
method.Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.