org.apache.flink.connectors.hive

Class HiveSource<T>

java.lang.Object

org.apache.flink.connector.file.src.AbstractFileSource<T,HiveSourceSplit>

org.apache.flink.connectors.hive.HiveSource<T>

Type Parameters:

T - the type of record returned by this source

All Implemented Interfaces:

Serializable, Source<T,HiveSourceSplit,PendingSplitsCheckpoint<HiveSourceSplit>>, SourceReaderFactory<T,HiveSourceSplit>, ResultTypeQueryable<T>

@PublicEvolving
public class HiveSource<T>
extends AbstractFileSource<T,HiveSourceSplit>

A unified data source that reads a hive table. HiveSource works on HiveSourceSplit and uses BulkFormat to read the data. A built-in BulkFormat is provided to return records in type of RowData. It's also possible to implement a custom BulkFormat to return data in different types. Use HiveSourceBuilder to build HiveSource instances.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.flink.connector.file.src.AbstractFileSource

AbstractFileSource.AbstractFileSourceBuilder<T,SplitT extends FileSourceSplit,SELF extends AbstractFileSource.AbstractFileSourceBuilder<T,SplitT,SELF>>

Method Summary

Modifier and Type	Method and Description
SplitEnumerator<HiveSourceSplit,PendingSplitsCheckpoint<HiveSourceSplit>>	createEnumerator(SplitEnumeratorContext<HiveSourceSplit> enumContext) Creates a new SplitEnumerator for this source, starting a new input.
SimpleVersionedSerializer<PendingSplitsCheckpoint<HiveSourceSplit>>	getEnumeratorCheckpointSerializer() Creates the serializer for the SplitEnumerator checkpoint.
SimpleVersionedSerializer<HiveSourceSplit>	getSplitSerializer() Creates a serializer for the source splits.
SplitEnumerator<HiveSourceSplit,PendingSplitsCheckpoint<HiveSourceSplit>>	restoreEnumerator(SplitEnumeratorContext<HiveSourceSplit> enumContext, PendingSplitsCheckpoint<HiveSourceSplit> checkpoint) Restores an enumerator from a checkpoint.

Methods inherited from class org.apache.flink.connector.file.src.AbstractFileSource

createReader, getAssignerFactory, getBoundedness, getContinuousEnumerationSettings, getProducedType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getSplitSerializer

public SimpleVersionedSerializer<HiveSourceSplit> getSplitSerializer()

Description copied from interface: Source

Creates a serializer for the source splits. Splits are serialized when sending them from enumerator to reader, and when checkpointing the reader's current state.

Specified by:

getSplitSerializer in interface Source<T,HiveSourceSplit,PendingSplitsCheckpoint<HiveSourceSplit>>

Specified by:

getSplitSerializer in class AbstractFileSource<T,HiveSourceSplit>

Returns:

The serializer for the split type.

getEnumeratorCheckpointSerializer

public SimpleVersionedSerializer<PendingSplitsCheckpoint<HiveSourceSplit>> getEnumeratorCheckpointSerializer()

Description copied from interface: Source

Creates the serializer for the SplitEnumerator checkpoint. The serializer is used for the result of the SplitEnumerator.snapshotState(long) method.

Specified by:

getEnumeratorCheckpointSerializer in interface Source<T,HiveSourceSplit,PendingSplitsCheckpoint<HiveSourceSplit>>

Overrides:

getEnumeratorCheckpointSerializer in class AbstractFileSource<T,HiveSourceSplit>

Returns:

The serializer for the SplitEnumerator checkpoint.

createEnumerator

public SplitEnumerator<HiveSourceSplit,PendingSplitsCheckpoint<HiveSourceSplit>> createEnumerator(SplitEnumeratorContext<HiveSourceSplit> enumContext)

Description copied from interface: Source

Creates a new SplitEnumerator for this source, starting a new input.

Specified by:

createEnumerator in interface Source<T,HiveSourceSplit,PendingSplitsCheckpoint<HiveSourceSplit>>

Overrides:

createEnumerator in class AbstractFileSource<T,HiveSourceSplit>

Parameters:

enumContext - The context for the split enumerator.

Returns:

A new SplitEnumerator.

restoreEnumerator

public SplitEnumerator<HiveSourceSplit,PendingSplitsCheckpoint<HiveSourceSplit>> restoreEnumerator(SplitEnumeratorContext<HiveSourceSplit> enumContext,
                                                                                                   PendingSplitsCheckpoint<HiveSourceSplit> checkpoint)

Description copied from interface: Source

Restores an enumerator from a checkpoint.

Specified by:

restoreEnumerator in interface Source<T,HiveSourceSplit,PendingSplitsCheckpoint<HiveSourceSplit>>

Overrides:

restoreEnumerator in class AbstractFileSource<T,HiveSourceSplit>

Parameters:

enumContext - The context for the restored split enumerator.

checkpoint - The checkpoint to restore the SplitEnumerator from.

Returns:

A SplitEnumerator restored from the given checkpoint.