org.apache.flink.table.connector.source

Interface ScanTableSource

All Superinterfaces:

DynamicTableSource

All Known Implementing Classes:

DataGenTableSource, FileSystemTableSource, HiveLookupTableSource, HiveTableSource, PythonDynamicTableSource, SocketDynamicTableSource, TestScanTableSourceWithWatermarkPushDown

@PublicEvolving
public interface ScanTableSource
extends DynamicTableSource

A DynamicTableSource that scans all rows from an external storage system during runtime.

The scanned rows don't have to contain only insertions but can also contain updates and deletions. Thus, the table source can be used to read a (finite or infinite) changelog. The given ChangelogMode indicates the set of changes that the planner can expect during runtime.

For regular batch scenarios, the source can emit a bounded stream of insert-only rows.

For regular streaming scenarios, the source can emit an unbounded stream of insert-only rows.

For change data capture (CDC) scenarios, the source can emit bounded or unbounded streams with insert, update, and delete rows. See also RowKind.

A ScanTableSource can implement the following abilities that might mutate an instance during planning:

• SupportsWatermarkPushDown
• SupportsSourceWatermark
• SupportsFilterPushDown
• SupportsAggregatePushDown
• SupportsProjectionPushDown
• SupportsPartitionPushDown
• SupportsReadingMetadata

In the last step, the planner will call getScanRuntimeProvider(ScanContext) for obtaining a provider of runtime implementation.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	ScanTableSource.ScanContext Context for creating runtime implementation via a ScanTableSource.ScanRuntimeProvider.
static interface	ScanTableSource.ScanRuntimeProvider Provides actual runtime implementation for reading the data.

Nested classes/interfaces inherited from interface org.apache.flink.table.connector.source.DynamicTableSource

DynamicTableSource.Context, DynamicTableSource.DataStructureConverter

Method Summary

Modifier and Type	Method and Description
ChangelogMode	getChangelogMode() Returns the set of changes that the planner can expect during runtime.
ScanTableSource.ScanRuntimeProvider	getScanRuntimeProvider(ScanTableSource.ScanContext runtimeProviderContext) Returns a provider of runtime implementation for reading the data.

Methods inherited from interface org.apache.flink.table.connector.source.DynamicTableSource

asSummaryString, copy

Method Detail

getChangelogMode

ChangelogMode getChangelogMode()

Returns the set of changes that the planner can expect during runtime.

See Also:

RowKind

getScanRuntimeProvider

ScanTableSource.ScanRuntimeProvider getScanRuntimeProvider(ScanTableSource.ScanContext runtimeProviderContext)

Returns a provider of runtime implementation for reading the data.

There might exist different interfaces for runtime implementation which is why ScanTableSource.ScanRuntimeProvider serves as the base interface. Concrete ScanTableSource.ScanRuntimeProvider interfaces might be located in other Flink modules.

Independent of the provider interface, the table runtime expects that a source implementation emits internal data structures (see RowData for more information).

The given ScanTableSource.ScanContext offers utilities by the planner for creating runtime implementation with minimal dependencies to internal data structures.

SourceProvider is the recommended core interface. SourceFunctionProvider in flink-table-api-java-bridge and InputFormatProvider are available for backwards compatibility.

See Also:

SourceProvider