@PublicEvolving public interface ScanTableSource extends DynamicTableSource
DynamicTableSource
that scans all rows from an external storage system during runtime.
The scanned rows don't have to contain only insertions but can also contain updates and
deletions. Thus, the table source can be used to read a (finite or infinite) changelog. The given
ChangelogMode
indicates the set of changes that the planner can expect during runtime.
For regular batch scenarios, the source can emit a bounded stream of insert-only rows.
For regular streaming scenarios, the source can emit an unbounded stream of insert-only rows.
For change data capture (CDC) scenarios, the source can emit bounded or unbounded streams with
insert, update, and delete rows. See also RowKind
.
A ScanTableSource
can implement the following abilities that might mutate an instance
during planning:
SupportsWatermarkPushDown
SupportsSourceWatermark
SupportsFilterPushDown
SupportsAggregatePushDown
SupportsProjectionPushDown
SupportsPartitionPushDown
SupportsReadingMetadata
In the last step, the planner will call getScanRuntimeProvider(ScanContext)
for
obtaining a provider of runtime implementation.
Modifier and Type | Interface and Description |
---|---|
static interface |
ScanTableSource.ScanContext
Context for creating runtime implementation via a
ScanTableSource.ScanRuntimeProvider . |
static interface |
ScanTableSource.ScanRuntimeProvider
Provides actual runtime implementation for reading the data.
|
DynamicTableSource.Context, DynamicTableSource.DataStructureConverter
Modifier and Type | Method and Description |
---|---|
ChangelogMode |
getChangelogMode()
Returns the set of changes that the planner can expect during runtime.
|
ScanTableSource.ScanRuntimeProvider |
getScanRuntimeProvider(ScanTableSource.ScanContext runtimeProviderContext)
Returns a provider of runtime implementation for reading the data.
|
asSummaryString, copy
ChangelogMode getChangelogMode()
RowKind
ScanTableSource.ScanRuntimeProvider getScanRuntimeProvider(ScanTableSource.ScanContext runtimeProviderContext)
There might exist different interfaces for runtime implementation which is why ScanTableSource.ScanRuntimeProvider
serves as the base interface. Concrete ScanTableSource.ScanRuntimeProvider
interfaces might be located in other Flink modules.
Independent of the provider interface, the table runtime expects that a source
implementation emits internal data structures (see RowData
for more information).
The given ScanTableSource.ScanContext
offers utilities by the planner for creating runtime
implementation with minimal dependencies to internal data structures.
See org.apache.flink.table.connector.source.SourceFunctionProvider
in flink-table-api-java-bridge
.
Copyright © 2014–2022 The Apache Software Foundation. All rights reserved.