@Internal public class KinesisDynamicSource extends Object implements ScanTableSource, SupportsReadingMetadata
ScanTableSource
.ScanTableSource.ScanContext, ScanTableSource.ScanRuntimeProvider
DynamicTableSource.Context, DynamicTableSource.DataStructureConverter
Constructor and Description |
---|
KinesisDynamicSource(DataType physicalDataType,
String stream,
Properties consumerProperties,
DecodingFormat<DeserializationSchema<RowData>> decodingFormat) |
KinesisDynamicSource(DataType physicalDataType,
String stream,
Properties consumerProperties,
DecodingFormat<DeserializationSchema<RowData>> decodingFormat,
DataType producedDataType,
List<RowDataKinesisDeserializationSchema.Metadata> requestedMetadataFields) |
Modifier and Type | Method and Description |
---|---|
void |
applyReadableMetadata(List<String> metadataKeys,
DataType producedDataType)
Provides a list of metadata keys that the produced
RowData must contain as appended
metadata columns. |
String |
asSummaryString()
Returns a string that summarizes this source for printing to a console or log.
|
DynamicTableSource |
copy()
Creates a copy of this instance during planning.
|
boolean |
equals(Object o) |
ChangelogMode |
getChangelogMode()
Returns the set of changes that the planner can expect during runtime.
|
ScanTableSource.ScanRuntimeProvider |
getScanRuntimeProvider(ScanTableSource.ScanContext runtimeProviderContext)
Returns a provider of runtime implementation for reading the data.
|
int |
hashCode() |
Map<String,DataType> |
listReadableMetadata()
Returns the map of metadata keys and their corresponding data types that can be produced by
this table source for reading.
|
public KinesisDynamicSource(DataType physicalDataType, String stream, Properties consumerProperties, DecodingFormat<DeserializationSchema<RowData>> decodingFormat)
public KinesisDynamicSource(DataType physicalDataType, String stream, Properties consumerProperties, DecodingFormat<DeserializationSchema<RowData>> decodingFormat, DataType producedDataType, List<RowDataKinesisDeserializationSchema.Metadata> requestedMetadataFields)
public ChangelogMode getChangelogMode()
ScanTableSource
getChangelogMode
in interface ScanTableSource
RowKind
public ScanTableSource.ScanRuntimeProvider getScanRuntimeProvider(ScanTableSource.ScanContext runtimeProviderContext)
ScanTableSource
There might exist different interfaces for runtime implementation which is why ScanTableSource.ScanRuntimeProvider
serves as the base interface. Concrete ScanTableSource.ScanRuntimeProvider
interfaces might be located in other Flink modules.
Independent of the provider interface, the table runtime expects that a source
implementation emits internal data structures (see RowData
for more information).
The given ScanTableSource.ScanContext
offers utilities by the planner for creating runtime
implementation with minimal dependencies to internal data structures.
See org.apache.flink.table.connector.source.SourceFunctionProvider
in flink-table-api-java-bridge
.
getScanRuntimeProvider
in interface ScanTableSource
public DynamicTableSource copy()
DynamicTableSource
copy
in interface DynamicTableSource
public String asSummaryString()
DynamicTableSource
asSummaryString
in interface DynamicTableSource
public Map<String,DataType> listReadableMetadata()
SupportsReadingMetadata
The returned map will be used by the planner for validation and insertion of explicit
casts (see LogicalTypeCasts.supportsExplicitCast(LogicalType, LogicalType)
) if
necessary.
The iteration order of the returned map determines the order of metadata keys in the list
passed in SupportsReadingMetadata.applyReadableMetadata(List, DataType)
. Therefore, it might be beneficial
to return a LinkedHashMap
if a strict metadata column order is required.
If a source forwards metadata from one or more formats, we recommend the following column order for consistency:
KEY FORMAT METADATA COLUMNS + VALUE FORMAT METADATA COLUMNS + SOURCE METADATA COLUMNS
Metadata key names follow the same pattern as mentioned in Factory
. In case of
duplicate names in format and source keys, format keys shall have higher precedence.
Regardless of the returned DataType
s, a metadata column is always represented
using internal data structures (see RowData
).
listReadableMetadata
in interface SupportsReadingMetadata
DecodingFormat.listReadableMetadata()
public void applyReadableMetadata(List<String> metadataKeys, DataType producedDataType)
SupportsReadingMetadata
RowData
must contain as appended
metadata columns.
Note: Use the passed data type instead of TableSchema.toPhysicalRowDataType()
for
describing the final output data type when creating TypeInformation
. If the source
implements SupportsProjectionPushDown
, the projection is already considered in the
given output data type.
applyReadableMetadata
in interface SupportsReadingMetadata
metadataKeys
- a subset of the keys returned by SupportsReadingMetadata.listReadableMetadata()
, ordered
by the iteration order of returned mapproducedDataType
- the final output type of the sourceDecodingFormat.applyReadableMetadata(List)
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.