@Internal public abstract class KafkaTableSource extends Object implements org.apache.flink.table.sources.StreamTableSource<Row>, org.apache.flink.table.sources.DefinedProctimeAttribute, org.apache.flink.table.sources.DefinedRowtimeAttributes, org.apache.flink.table.sources.DefinedFieldMapping
StreamTableSource
.
The version-specific Kafka consumers need to extend this class and
override createKafkaConsumer(String, Properties, DeserializationSchema)
}.
Modifier and Type | Class and Description |
---|---|
protected static class |
KafkaTableSource.Builder<T extends KafkaTableSource,B extends KafkaTableSource.Builder>
Deprecated.
Use the
Kafka descriptor together
with descriptors for schema and format instead. Descriptors allow for
implementation-agnostic definition of tables. See also
TableEnvironment.connect(ConnectorDescriptor) . |
Modifier | Constructor and Description |
---|---|
protected |
KafkaTableSource(org.apache.flink.table.api.TableSchema schema,
Optional<String> proctimeAttribute,
List<org.apache.flink.table.sources.RowtimeAttributeDescriptor> rowtimeAttributeDescriptors,
Optional<Map<String,String>> fieldMapping,
String topic,
Properties properties,
DeserializationSchema<Row> deserializationSchema,
StartupMode startupMode,
Map<KafkaTopicPartition,Long> specificStartupOffsets)
Creates a generic Kafka
StreamTableSource . |
protected |
KafkaTableSource(org.apache.flink.table.api.TableSchema schema,
String topic,
Properties properties,
DeserializationSchema<Row> deserializationSchema)
Creates a generic Kafka
StreamTableSource . |
Modifier and Type | Method and Description |
---|---|
protected abstract FlinkKafkaConsumerBase<Row> |
createKafkaConsumer(String topic,
Properties properties,
DeserializationSchema<Row> deserializationSchema)
Creates a version-specific Kafka consumer.
|
boolean |
equals(Object o) |
String |
explainSource() |
DataStream<Row> |
getDataStream(StreamExecutionEnvironment env)
NOTE: This method is for internal use only for defining a TableSource.
|
DeserializationSchema<Row> |
getDeserializationSchema()
Returns the deserialization schema.
|
Map<String,String> |
getFieldMapping() |
protected FlinkKafkaConsumerBase<Row> |
getKafkaConsumer(String topic,
Properties properties,
DeserializationSchema<Row> deserializationSchema)
Returns a version-specific Kafka consumer with the start position configured.
|
String |
getProctimeAttribute() |
Properties |
getProperties()
Returns the properties for the Kafka consumer.
|
TypeInformation<Row> |
getReturnType() |
List<org.apache.flink.table.sources.RowtimeAttributeDescriptor> |
getRowtimeAttributeDescriptors() |
org.apache.flink.table.api.TableSchema |
getTableSchema() |
int |
hashCode() |
protected void |
setFieldMapping(Map<String,String> fieldMapping)
Deprecated.
Use table descriptors instead of implementation-specific builders.
|
protected void |
setProctimeAttribute(String proctimeAttribute)
Deprecated.
Use table descriptors instead of implementation-specific builders.
|
protected void |
setRowtimeAttributeDescriptors(List<org.apache.flink.table.sources.RowtimeAttributeDescriptor> rowtimeAttributeDescriptors)
Deprecated.
Use table descriptors instead of implementation-specific builders.
|
protected void |
setSpecificStartupOffsets(Map<KafkaTopicPartition,Long> specificStartupOffsets)
Deprecated.
Use table descriptors instead of implementation-specific builders.
|
protected void |
setStartupMode(StartupMode startupMode)
Deprecated.
Use table descriptors instead of implementation-specific builders.
|
protected KafkaTableSource(org.apache.flink.table.api.TableSchema schema, Optional<String> proctimeAttribute, List<org.apache.flink.table.sources.RowtimeAttributeDescriptor> rowtimeAttributeDescriptors, Optional<Map<String,String>> fieldMapping, String topic, Properties properties, DeserializationSchema<Row> deserializationSchema, StartupMode startupMode, Map<KafkaTopicPartition,Long> specificStartupOffsets)
StreamTableSource
.schema
- Schema of the produced table.proctimeAttribute
- Field name of the processing time attribute.rowtimeAttributeDescriptors
- Descriptor for a rowtime attributefieldMapping
- Mapping for the fields of the table schema to
fields of the physical returned type.topic
- Kafka topic to consume.properties
- Properties for the Kafka consumer.deserializationSchema
- Deserialization schema for decoding records from Kafka.startupMode
- Startup mode for the contained consumer.specificStartupOffsets
- Specific startup offsets; only relevant when startup
mode is StartupMode.SPECIFIC_OFFSETS
.protected KafkaTableSource(org.apache.flink.table.api.TableSchema schema, String topic, Properties properties, DeserializationSchema<Row> deserializationSchema)
StreamTableSource
.schema
- Schema of the produced table.topic
- Kafka topic to consume.properties
- Properties for the Kafka consumer.deserializationSchema
- Deserialization schema for decoding records from Kafka.public DataStream<Row> getDataStream(StreamExecutionEnvironment env)
getDataStream
in interface org.apache.flink.table.sources.StreamTableSource<Row>
public TypeInformation<Row> getReturnType()
getReturnType
in interface org.apache.flink.table.sources.TableSource<Row>
public org.apache.flink.table.api.TableSchema getTableSchema()
getTableSchema
in interface org.apache.flink.table.sources.TableSource<Row>
public String getProctimeAttribute()
getProctimeAttribute
in interface org.apache.flink.table.sources.DefinedProctimeAttribute
public List<org.apache.flink.table.sources.RowtimeAttributeDescriptor> getRowtimeAttributeDescriptors()
getRowtimeAttributeDescriptors
in interface org.apache.flink.table.sources.DefinedRowtimeAttributes
public Map<String,String> getFieldMapping()
getFieldMapping
in interface org.apache.flink.table.sources.DefinedFieldMapping
public String explainSource()
explainSource
in interface org.apache.flink.table.sources.TableSource<Row>
public Properties getProperties()
public DeserializationSchema<Row> getDeserializationSchema()
protected FlinkKafkaConsumerBase<Row> getKafkaConsumer(String topic, Properties properties, DeserializationSchema<Row> deserializationSchema)
topic
- Kafka topic to consume.properties
- Properties for the Kafka consumer.deserializationSchema
- Deserialization schema to use for Kafka records.@Deprecated protected void setProctimeAttribute(String proctimeAttribute)
proctimeAttribute
- The name of the field that becomes the processing time field.@Deprecated protected void setRowtimeAttributeDescriptors(List<org.apache.flink.table.sources.RowtimeAttributeDescriptor> rowtimeAttributeDescriptors)
rowtimeAttributeDescriptors
- The descriptors of the rowtime attributes.@Deprecated protected void setStartupMode(StartupMode startupMode)
startupMode
- The startup mode.@Deprecated protected void setSpecificStartupOffsets(Map<KafkaTopicPartition,Long> specificStartupOffsets)
StartupMode.SPECIFIC_OFFSETS
.specificStartupOffsets
- The startup offsets for different partitions.@Deprecated protected void setFieldMapping(Map<String,String> fieldMapping)
fieldMapping
- The mapping from table schema fields to format schema fields.protected abstract FlinkKafkaConsumerBase<Row> createKafkaConsumer(String topic, Properties properties, DeserializationSchema<Row> deserializationSchema)
topic
- Kafka topic to consume.properties
- Properties for the Kafka consumer.deserializationSchema
- Deserialization schema to use for Kafka records.Copyright © 2014–2020 The Apache Software Foundation. All rights reserved.