org.apache.flink.streaming.connectors.kafka

Class KafkaTableSource.Builder<T extends KafkaTableSource,B extends KafkaTableSource.Builder>

java.lang.Object

org.apache.flink.streaming.connectors.kafka.KafkaTableSource.Builder<T,B>

Type Parameters:

T - Type of the KafkaTableSource produced by the builder.

B - Type of the KafkaTableSource.Builder subclass.

Direct Known Subclasses:

KafkaAvroTableSource.Builder, KafkaJsonTableSource.Builder

Enclosing class:

KafkaTableSource

protected abstract static class KafkaTableSource.Builder<T extends KafkaTableSource,B extends KafkaTableSource.Builder>
extends Object

Abstract builder for a KafkaTableSource to be extended by builders of subclasses of KafkaTableSource.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Builder()

Method Summary

Modifier and Type	Method and Description
protected abstract KafkaTableSource	build() Builds the configured KafkaTableSource.
protected abstract B	builder() Returns the builder.
protected void	configureTableSource(T tableSource) Configures a TableSource with optional parameters.
B	forTopic(String topic) Sets the topic from which the table is read.
B	fromEarliest() Configures the TableSource to start reading from the earliest offset for all partitions.
B	fromGroupOffsets() Configures the TableSource to start reading from any committed group offsets found in Zookeeper / Kafka brokers.
B	fromLatest() Configures the TableSource to start reading from the latest offset for all partitions.
B	fromSpecificOffsets(Map<KafkaTopicPartition,Long> specificStartupOffsets) Configures the TableSource to start reading partitions from specific offsets, set independently for each partition.
protected Properties	getKafkaProps() Returns the configured Kafka properties.
protected org.apache.flink.table.api.TableSchema	getTableSchema() Returns the configured table schema.
protected String	getTopic() Returns the configured topic.
protected abstract boolean	supportsKafkaTimestamps() True if the KafkaSource supports Kafka timestamps, false otherwise.
B	withKafkaProperties(Properties props) Sets the configuration properties for the Kafka consumer.
B	withKafkaTimestampAsRowtimeAttribute(String rowtimeAttribute, org.apache.flink.table.sources.wmstrategies.WatermarkStrategy watermarkStrategy) Configures the Kafka timestamp to be a rowtime attribute.
B	withProctimeAttribute(String proctimeAttribute) Configures a field of the table to be a processing time attribute.
B	withRowtimeAttribute(String rowtimeAttribute, org.apache.flink.table.sources.tsextractors.TimestampExtractor timestampExtractor, org.apache.flink.table.sources.wmstrategies.WatermarkStrategy watermarkStrategy) Configures a field of the table to be a rowtime attribute.
B	withSchema(org.apache.flink.table.api.TableSchema schema) Sets the schema of the produced table.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

protected Builder()

Method Detail

forTopic

public B forTopic(String topic)

Sets the topic from which the table is read.

Parameters:

topic - The topic from which the table is read.

Returns:

The builder.

withKafkaProperties

public B withKafkaProperties(Properties props)

Sets the configuration properties for the Kafka consumer.

Parameters:

props - The configuration properties for the Kafka consumer.

Returns:

The builder.

withSchema

public B withSchema(org.apache.flink.table.api.TableSchema schema)

Sets the schema of the produced table.

Parameters:

schema - The schema of the produced table.

Returns:

The builder.

withProctimeAttribute

public B withProctimeAttribute(String proctimeAttribute)

Configures a field of the table to be a processing time attribute. The configured field must be present in the table schema and of type Types.SQL_TIMESTAMP().

Parameters:

proctimeAttribute - The name of the processing time attribute in the table schema.

Returns:

The builder.

withRowtimeAttribute

public B withRowtimeAttribute(String rowtimeAttribute,
                              org.apache.flink.table.sources.tsextractors.TimestampExtractor timestampExtractor,
                              org.apache.flink.table.sources.wmstrategies.WatermarkStrategy watermarkStrategy)

Configures a field of the table to be a rowtime attribute. The configured field must be present in the table schema and of type Types.SQL_TIMESTAMP().

Parameters:

rowtimeAttribute - The name of the rowtime attribute in the table schema.

timestampExtractor - The TimestampExtractor to extract the rowtime attribute from the physical type.

watermarkStrategy - The WatermarkStrategy to generate watermarks for the rowtime attribute.

Returns:

The builder.

withKafkaTimestampAsRowtimeAttribute

public B withKafkaTimestampAsRowtimeAttribute(String rowtimeAttribute,
                                              org.apache.flink.table.sources.wmstrategies.WatermarkStrategy watermarkStrategy)

Configures the Kafka timestamp to be a rowtime attribute.

Note: Kafka supports message timestamps only since version 0.10.

Parameters:

rowtimeAttribute - The name of the rowtime attribute in the table schema.

watermarkStrategy - The WatermarkStrategy to generate watermarks for the rowtime attribute.

Returns:

The builder.

fromEarliest

public B fromEarliest()

Configures the TableSource to start reading from the earliest offset for all partitions.

See Also:

FlinkKafkaConsumerBase.setStartFromEarliest()

fromLatest

public B fromLatest()

Configures the TableSource to start reading from the latest offset for all partitions.

See Also:

FlinkKafkaConsumerBase.setStartFromLatest()

fromGroupOffsets

public B fromGroupOffsets()

Configures the TableSource to start reading from any committed group offsets found in Zookeeper / Kafka brokers.

See Also:

FlinkKafkaConsumerBase.setStartFromGroupOffsets()

fromSpecificOffsets

public B fromSpecificOffsets(Map<KafkaTopicPartition,Long> specificStartupOffsets)

Configures the TableSource to start reading partitions from specific offsets, set independently for each partition.

Parameters:

specificStartupOffsets - the specified offsets for partitions

See Also:

FlinkKafkaConsumerBase.setStartFromSpecificOffsets(Map)

getTopic

protected String getTopic()

Returns the configured topic.

Returns:

the configured topic.

getKafkaProps

protected Properties getKafkaProps()

Returns the configured Kafka properties.

Returns:

the configured Kafka properties.

getTableSchema

protected org.apache.flink.table.api.TableSchema getTableSchema()

Returns the configured table schema.

Returns:

the configured table schema.

supportsKafkaTimestamps

protected abstract boolean supportsKafkaTimestamps()

True if the KafkaSource supports Kafka timestamps, false otherwise.

Returns:

True if the KafkaSource supports Kafka timestamps, false otherwise.

configureTableSource

protected void configureTableSource(T tableSource)

Configures a TableSource with optional parameters.

Parameters:

tableSource - The TableSource to configure.

builder

protected abstract B builder()

Returns the builder.

Returns:

the builder.

build

protected abstract KafkaTableSource build()

Builds the configured KafkaTableSource.

Returns:

The configured KafkaTableSource.