org.apache.flink.connector.kafka.sink

Class KafkaSinkBuilder<IN>

java.lang.Object

org.apache.flink.connector.kafka.sink.KafkaSinkBuilder<IN>

Type Parameters:

IN - type of the records written to Kafka

@PublicEvolving
public class KafkaSinkBuilder<IN>
extends Object

Builder to construct KafkaSink.

The following example shows the minimum setup to create a KafkaSink that writes String values to a Kafka topic.

 KafkaSink<String> sink = KafkaSink
     .<String>builder
     .setBootstrapServers(MY_BOOTSTRAP_SERVERS)
     .setRecordSerializer(MY_RECORD_SERIALIZER)
     .build();
 

One can also configure different DeliveryGuarantee by using setDeliveryGuarantee(DeliveryGuarantee) but keep in mind when using DeliveryGuarantee.EXACTLY_ONCE one must set the transactionalIdPrefix setTransactionalIdPrefix(String).

See Also:

for a more detailed explanation of the different guarantees.

Method Summary

Modifier and Type	Method and Description
KafkaSink<IN>	build() Constructs the KafkaSink with the configured properties.
KafkaSinkBuilder<IN>	setBootstrapServers(String bootstrapServers) Sets the Kafka bootstrap servers.
KafkaSinkBuilder<IN>	setDeliverGuarantee(DeliveryGuarantee deliveryGuarantee) Deprecated. Will be removed in future versions. Use setDeliveryGuarantee(org.apache.flink.connector.base.DeliveryGuarantee) instead.
KafkaSinkBuilder<IN>	setDeliveryGuarantee(DeliveryGuarantee deliveryGuarantee) Sets the wanted the DeliveryGuarantee.
KafkaSinkBuilder<IN>	setKafkaProducerConfig(Properties props) Sets the configuration which used to instantiate all used KafkaProducer.
KafkaSinkBuilder<IN>	setProperty(String key, String value)
KafkaSinkBuilder<IN>	setRecordSerializer(KafkaRecordSerializationSchema<IN> recordSerializer) Sets the KafkaRecordSerializationSchema that transforms incoming records to ProducerRecords.
KafkaSinkBuilder<IN>	setTransactionalIdPrefix(String transactionalIdPrefix) Sets the prefix for all created transactionalIds if DeliveryGuarantee.EXACTLY_ONCE is configured.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

setDeliveryGuarantee

public KafkaSinkBuilder<IN> setDeliveryGuarantee(DeliveryGuarantee deliveryGuarantee)

Sets the wanted the DeliveryGuarantee. The default delivery guarantee is deliveryGuarantee.

Parameters:

deliveryGuarantee -

Returns:

KafkaSinkBuilder

setDeliverGuarantee

@Deprecated
public KafkaSinkBuilder<IN> setDeliverGuarantee(DeliveryGuarantee deliveryGuarantee)

Deprecated. Will be removed in future versions. Use setDeliveryGuarantee(org.apache.flink.connector.base.DeliveryGuarantee) instead.

Sets the wanted the DeliveryGuarantee. The default delivery guarantee is deliveryGuarantee.

Parameters:

deliveryGuarantee -

Returns:

KafkaSinkBuilder

setKafkaProducerConfig

public KafkaSinkBuilder<IN> setKafkaProducerConfig(Properties props)

Sets the configuration which used to instantiate all used KafkaProducer.

Parameters:

props -

Returns:

KafkaSinkBuilder

setProperty

public KafkaSinkBuilder<IN> setProperty(String key,
                                        String value)

setRecordSerializer

public KafkaSinkBuilder<IN> setRecordSerializer(KafkaRecordSerializationSchema<IN> recordSerializer)

Sets the KafkaRecordSerializationSchema that transforms incoming records to ProducerRecords.

Parameters:

recordSerializer -

Returns:

KafkaSinkBuilder

setTransactionalIdPrefix

public KafkaSinkBuilder<IN> setTransactionalIdPrefix(String transactionalIdPrefix)

Sets the prefix for all created transactionalIds if DeliveryGuarantee.EXACTLY_ONCE is configured.

It is mandatory to always set this value with DeliveryGuarantee.EXACTLY_ONCE to prevent corrupted transactions if multiple jobs using the KafkaSink run against the same Kafka Cluster. The default prefix is transactionalIdPrefix.

The size of the prefix is capped by MAXIMUM_PREFIX_BYTES formatted with UTF-8.

It is important to keep the prefix stable across application restarts. If the prefix changes it might happen that lingering transactions are not correctly aborted and newly written messages are not immediately consumable until the transactions timeout.

Parameters:

transactionalIdPrefix -

Returns:

KafkaSinkBuilder

setBootstrapServers

public KafkaSinkBuilder<IN> setBootstrapServers(String bootstrapServers)

Sets the Kafka bootstrap servers.

Parameters:

bootstrapServers - a comma separated list of valid URIs to reach the Kafka broker

Returns:

KafkaSinkBuilder

build

public KafkaSink<IN> build()

Constructs the KafkaSink with the configured properties.

Returns:

KafkaSink