pyflink.datastream.connectors.pulsar.PulsarSink

class PulsarSink(j_pulsar_sink)

The Sink implementation of Pulsar. Please use a PulsarSinkBuilder to construct a PulsarSink. The following example shows how to create a PulsarSink receiving records of String type.

Example:

>>> sink = PulsarSink.builder() \
...     .set_service_url(PULSAR_BROKER_URL) \
...     .set_admin_url(PULSAR_BROKER_HTTP_URL) \
...     .set_topics(topic) \
...     .set_serialization_schema(SimpleStringSchema()) \
...     .build()

The sink supports all delivery guarantees described by DeliveryGuarantee.

DeliveryGuarantee#NONE does not provide any guarantees: messages may be lost in case of issues on the Pulsar broker and messages may be duplicated in case of a Flink failure.

DeliveryGuarantee#AT_LEAST_ONCE the sink will wait for all outstanding records in the Pulsar buffers to be acknowledged by the Pulsar producer on a checkpoint. No messages will be lost in case of any issue with the Pulsar brokers but messages may be duplicated when Flink restarts.

DeliveryGuarantee#EXACTLY_ONCE: In this mode the PulsarSink will write all messages in a Pulsar transaction that will be committed to Pulsar on a checkpoint. Thus, no duplicates will be seen in case of a Flink restart. However, this delays record writing effectively until a checkpoint is written, so adjust the checkpoint duration accordingly. Additionally, it is highly recommended to tweak Pulsar transaction timeout (link) >> maximum checkpoint duration + maximum restart duration or data loss may happen when Pulsar expires an uncommitted transaction.

See PulsarSinkBuilder for more details.

Methods

builder()	Get a PulsarSinkBuilder to builder a PulsarSink.
get_java_function()