pyflink.datastream.connectors.kinesis.KinesisFirehoseSinkBuilder#

class KinesisFirehoseSinkBuilder[source]#

Builder to construct KinesisFirehoseSink.

The following example shows the minimum setup to create a KinesisFirehoseSink that writes String values to a Kinesis Data Firehose delivery stream named delivery-stream-name.

Example:

>>> from pyflink.common.serialization import SimpleStringSchema
>>> sink_properties = {"aws.region": "eu-west-1"}
>>> sink = KinesisFirehoseSink.builder() \
...     .set_firehose_client_properties(sink_properties) \
...     .set_delivery_stream_name("delivery-stream-name") \
...     .set_serialization_schema(SimpleStringSchema()) \
...     .set_max_batch_size(20) \
...     .build()

If the following parameters are not set in this builder, the following defaults will be used: - maxBatchSize will be 500 - maxInFlightRequests will be 50 - maxBufferedRequests will be 10000 - maxBatchSizeInBytes will be 4 MB i.e. 4 * 1024 * 1024 - maxTimeInBufferMS will be 5000ms - maxRecordSizeInBytes will be 1000 KB i.e. 1000 * 1024 - failOnError will be false

Methods

`build`()	Build thd KinesisFirehoseSink.
`set_delivery_stream_name`(delivery_stream_name)	Sets the name of the KDF delivery stream that the sink will connect to.
`set_fail_on_error`(fail_on_error)	If writing to Kinesis Data Firehose results in a partial or full failure being returned, the job will fail
`set_firehose_client_properties`(...)	A set of properties used by the sink to create the firehose client.
`set_max_batch_size`(max_batch_size)	Maximum number of elements that may be passed in a list to be written downstream.
`set_max_batch_size_in_bytes`(...)	The flush will be attempted if the most recent call to write introduces an element to the buffer such that the total size of the buffer is greater than or equal to this threshold value.
`set_max_buffered_requests`(max_buffered_requests)	The maximum buffer length.
`set_max_in_flight_requests`(...)	Maximum number of uncompleted calls to submitRequestEntries that the SinkWriter will allow at any given point.
`set_max_record_size_in_bytes`(...)	The maximum size of each records in bytes.
`set_max_time_in_buffer_ms`(max_time_in_buffer_ms)	The maximum amount of time an element may remain in the buffer.
`set_serialization_schema`(serialization_schema)	Allows the user to specify a serialization schema to serialize each record to persist to Firehose.