org.apache.flink.connector.base.sink

Class AsyncSinkBaseBuilder<InputT,RequestEntryT extends Serializable,ConcreteBuilderT extends AsyncSinkBaseBuilder<?,?,?>>

java.lang.Object

org.apache.flink.connector.base.sink.AsyncSinkBaseBuilder<InputT,RequestEntryT,ConcreteBuilderT>

Type Parameters:

InputT - type of elements that should be persisted in the destination

RequestEntryT - type of payload that contains the element and additional metadata that is required to submit a single element to the destination

ConcreteBuilderT - type of concrete implementation of this builder class

@PublicEvolving
public abstract class AsyncSinkBaseBuilder<InputT,RequestEntryT extends Serializable,ConcreteBuilderT extends AsyncSinkBaseBuilder<?,?,?>>
extends Object

Abstract builder for constructing a concrete implementation of AsyncSinkBase.

Constructor Summary

Constructors

Constructor and Description
AsyncSinkBaseBuilder()

Method Summary

Modifier and Type	Method and Description
abstract AsyncSinkBase<InputT,RequestEntryT>	build() Builds the Sink with the settings applied to this builder.
protected Integer	getMaxBatchSize()
protected Long	getMaxBatchSizeInBytes()
protected Integer	getMaxBufferedRequests()
protected Integer	getMaxInFlightRequests()
protected Long	getMaxRecordSizeInBytes()
protected Long	getMaxTimeInBufferMS()
ConcreteBuilderT	setMaxBatchSize(int maxBatchSize)
ConcreteBuilderT	setMaxBatchSizeInBytes(long maxBatchSizeInBytes)
ConcreteBuilderT	setMaxBufferedRequests(int maxBufferedRequests)
ConcreteBuilderT	setMaxInFlightRequests(int maxInFlightRequests)
ConcreteBuilderT	setMaxRecordSizeInBytes(long maxRecordSizeInBytes)
ConcreteBuilderT	setMaxTimeInBufferMS(long maxTimeInBufferMS)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AsyncSinkBaseBuilder

public AsyncSinkBaseBuilder()

Method Detail

setMaxBatchSize

public ConcreteBuilderT setMaxBatchSize(int maxBatchSize)

Parameters:

maxBatchSize - maximum number of elements that may be passed in a list to be written downstream.

Returns:

ConcreteBuilderT itself

setMaxInFlightRequests

public ConcreteBuilderT setMaxInFlightRequests(int maxInFlightRequests)

Parameters:

maxInFlightRequests - maximum number of uncompleted calls to submitRequestEntries that the SinkWriter will allow at any given point. Once this point has reached, writes and callbacks to add elements to the buffer may block until one or more requests to submitRequestEntries completes.

Returns:

ConcreteBuilderT itself

setMaxBufferedRequests

public ConcreteBuilderT setMaxBufferedRequests(int maxBufferedRequests)

Parameters:

maxBufferedRequests - the maximum buffer length. Callbacks to add elements to the buffer and calls to write will block if this length has been reached and will only unblock if elements from the buffer have been removed for flushing.

Returns:

ConcreteBuilderT itself

setMaxBatchSizeInBytes

public ConcreteBuilderT setMaxBatchSizeInBytes(long maxBatchSizeInBytes)

Parameters:

maxBatchSizeInBytes - a flush will be attempted if the most recent call to write introduces an element to the buffer such that the total size of the buffer is greater than or equal to this threshold value. If this happens, the maximum number of elements from the head of the buffer will be selected, that is smaller than maxBatchSizeInBytes in size will be flushed.

Returns:

ConcreteBuilderT itself

setMaxTimeInBufferMS

public ConcreteBuilderT setMaxTimeInBufferMS(long maxTimeInBufferMS)

Parameters:

maxTimeInBufferMS - the maximum amount of time an element may remain in the buffer. In most cases elements are flushed as a result of the batch size (in bytes or number) being reached or during a snapshot. However, there are scenarios where an element may remain in the buffer forever or a long period of time. To mitigate this, a timer is constantly active in the buffer such that: while the buffer is not empty, it will flush every maxTimeInBufferMS milliseconds.

Returns:

ConcreteBuilderT itself

setMaxRecordSizeInBytes

public ConcreteBuilderT setMaxRecordSizeInBytes(long maxRecordSizeInBytes)

Parameters:

maxRecordSizeInBytes - the maximum size of each records in bytes. If a record larger than this is passed to the sink, it will throw an IllegalArgumentException.

Returns:

ConcreteBuilderT itself

build

public abstract AsyncSinkBase<InputT,RequestEntryT> build()

Builds the Sink with the settings applied to this builder.

getMaxBatchSize

protected Integer getMaxBatchSize()

getMaxInFlightRequests

protected Integer getMaxInFlightRequests()

getMaxBufferedRequests

protected Integer getMaxBufferedRequests()

getMaxBatchSizeInBytes

protected Long getMaxBatchSizeInBytes()

getMaxTimeInBufferMS

protected Long getMaxTimeInBufferMS()

getMaxRecordSizeInBytes

protected Long getMaxRecordSizeInBytes()