Package org.apache.flink.connector.base.sink

Class AsyncSinkBase<InputT,​RequestEntryT extends Serializable>

  • All Implemented Interfaces:
    Serializable, Sink<InputT>, SupportsWriterState<InputT,​BufferedRequestState<RequestEntryT>>
    @PublicEvolving
public abstract class AsyncSinkBase<InputT,​RequestEntryT extends Serializable>
extends Object
implements SupportsWriterState<InputT,​BufferedRequestState<RequestEntryT>>, Sink<InputT>
    A generic sink for destinations that provide an async client to persist data.

    The design of the sink focuses on extensibility and a broad support of destinations. The core of the sink is kept generic and free of any connector specific dependencies. The sink is designed to participate in checkpointing to provide at-least once semantics, but it is limited to destinations that provide a client that supports async requests.

    Limitations:

    • The sink is designed for destinations that provide an async client. Destinations that cannot ingest events in an async fashion cannot be supported by the sink.
    • The sink usually persist InputTs in the order they are added to the sink, but reorderings may occur, eg, when RequestEntryTs need to be retried.
    • We are not considering support for exactly-once semantics at this point.
    See Also:
    Serialized Form

    • Constructor Detail

      • AsyncSinkBase

        protected AsyncSinkBase​(ElementConverter<InputT,​RequestEntryT> elementConverter,
                        int maxBatchSize,
                        int maxInFlightRequests,
                        int maxBufferedRequests,
                        long maxBatchSizeInBytes,
                        long maxTimeInBufferMS,
                        long maxRecordSizeInBytes)

      • AsyncSinkBase

        protected AsyncSinkBase​(ElementConverter<InputT,​RequestEntryT> elementConverter,
                        int maxBatchSize,
                        int maxInFlightRequests,
                        int maxBufferedRequests,
                        long maxBatchSizeInBytes,
                        long maxTimeInBufferMS,
                        long maxRecordSizeInBytes,
                        long requestTimeoutMS,
                        boolean failOnTimeout)

    • Method Detail

      • getMaxBatchSize

        protected int getMaxBatchSize()

      • getMaxInFlightRequests

        protected int getMaxInFlightRequests()

      • getMaxBufferedRequests

        protected int getMaxBufferedRequests()

      • getMaxBatchSizeInBytes

        protected long getMaxBatchSizeInBytes()

      • getMaxTimeInBufferMS

        protected long getMaxTimeInBufferMS()

      • getMaxRecordSizeInBytes

        protected long getMaxRecordSizeInBytes()

      • getRequestTimeoutMS

        protected long getRequestTimeoutMS()

      • getFailOnTimeout

        protected boolean getFailOnTimeout()