org.apache.flink.streaming.connectors.elasticsearch

Interface ElasticsearchApiCallBridge<C extends AutoCloseable>

Type Parameters:

C - The Elasticsearch client, that implements AutoCloseable.

All Superinterfaces:

Serializable

All Known Implementing Classes:

Elasticsearch1ApiCallBridge, Elasticsearch2ApiCallBridge, Elasticsearch5ApiCallBridge, Elasticsearch6ApiCallBridge

@Internal
public interface ElasticsearchApiCallBridge<C extends AutoCloseable>
extends Serializable

An ElasticsearchApiCallBridge is used to bridge incompatible Elasticsearch Java API calls across different versions. This includes calls to create Elasticsearch clients, handle failed item responses, etc. Any incompatible Elasticsearch Java APIs should be bridged using this interface.

Implementations are allowed to be stateful. For example, for Elasticsearch 1.x, since connecting via an embedded node is allowed, the call bridge will hold reference to the created embedded node. Each instance of the sink will hold exactly one instance of the call bridge, and state cleanup is performed when the sink is closed.

Method Summary

Modifier and Type	Method and Description
default void	cleanup() Perform any necessary state cleanup.
void	configureBulkProcessorBackoff(org.elasticsearch.action.bulk.BulkProcessor.Builder builder, ElasticsearchSinkBase.BulkFlushBackoffPolicy flushBackoffPolicy) Set backoff-related configurations on the provided BulkProcessor.Builder.
org.elasticsearch.action.bulk.BulkProcessor.Builder	createBulkProcessorBuilder(C client, org.elasticsearch.action.bulk.BulkProcessor.Listener listener) Creates a BulkProcessor.Builder for creating the bulk processor.
default RequestIndexer	createBulkProcessorIndexer(org.elasticsearch.action.bulk.BulkProcessor bulkProcessor, boolean flushOnCheckpoint, AtomicLong numPendingRequestsRef) Creates a RequestIndexer that is able to work with BulkProcessor binary compatible.
C	createClient(Map<String,String> clientConfig) Creates an Elasticsearch client implementing AutoCloseable.
Throwable	extractFailureCauseFromBulkItemResponse(org.elasticsearch.action.bulk.BulkItemResponse bulkItemResponse) Extracts the cause of failure of a bulk item action.

Method Detail

createClient

C createClient(Map<String,String> clientConfig)
        throws IOException

Creates an Elasticsearch client implementing AutoCloseable.

Parameters:

clientConfig - The configuration to use when constructing the client.

Returns:

The created client.

Throws:

IOException

createBulkProcessorBuilder

org.elasticsearch.action.bulk.BulkProcessor.Builder createBulkProcessorBuilder(C client,
                                                                               org.elasticsearch.action.bulk.BulkProcessor.Listener listener)

Creates a BulkProcessor.Builder for creating the bulk processor.

Parameters:

client - the Elasticsearch client.

listener - the bulk processor listender.

Returns:

the bulk processor builder.

extractFailureCauseFromBulkItemResponse

@Nullable
Throwable extractFailureCauseFromBulkItemResponse(org.elasticsearch.action.bulk.BulkItemResponse bulkItemResponse)

Extracts the cause of failure of a bulk item action.

Parameters:

bulkItemResponse - the bulk item response to extract cause of failure

Returns:

the extracted Throwable from the response (null is the response is successful).

configureBulkProcessorBackoff

void configureBulkProcessorBackoff(org.elasticsearch.action.bulk.BulkProcessor.Builder builder,
                                   @Nullable
                                   ElasticsearchSinkBase.BulkFlushBackoffPolicy flushBackoffPolicy)

Set backoff-related configurations on the provided BulkProcessor.Builder. The builder will be later on used to instantiate the actual BulkProcessor.

Parameters:

builder - the BulkProcessor.Builder to configure.

flushBackoffPolicy - user-provided backoff retry settings (null if the user disabled backoff retries).

createBulkProcessorIndexer

default RequestIndexer createBulkProcessorIndexer(org.elasticsearch.action.bulk.BulkProcessor bulkProcessor,
                                                  boolean flushOnCheckpoint,
                                                  AtomicLong numPendingRequestsRef)

Creates a RequestIndexer that is able to work with BulkProcessor binary compatible.

cleanup

default void cleanup()

Perform any necessary state cleanup.