org.apache.flink.table.connector.sink

Interface DynamicTableSink

All Known Implementing Classes:

AsyncDynamicTableSink, CollectDynamicSink, FileSystemTableSink, HiveTableSink

@PublicEvolving
public interface DynamicTableSink

Sink of a dynamic table to an external storage system.

Dynamic tables are the core concept of Flink's Table & SQL API for processing both bounded and unbounded data in a unified fashion. By definition, a dynamic table can change over time.

When writing a dynamic table, the content can always be considered as a changelog (finite or infinite) for which all changes are written out continuously until the changelog is exhausted. The given ChangelogMode indicates the set of changes that the sink accepts during runtime.

For regular batch scenarios, the sink can solely accept insert-only rows and write out bounded streams.

For regular streaming scenarios, the sink can solely accept insert-only rows and can write out unbounded streams.

For change data capture (CDC) scenarios, the sink can write out bounded or unbounded streams with insert, update, and delete rows. See also RowKind.

Instances of DynamicTableSink can be seen as factories that eventually produce concrete runtime implementation for writing the actual data.

Depending on the optionally declared abilities, the planner might apply changes to an instance and thus mutate the produced runtime implementation.

A DynamicTableSink can implement the following abilities:

• SupportsBucketing
• SupportsPartitioning
• SupportsOverwrite
• SupportsWritingMetadata

In the last step, the planner will call getSinkRuntimeProvider(Context) for obtaining a provider of runtime implementation.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	DynamicTableSink.Context Context for creating runtime implementation via a DynamicTableSink.SinkRuntimeProvider.
static interface	DynamicTableSink.DataStructureConverter Converter for mapping between Flink's internal data structures and objects specified by the given DataType that can be passed into a runtime implementation.
static interface	DynamicTableSink.SinkRuntimeProvider Provides actual runtime implementation for writing the data.

Method Summary

Modifier and Type	Method and Description
String	asSummaryString() Returns a string that summarizes this sink for printing to a console or log.
DynamicTableSink	copy() Creates a copy of this instance during planning.
ChangelogMode	getChangelogMode(ChangelogMode requestedMode) Returns the set of changes that the sink accepts during runtime.
DynamicTableSink.SinkRuntimeProvider	getSinkRuntimeProvider(DynamicTableSink.Context context) Returns a provider of runtime implementation for writing the data.

Method Detail

getChangelogMode

ChangelogMode getChangelogMode(ChangelogMode requestedMode)

Returns the set of changes that the sink accepts during runtime.

The planner can make suggestions but the sink has the final decision what it requires. If the planner does not support this mode, it will throw an error. For example, the sink can return that it only supports ChangelogMode.insertOnly().

Parameters:

requestedMode - expected set of changes by the current plan

getSinkRuntimeProvider

DynamicTableSink.SinkRuntimeProvider getSinkRuntimeProvider(DynamicTableSink.Context context)

Returns a provider of runtime implementation for writing the data.

There might exist different interfaces for runtime implementation which is why DynamicTableSink.SinkRuntimeProvider serves as the base interface. Concrete DynamicTableSink.SinkRuntimeProvider interfaces might be located in other Flink modules.

Independent of the provider interface, the table runtime expects that a sink implementation accepts internal data structures (see RowData for more information).

The given DynamicTableSink.Context offers utilities by the planner for creating runtime implementation with minimal dependencies to internal data structures.

SinkV2Provider is the recommended core interface. SinkProvider, SinkFunctionProvider in flink-table-api-java-bridge and OutputFormatProvider are available for backwards compatibility.

See Also:

SinkV2Provider

copy

DynamicTableSink copy()

Creates a copy of this instance during planning. The copy should be a deep copy of all mutable members.

asSummaryString

String asSummaryString()

Returns a string that summarizes this sink for printing to a console or log.