public class HiveTableSink extends Object implements DynamicTableSink, SupportsPartitioning, SupportsOverwrite
DynamicTableSink.Context, DynamicTableSink.DataStructureConverter, DynamicTableSink.SinkRuntimeProvider
Modifier and Type | Field and Description |
---|---|
static String |
BATCH_COMPACT_WRITER_OP_NAME |
Constructor and Description |
---|
HiveTableSink(ReadableConfig flinkConf,
org.apache.hadoop.mapred.JobConf jobConf,
ObjectIdentifier identifier,
ResolvedCatalogTable table,
Integer configuredSinkParallelism) |
Modifier and Type | Method and Description |
---|---|
void |
applyOverwrite(boolean overwrite)
Provides whether existing data should be overwritten or not.
|
void |
applyStaticPartition(Map<String,String> partition)
Provides the static part of a partition.
|
String |
asSummaryString()
Returns a string that summarizes this sink for printing to a console or log.
|
DynamicTableSink |
copy()
Creates a copy of this instance during planning.
|
ChangelogMode |
getChangelogMode(ChangelogMode requestedMode)
Returns the set of changes that the sink accepts during runtime.
|
org.apache.hadoop.mapred.JobConf |
getJobConf() |
DynamicTableSink.SinkRuntimeProvider |
getSinkRuntimeProvider(DynamicTableSink.Context context)
Returns a provider of runtime implementation for writing the data.
|
boolean |
requiresPartitionGrouping(boolean supportsGrouping)
Returns whether data needs to be grouped by partition before it is consumed by the sink.
|
public static final String BATCH_COMPACT_WRITER_OP_NAME
public HiveTableSink(ReadableConfig flinkConf, org.apache.hadoop.mapred.JobConf jobConf, ObjectIdentifier identifier, ResolvedCatalogTable table, @Nullable Integer configuredSinkParallelism)
public DynamicTableSink.SinkRuntimeProvider getSinkRuntimeProvider(DynamicTableSink.Context context)
DynamicTableSink
There might exist different interfaces for runtime implementation which is why DynamicTableSink.SinkRuntimeProvider
serves as the base interface. Concrete DynamicTableSink.SinkRuntimeProvider
interfaces might be located in other Flink modules.
Independent of the provider interface, the table runtime expects that a sink
implementation accepts internal data structures (see RowData
for more information).
The given DynamicTableSink.Context
offers utilities by the planner for creating runtime
implementation with minimal dependencies to internal data structures.
SinkV2Provider
is the recommended core interface. SinkProvider
, SinkFunctionProvider
in flink-table-api-java-bridge
and OutputFormatProvider
are available for backwards compatibility.
getSinkRuntimeProvider
in interface DynamicTableSink
SinkV2Provider
public boolean requiresPartitionGrouping(boolean supportsGrouping)
SupportsPartitioning
If this method returns true, the sink can expect that all records will be grouped by the partition keys before consumed by the sink. In other words: The sink will receive all elements of one partition and then all elements of another partition. Elements of different partitions will not be mixed. For some sinks, this can be used to reduce the number of partition writers and improve writing performance by writing one partition at a time.
The given argument indicates whether the current execution mode supports grouping or not. For example, depending on the execution mode a sorting operation might not be available during runtime.
requiresPartitionGrouping
in interface SupportsPartitioning
supportsGrouping
- whether the current execution mode supports groupingsupportsGrouping
is false, it should never return true, otherwise the planner will fail.public void applyStaticPartition(Map<String,String> partition)
SupportsPartitioning
A single partition maps each partition key to a partition value. Depending on the user-defined statement, the partition might not include all partition keys.
See the documentation of SupportsPartitioning
for more information.
applyStaticPartition
in interface SupportsPartitioning
partition
- user-defined (possibly partial) static partitionpublic void applyOverwrite(boolean overwrite)
SupportsOverwrite
applyOverwrite
in interface SupportsOverwrite
public ChangelogMode getChangelogMode(ChangelogMode requestedMode)
DynamicTableSink
The planner can make suggestions but the sink has the final decision what it requires. If
the planner does not support this mode, it will throw an error. For example, the sink can
return that it only supports ChangelogMode.insertOnly()
.
getChangelogMode
in interface DynamicTableSink
requestedMode
- expected set of changes by the current planpublic DynamicTableSink copy()
DynamicTableSink
copy
in interface DynamicTableSink
public String asSummaryString()
DynamicTableSink
asSummaryString
in interface DynamicTableSink
@VisibleForTesting public org.apache.hadoop.mapred.JobConf getJobConf()
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.