Interface PartitionableTableSink
-
@Deprecated @Internal public interface PartitionableTableSink
Deprecated.This interface will not be supported in the new sink design aroundDynamicTableSink
. UseSupportsPartitioning
instead. See FLIP-95 for more information.An interface for partitionableTableSink
. A partitionable sink can writes query results to partitions.Partition columns are defined via catalog table.
For example, a partitioned table named
my_table
with a table schema[a INT, b VARCHAR, c DOUBLE, dt VARCHAR, country VARCHAR]
is partitioned on columnsdt, country
. Thendt
is the first partition column, andcountry
is the secondary partition column.We can insert data into table partitions using INSERT INTO PARTITION syntax, for example:
INSERT INTO my_table PARTITION (dt='2019-06-20', country='bar') select a, b, c from my_view
When all the partition columns are set a value in PARTITION clause, it is inserting into a static partition. It will writes the query result into a static partition, i.e.
dt='2019-06-20', country='bar'
. The user specified static partitions will be told to the sink viasetStaticPartition(Map)
.The INSERT INTO PARTITION syntax also supports dynamic partition inserts.
INSERT INTO my_table PARTITION (dt='2019-06-20') select a, b, c, country from another_view
When partial partition columns (prefix part of all partition columns) are set a value in PARTITION clause, it is writing the query result into a dynamic partition. In the above example, the static partition part is
dt='2019-06-20'
which will be told to the sink viasetStaticPartition(Map)
. And thecountry
is the dynamic partition which will be get from each record.
-
-
Method Summary
All Methods Instance Methods Abstract Methods Default Methods Deprecated Methods Modifier and Type Method Description default boolean
configurePartitionGrouping(boolean supportsGrouping)
Deprecated.If returns true, sink can trust all records will definitely be grouped by partition fields before consumed by theTableSink
, i.e. the sink will receive all elements of one partition and then all elements of another partition, elements of different partitions will not be mixed.void
setStaticPartition(Map<String,String> partitions)
Deprecated.Sets the static partition into theTableSink
.
-
-
-
Method Detail
-
setStaticPartition
void setStaticPartition(Map<String,String> partitions)
Deprecated.Sets the static partition into theTableSink
. The static partition may be partial of all partition columns. See the class Javadoc for more details.The static partition is represented as a
Map<String, String>
which maps from partition field name to partition value. The partition values are all encoded as strings, i.e. encoded using String.valueOf(...). For example, if we have a static partitionf0=1024, f1="foo", f2="bar"
. f0 is an integer type, f1 and f2 are string types. They will all be encoded as strings: "1024", "foo", "bar". And can be decoded to original literals based on the field types.- Parameters:
partitions
- user specified static partition
-
configurePartitionGrouping
default boolean configurePartitionGrouping(boolean supportsGrouping)
Deprecated.If returns true, sink can trust all records will definitely be grouped by partition fields before consumed by theTableSink
, i.e. the sink will receive all elements of one partition and then all elements of another partition, elements of different partitions will not be mixed. For some sinks, this can be used to reduce number of the partition writers to improve writing performance.This method is used to configure the behavior of input whether to be grouped by partition, if true, at the same time the sink should also configure itself, i.e. set an internal field that changes the writing behavior (writing one partition at a time).
- Parameters:
supportsGrouping
- whether the execution mode supports grouping, e.g. grouping (usually use sort to implement) is only supported in batch mode, not supported in streaming mode.- Returns:
- whether data need to be grouped by partition before consumed by the sink. Default is
false. If
supportsGrouping
is false, it should never return true (requires grouping), otherwise it will fail.
-
-