@Experimental public interface PartitionableTableSource
TableSource
.
A PartitionableTableSource
can exclude partitions from reading, which includes
skipping the metadata. This is especially useful when there are thousands of partitions in a
table.
A partition is represented as a Map<String, String>
which maps from partition field
name to partition value. Since the map is NOT ordered, the correct order of partition fields
should be obtained via partition keys of catalog table.
Modifier and Type | Method and Description |
---|---|
TableSource |
applyPartitionPruning(List<Map<String,String>> remainingPartitions)
Applies the remaining partitions to the table source.
|
List<Map<String,String>> |
getPartitions()
Returns all the partitions of this
PartitionableTableSource . |
List<Map<String,String>> getPartitions()
PartitionableTableSource
.TableSource applyPartitionPruning(List<Map<String,String>> remainingPartitions)
remainingPartitions
is the
remaining partitions of getPartitions()
after partition pruning applied.
After trying to apply partition pruning, we should return a new TableSource
instance which holds all pruned-partitions.
remainingPartitions
- Remaining partitions after partition pruning applied.TableSource
holds all pruned-partitions.Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.