@Experimental public interface PartitionCommitPolicy
The implemented commit method needs to be idempotent because the same partition may be committed multiple times.
Default implementations: See MetastoreCommitPolicy
. See SuccessFileCommitPolicy
.
Further more, you can implement your own policy, like: - RPC to notify downstream applications. - Trigger hive to analysis partition for generating statistics. ...
Modifier and Type | Interface and Description |
---|---|
static interface |
PartitionCommitPolicy.Context
Context of policy, including table information and partition information.
|
Modifier and Type | Field and Description |
---|---|
static String |
CUSTOM |
static String |
METASTORE |
static String |
SUCCESS_FILE |
Modifier and Type | Method and Description |
---|---|
void |
commit(PartitionCommitPolicy.Context context)
Commit a partition.
|
static List<PartitionCommitPolicy> |
createPolicyChain(ClassLoader cl,
String policyKind,
String customClass,
String successFileName,
java.util.function.Supplier<FileSystem> fsSupplier)
Create a policy chain from config.
|
static void |
validatePolicyChain(boolean isEmptyMetastore,
String policyKind)
Validate commit policy.
|
static final String METASTORE
static final String SUCCESS_FILE
static final String CUSTOM
void commit(PartitionCommitPolicy.Context context) throws Exception
Exception
static List<PartitionCommitPolicy> createPolicyChain(ClassLoader cl, String policyKind, String customClass, String successFileName, java.util.function.Supplier<FileSystem> fsSupplier)
static void validatePolicyChain(boolean isEmptyMetastore, String policyKind)
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.