org.apache.flink.table.catalog.hive

Class HiveCatalog

java.lang.Object

org.apache.flink.table.catalog.AbstractCatalog

org.apache.flink.table.catalog.hive.HiveCatalog

All Implemented Interfaces:

Catalog

public class HiveCatalog
extends AbstractCatalog

A catalog implementation for Hive.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_DB
static String	HIVE_SITE_FILE

Constructor Summary

Constructors

Constructor and Description
HiveCatalog(String catalogName, String defaultDatabase, HiveConf hiveConf, String hiveVersion)
HiveCatalog(String catalogName, String defaultDatabase, HiveConf hiveConf, String hiveVersion, boolean allowEmbedded)
HiveCatalog(String catalogName, String defaultDatabase, String hiveConfDir)
HiveCatalog(String catalogName, String defaultDatabase, String hiveConfDir, String hiveVersion)
HiveCatalog(String catalogName, String defaultDatabase, String hiveConfDir, String hadoopConfDir, String hiveVersion)

Method Summary

Modifier and Type	Method and Description
void	alterDatabase(String databaseName, CatalogDatabase newDatabase, boolean ignoreIfNotExists) Modify an existing database.
void	alterFunction(ObjectPath functionPath, CatalogFunction newFunction, boolean ignoreIfNotExists) Modify an existing function.
void	alterPartition(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, CatalogPartition newPartition, boolean ignoreIfNotExists) Alter a partition.
void	alterPartitionColumnStatistics(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, CatalogColumnStatistics columnStatistics, boolean ignoreIfNotExists) Update the column statistics of a table partition.
void	alterPartitionStatistics(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, CatalogTableStatistics partitionStatistics, boolean ignoreIfNotExists) Update the statistics of a table partition.
void	alterTable(ObjectPath tablePath, CatalogBaseTable newCatalogTable, boolean ignoreIfNotExists) Modifies an existing table or view.
void	alterTableColumnStatistics(ObjectPath tablePath, CatalogColumnStatistics columnStatistics, boolean ignoreIfNotExists) Update the column statistics of a table.
void	alterTableStatistics(ObjectPath tablePath, CatalogTableStatistics tableStatistics, boolean ignoreIfNotExists) Update the statistics of a table.
List<CatalogColumnStatistics>	bulkGetPartitionColumnStatistics(ObjectPath tablePath, List<CatalogPartitionSpec> partitionSpecs) Get a list of column statistics for given partitions.
List<CatalogTableStatistics>	bulkGetPartitionStatistics(ObjectPath tablePath, List<CatalogPartitionSpec> partitionSpecs) Get a list of statistics of given partitions.
void	close() Close the catalog when it is no longer needed and release any resource that it might be holding.
void	createDatabase(String databaseName, CatalogDatabase database, boolean ignoreIfExists) Create a database.
void	createFunction(ObjectPath functionPath, CatalogFunction function, boolean ignoreIfExists) Create a function.
static HiveConf	createHiveConf(String hiveConfDir, String hadoopConfDir)
void	createPartition(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, CatalogPartition partition, boolean ignoreIfExists) Create a partition.
void	createTable(ObjectPath tablePath, CatalogBaseTable table, boolean ignoreIfExists) Creates a new table or view.
boolean	databaseExists(String databaseName) Check if a database exists in this catalog.
void	dropDatabase(String name, boolean ignoreIfNotExists, boolean cascade) Drop a database.
void	dropFunction(ObjectPath functionPath, boolean ignoreIfNotExists) Drop a function.
void	dropPartition(ObjectPath tablePath, CatalogPartitionSpec partitionSpec, boolean ignoreIfNotExists) Drop a partition.
void	dropTable(ObjectPath tablePath, boolean ignoreIfNotExists) Drop a table or view.
boolean	functionExists(ObjectPath functionPath) Check whether a function exists or not.
CatalogDatabase	getDatabase(String databaseName) Get a database from this catalog.
Optional<Factory>	getFactory() Returns a factory for creating instances from catalog objects.
static List<String>	getFieldNames(List<org.apache.hadoop.hive.metastore.api.FieldSchema> fieldSchemas) Get field names from field schemas.
CatalogFunction	getFunction(ObjectPath functionPath) Get the function.
Optional<FunctionDefinitionFactory>	getFunctionDefinitionFactory() Get an optional FunctionDefinitionFactory instance that's responsible for instantiating function definitions.
HiveConf	getHiveConf()
org.apache.hadoop.hive.metastore.api.Database	getHiveDatabase(String databaseName)
org.apache.hadoop.hive.metastore.api.Partition	getHivePartition(org.apache.hadoop.hive.metastore.api.Table hiveTable, CatalogPartitionSpec partitionSpec)
org.apache.hadoop.hive.metastore.api.Table	getHiveTable(ObjectPath tablePath)
String	getHiveVersion()
CatalogPartition	getPartition(ObjectPath tablePath, CatalogPartitionSpec partitionSpec) Get a partition of the given table.
CatalogColumnStatistics	getPartitionColumnStatistics(ObjectPath tablePath, CatalogPartitionSpec partitionSpec) Get the column statistics of a partition.
CatalogTableStatistics	getPartitionStatistics(ObjectPath tablePath, CatalogPartitionSpec partitionSpec) Get the statistics of a partition.
CatalogBaseTable	getTable(ObjectPath tablePath) Returns a CatalogTable or CatalogView identified by the given ObjectPath.
CatalogColumnStatistics	getTableColumnStatistics(ObjectPath tablePath) Get the column statistics of a table.
Optional<TableFactory>	getTableFactory() Get an optional TableFactory instance that's responsible for generating table-related instances stored in this catalog, instances such as source/sink.
CatalogTableStatistics	getTableStatistics(ObjectPath tablePath) Get the statistics of a table.
static boolean	isEmbeddedMetastore(HiveConf hiveConf)
static boolean	isHiveTable(Map<String,String> properties)
static boolean	isHiveTable(org.apache.hadoop.hive.metastore.api.Table table)
List<String>	listDatabases() Get the names of all databases in this catalog.
List<String>	listFunctions(String databaseName) List the names of all functions in the given database.
List<CatalogPartitionSpec>	listPartitions(ObjectPath tablePath) Get CatalogPartitionSpec of all partitions of the table.
List<CatalogPartitionSpec>	listPartitions(ObjectPath tablePath, CatalogPartitionSpec partitionSpec) Get CatalogPartitionSpec of all partitions that is under the given CatalogPartitionSpec in the table.
List<CatalogPartitionSpec>	listPartitionsByFilter(ObjectPath tablePath, List<Expression> expressions) Get CatalogPartitionSpec of partitions by expression filters in the table.
List<String>	listTables(String databaseName) Get names of all tables and views under this database.
List<String>	listViews(String databaseName) Get names of all views under this database.
void	loadPartition(org.apache.hadoop.fs.Path loadPath, ObjectPath tablePath, Map<String,String> partSpec, boolean isOverwrite, boolean isSrcLocal)
void	loadTable(org.apache.hadoop.fs.Path loadPath, ObjectPath tablePath, boolean isOverwrite, boolean isSrcLocal)
void	open() Open the catalog.
boolean	partitionExists(ObjectPath tablePath, CatalogPartitionSpec partitionSpec) Check whether a partition exists or not.
void	renameTable(ObjectPath tablePath, String newTableName, boolean ignoreIfNotExists) Rename an existing table or view.
boolean	supportsManagedTable() If true, tables which do not specify a connector will be translated to managed tables.
boolean	tableExists(ObjectPath tablePath) Check if a table or view exists in this catalog.

Methods inherited from class org.apache.flink.table.catalog.AbstractCatalog

getDefaultDatabase, getName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.catalog.Catalog

alterTable, dropDatabase, getProcedure, getTable, listProcedures

Field Detail

DEFAULT_DB

public static final String DEFAULT_DB

See Also:

Constant Field Values

HIVE_SITE_FILE

public static final String HIVE_SITE_FILE

See Also:

Constant Field Values

Constructor Detail

HiveCatalog

public HiveCatalog(String catalogName,
                   @Nullable
                   String defaultDatabase,
                   @Nullable
                   String hiveConfDir)

HiveCatalog

public HiveCatalog(String catalogName,
                   @Nullable
                   String defaultDatabase,
                   @Nullable
                   String hiveConfDir,
                   String hiveVersion)

HiveCatalog

public HiveCatalog(String catalogName,
                   @Nullable
                   String defaultDatabase,
                   @Nullable
                   String hiveConfDir,
                   @Nullable
                   String hadoopConfDir,
                   @Nullable
                   String hiveVersion)

HiveCatalog

public HiveCatalog(String catalogName,
                   @Nullable
                   String defaultDatabase,
                   @Nullable
                   HiveConf hiveConf,
                   @Nullable
                   String hiveVersion)

HiveCatalog

public HiveCatalog(String catalogName,
                   @Nullable
                   String defaultDatabase,
                   @Nullable
                   HiveConf hiveConf,
                   String hiveVersion,
                   boolean allowEmbedded)

Method Detail

createHiveConf

public static HiveConf createHiveConf(@Nullable
                                      String hiveConfDir,
                                      @Nullable
                                      String hadoopConfDir)

getHiveConf

public HiveConf getHiveConf()

getHiveVersion

@Internal
public String getHiveVersion()

open

public void open()
          throws CatalogException

Description copied from interface: Catalog

Open the catalog. Used for any required preparation in initialization phase.

Throws:

CatalogException - in case of any runtime exception

close

public void close()
           throws CatalogException

Description copied from interface: Catalog

Close the catalog when it is no longer needed and release any resource that it might be holding.

Throws:

CatalogException - in case of any runtime exception

getFactory

public Optional<Factory> getFactory()

Description copied from interface: Catalog

Returns a factory for creating instances from catalog objects.

This method enables bypassing the discovery process. Implementers can directly pass internal catalog-specific objects to their own factory. For example, a custom CatalogTable can be processed by a custom DynamicTableFactory.

If this catalog support to create materialized table, you should also override this method to provide DynamicTableFactory which help planner to find DynamicTableSource and DynamicTableSink correctly during compile optimization phase. If you don't override this method, you must specify the physical connector identifier that this catalog represents storage when create materialized table. Otherwise, the planner can't find the DynamicTableFactory.

Because all factories are interfaces, the returned Factory instance can implement multiple supported extension points. An instanceof check is performed by the caller that checks whether a required factory is implemented; otherwise the discovery process is used.

getTableFactory

public Optional<TableFactory> getTableFactory()

Description copied from interface: Catalog

Get an optional TableFactory instance that's responsible for generating table-related instances stored in this catalog, instances such as source/sink.

Returns:

an optional TableFactory instance

getFunctionDefinitionFactory

public Optional<FunctionDefinitionFactory> getFunctionDefinitionFactory()

Description copied from interface: Catalog

Get an optional FunctionDefinitionFactory instance that's responsible for instantiating function definitions.

Returns:

an optional FunctionDefinitionFactory instance

getDatabase

public CatalogDatabase getDatabase(String databaseName)
                            throws DatabaseNotExistException,
                                   CatalogException

Description copied from interface: Catalog

Get a database from this catalog.

Parameters:

databaseName - Name of the database

Returns:

The requested database

Throws:

DatabaseNotExistException - if the database does not exist

CatalogException - in case of any runtime exception

createDatabase

public void createDatabase(String databaseName,
                           CatalogDatabase database,
                           boolean ignoreIfExists)
                    throws DatabaseAlreadyExistException,
                           CatalogException

Description copied from interface: Catalog

Create a database.

Parameters:

databaseName - Name of the database to be created

database - The database definition

ignoreIfExists - Flag to specify behavior when a database with the given name already exists: if set to false, throw a DatabaseAlreadyExistException, if set to true, do nothing.

Throws:

DatabaseAlreadyExistException - if the given database already exists and ignoreIfExists is false

CatalogException - in case of any runtime exception

alterDatabase

public void alterDatabase(String databaseName,
                          CatalogDatabase newDatabase,
                          boolean ignoreIfNotExists)
                   throws DatabaseNotExistException,
                          CatalogException

Description copied from interface: Catalog

Modify an existing database.

Parameters:

databaseName - Name of the database to be modified

newDatabase - The new database definition

ignoreIfNotExists - Flag to specify behavior when the given database does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

DatabaseNotExistException - if the given database does not exist

CatalogException - in case of any runtime exception

listDatabases

public List<String> listDatabases()
                           throws CatalogException

Description copied from interface: Catalog

Get the names of all databases in this catalog.

Returns:

a list of the names of all databases

Throws:

CatalogException - in case of any runtime exception

databaseExists

public boolean databaseExists(String databaseName)
                       throws CatalogException

Description copied from interface: Catalog

Check if a database exists in this catalog.

Parameters:

databaseName - Name of the database

Returns:

true if the given database exists in the catalog false otherwise

Throws:

CatalogException - in case of any runtime exception

dropDatabase

public void dropDatabase(String name,
                         boolean ignoreIfNotExists,
                         boolean cascade)
                  throws DatabaseNotExistException,
                         DatabaseNotEmptyException,
                         CatalogException

Description copied from interface: Catalog

Drop a database.

Parameters:

name - Name of the database to be dropped.

ignoreIfNotExists - Flag to specify behavior when the database does not exist: if set to false, throw an exception, if set to true, do nothing.

cascade - Flag to specify behavior when the database contains table or function: if set to true, delete all tables and functions in the database and then delete the database, if set to false, throw an exception.

Throws:

DatabaseNotExistException - if the given database does not exist

DatabaseNotEmptyException - if the given database is not empty and isRestrict is true

CatalogException - in case of any runtime exception

getHiveDatabase

@VisibleForTesting
public org.apache.hadoop.hive.metastore.api.Database getHiveDatabase(String databaseName)
                                                                                 throws DatabaseNotExistException

Throws:

DatabaseNotExistException

getTable

public CatalogBaseTable getTable(ObjectPath tablePath)
                          throws TableNotExistException,
                                 CatalogException

Description copied from interface: Catalog

Returns a CatalogTable or CatalogView identified by the given ObjectPath. The framework will resolve the metadata objects when necessary.

Parameters:

tablePath - Path of the table or view

Returns:

The requested table or view

Throws:

TableNotExistException - if the target does not exist

CatalogException - in case of any runtime exception

createTable

public void createTable(ObjectPath tablePath,
                        CatalogBaseTable table,
                        boolean ignoreIfExists)
                 throws TableAlreadyExistException,
                        DatabaseNotExistException,
                        CatalogException

Description copied from interface: Catalog

Creates a new table or view.

The framework will make sure to call this method with fully validated ResolvedCatalogTable or ResolvedCatalogView. Those instances are easy to serialize for a durable catalog implementation.

Parameters:

tablePath - path of the table or view to be created

table - the table definition

ignoreIfExists - flag to specify behavior when a table or view already exists at the given path: if set to false, it throws a TableAlreadyExistException, if set to true, do nothing.

Throws:

TableAlreadyExistException - if table already exists and ignoreIfExists is false

DatabaseNotExistException - if the database in tablePath doesn't exist

CatalogException - in case of any runtime exception

renameTable

public void renameTable(ObjectPath tablePath,
                        String newTableName,
                        boolean ignoreIfNotExists)
                 throws TableNotExistException,
                        TableAlreadyExistException,
                        CatalogException

Description copied from interface: Catalog

Rename an existing table or view.

Parameters:

tablePath - Path of the table or view to be renamed

newTableName - the new name of the table or view

ignoreIfNotExists - Flag to specify behavior when the table or view does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

TableNotExistException - if the table does not exist

CatalogException - in case of any runtime exception

TableAlreadyExistException

alterTable

public void alterTable(ObjectPath tablePath,
                       CatalogBaseTable newCatalogTable,
                       boolean ignoreIfNotExists)
                throws TableNotExistException,
                       CatalogException

Description copied from interface: Catalog

Modifies an existing table or view. Note that the new and old CatalogBaseTable must be of the same kind. For example, this doesn't allow altering a regular table to partitioned table, or altering a view to a table, and vice versa.

The framework will make sure to call this method with fully validated ResolvedCatalogTable or ResolvedCatalogView. Those instances are easy to serialize for a durable catalog implementation.

Parameters:

tablePath - path of the table or view to be modified

newCatalogTable - the new table definition

ignoreIfNotExists - flag to specify behavior when the table or view does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

TableNotExistException - if the table does not exist

CatalogException - in case of any runtime exception

dropTable

public void dropTable(ObjectPath tablePath,
                      boolean ignoreIfNotExists)
               throws TableNotExistException,
                      CatalogException

Description copied from interface: Catalog

Drop a table or view.

Parameters:

tablePath - Path of the table or view to be dropped

ignoreIfNotExists - Flag to specify behavior when the table or view does not exist: if set to false, throw an exception, if set to true, do nothing.

Throws:

TableNotExistException - if the table or view does not exist

CatalogException - in case of any runtime exception

listTables

public List<String> listTables(String databaseName)
                        throws DatabaseNotExistException,
                               CatalogException

Description copied from interface: Catalog

Get names of all tables and views under this database. An empty list is returned if none exists.

Returns:

a list of the names of all tables and views in this database

Throws:

DatabaseNotExistException - if the database does not exist

CatalogException - in case of any runtime exception

listViews

public List<String> listViews(String databaseName)
                       throws DatabaseNotExistException,
                              CatalogException

Description copied from interface: Catalog

Get names of all views under this database. An empty list is returned if none exists.

Parameters:

databaseName - the name of the given database

Returns:

a list of the names of all views in the given database

Throws:

DatabaseNotExistException - if the database does not exist

CatalogException - in case of any runtime exception

tableExists

public boolean tableExists(ObjectPath tablePath)
                    throws CatalogException

Description copied from interface: Catalog

Check if a table or view exists in this catalog.

Parameters:

tablePath - Path of the table or view

Returns:

true if the given table exists in the catalog false otherwise

Throws:

CatalogException - in case of any runtime exception

getHiveTable

@Internal
public org.apache.hadoop.hive.metastore.api.Table getHiveTable(ObjectPath tablePath)
                                                                  throws TableNotExistException

Throws:

TableNotExistException

partitionExists

public boolean partitionExists(ObjectPath tablePath,
                               CatalogPartitionSpec partitionSpec)
                        throws CatalogException

Description copied from interface: Catalog

Check whether a partition exists or not.

Parameters:

tablePath - path of the table

partitionSpec - partition spec of the partition to check

Throws:

CatalogException - in case of any runtime exception

createPartition

public void createPartition(ObjectPath tablePath,
                            CatalogPartitionSpec partitionSpec,
                            CatalogPartition partition,
                            boolean ignoreIfExists)
                     throws TableNotExistException,
                            TableNotPartitionedException,
                            PartitionSpecInvalidException,
                            PartitionAlreadyExistsException,
                            CatalogException

Description copied from interface: Catalog

Create a partition.

Parameters:

tablePath - path of the table.

partitionSpec - partition spec of the partition

partition - the partition to add.

ignoreIfExists - flag to specify behavior if a partition with the given name already exists: if set to false, it throws a PartitionAlreadyExistsException, if set to true, nothing happens.

Throws:

TableNotExistException - thrown if the target table does not exist

TableNotPartitionedException - thrown if the target table is not partitioned

PartitionSpecInvalidException - thrown if the given partition spec is invalid

PartitionAlreadyExistsException - thrown if the target partition already exists

CatalogException - in case of any runtime exception

dropPartition

public void dropPartition(ObjectPath tablePath,
                          CatalogPartitionSpec partitionSpec,
                          boolean ignoreIfNotExists)
                   throws PartitionNotExistException,
                          CatalogException

Description copied from interface: Catalog

Drop a partition.

Parameters:

tablePath - path of the table.

partitionSpec - partition spec of the partition to drop

ignoreIfNotExists - flag to specify behavior if the partition does not exist: if set to false, throw an exception, if set to true, nothing happens.

Throws:

PartitionNotExistException - thrown if the target partition does not exist

CatalogException - in case of any runtime exception

listPartitions

public List<CatalogPartitionSpec> listPartitions(ObjectPath tablePath)
                                          throws TableNotExistException,
                                                 TableNotPartitionedException,
                                                 CatalogException

Description copied from interface: Catalog

Get CatalogPartitionSpec of all partitions of the table.

Parameters:

tablePath - path of the table

Returns:

a list of CatalogPartitionSpec of the table

Throws:

TableNotExistException - thrown if the table does not exist in the catalog

TableNotPartitionedException - thrown if the table is not partitioned

CatalogException - in case of any runtime exception

listPartitions

public List<CatalogPartitionSpec> listPartitions(ObjectPath tablePath,
                                                 CatalogPartitionSpec partitionSpec)
                                          throws TableNotExistException,
                                                 TableNotPartitionedException,
                                                 PartitionSpecInvalidException,
                                                 CatalogException

Description copied from interface: Catalog

Get CatalogPartitionSpec of all partitions that is under the given CatalogPartitionSpec in the table.

Parameters:

tablePath - path of the table

partitionSpec - the partition spec to list

Returns:

a list of CatalogPartitionSpec that is under the given CatalogPartitionSpec in the table

Throws:

TableNotExistException - thrown if the table does not exist in the catalog

TableNotPartitionedException - thrown if the table is not partitioned

CatalogException - in case of any runtime exception

PartitionSpecInvalidException

listPartitionsByFilter

public List<CatalogPartitionSpec> listPartitionsByFilter(ObjectPath tablePath,
                                                         List<Expression> expressions)
                                                  throws TableNotExistException,
                                                         TableNotPartitionedException,
                                                         CatalogException

Description copied from interface: Catalog

Get CatalogPartitionSpec of partitions by expression filters in the table.

NOTE: For FieldReferenceExpression, the field index is based on schema of this table instead of partition columns only.

The passed in predicates have been translated in conjunctive form.

If catalog does not support this interface at present, throw an UnsupportedOperationException directly. If the catalog does not have a valid filter, throw the UnsupportedOperationException directly. Planner will fallback to get all partitions and filter by itself.

Parameters:

tablePath - path of the table

expressions - filters to push down filter to catalog

Returns:

a list of CatalogPartitionSpec that is under the given CatalogPartitionSpec in the table

Throws:

TableNotExistException - thrown if the table does not exist in the catalog

TableNotPartitionedException - thrown if the table is not partitioned

CatalogException - in case of any runtime exception

getPartition

public CatalogPartition getPartition(ObjectPath tablePath,
                                     CatalogPartitionSpec partitionSpec)
                              throws PartitionNotExistException,
                                     CatalogException

Description copied from interface: Catalog

Get a partition of the given table. The given partition spec keys and values need to be matched exactly for a result.

Parameters:

tablePath - path of the table

partitionSpec - partition spec of partition to get

Returns:

the requested partition

Throws:

PartitionNotExistException - thrown if the partition doesn't exist

CatalogException - in case of any runtime exception

alterPartition

public void alterPartition(ObjectPath tablePath,
                           CatalogPartitionSpec partitionSpec,
                           CatalogPartition newPartition,
                           boolean ignoreIfNotExists)
                    throws PartitionNotExistException,
                           CatalogException

Description copied from interface: Catalog

Alter a partition.

Parameters:

tablePath - path of the table

partitionSpec - partition spec of the partition

newPartition - new partition to replace the old one

ignoreIfNotExists - flag to specify behavior if the partition does not exist: if set to false, throw an exception, if set to true, nothing happens.

Throws:

PartitionNotExistException - thrown if the target partition does not exist

CatalogException - in case of any runtime exception

getFieldNames

public static List<String> getFieldNames(List<org.apache.hadoop.hive.metastore.api.FieldSchema> fieldSchemas)

Get field names from field schemas.

getHivePartition

@VisibleForTesting
public org.apache.hadoop.hive.metastore.api.Partition getHivePartition(org.apache.hadoop.hive.metastore.api.Table hiveTable,
                                                                                          CatalogPartitionSpec partitionSpec)
                                                                                   throws PartitionSpecInvalidException,
                                                                                          org.apache.thrift.TException

Throws:

PartitionSpecInvalidException

org.apache.thrift.TException

createFunction

public void createFunction(ObjectPath functionPath,
                           CatalogFunction function,
                           boolean ignoreIfExists)
                    throws FunctionAlreadyExistException,
                           DatabaseNotExistException,
                           CatalogException

Description copied from interface: Catalog

Create a function. Function name should be handled in a case insensitive way.

Parameters:

functionPath - path of the function

function - the function to be created

ignoreIfExists - flag to specify behavior if a function with the given name already exists: if set to false, it throws a FunctionAlreadyExistException, if set to true, nothing happens.

Throws:

FunctionAlreadyExistException - if the function already exist

DatabaseNotExistException - if the given database does not exist

CatalogException - in case of any runtime exception

alterFunction

public void alterFunction(ObjectPath functionPath,
                          CatalogFunction newFunction,
                          boolean ignoreIfNotExists)
                   throws FunctionNotExistException,
                          CatalogException

Description copied from interface: Catalog

Modify an existing function. Function name should be handled in a case insensitive way.

Parameters:

functionPath - path of the function

newFunction - the function to be modified

ignoreIfNotExists - flag to specify behavior if the function does not exist: if set to false, throw an exception if set to true, nothing happens

Throws:

FunctionNotExistException - if the function does not exist

CatalogException - in case of any runtime exception

dropFunction

public void dropFunction(ObjectPath functionPath,
                         boolean ignoreIfNotExists)
                  throws FunctionNotExistException,
                         CatalogException

Description copied from interface: Catalog

Drop a function. Function name should be handled in a case insensitive way.

Parameters:

functionPath - path of the function to be dropped

ignoreIfNotExists - flag to specify behavior if the function does not exist: if set to false, throw an exception if set to true, nothing happens

Throws:

FunctionNotExistException - if the function does not exist

CatalogException - in case of any runtime exception

listFunctions

public List<String> listFunctions(String databaseName)
                           throws DatabaseNotExistException,
                                  CatalogException

Description copied from interface: Catalog

List the names of all functions in the given database. An empty list is returned if none is registered.

Parameters:

databaseName - name of the database.

Returns:

a list of the names of the functions in this database

Throws:

DatabaseNotExistException - if the database does not exist

CatalogException - in case of any runtime exception

getFunction

public CatalogFunction getFunction(ObjectPath functionPath)
                            throws FunctionNotExistException,
                                   CatalogException

Description copied from interface: Catalog

Get the function. Function name should be handled in a case insensitive way.

Parameters:

functionPath - path of the function

Returns:

the requested function

Throws:

FunctionNotExistException - if the function does not exist in the catalog

CatalogException - in case of any runtime exception

functionExists

public boolean functionExists(ObjectPath functionPath)
                       throws CatalogException

Description copied from interface: Catalog

Check whether a function exists or not. Function name should be handled in a case insensitive way.

Parameters:

functionPath - path of the function

Returns:

true if the function exists in the catalog false otherwise

Throws:

CatalogException - in case of any runtime exception

alterTableStatistics

public void alterTableStatistics(ObjectPath tablePath,
                                 CatalogTableStatistics tableStatistics,
                                 boolean ignoreIfNotExists)
                          throws TableNotExistException,
                                 CatalogException

Description copied from interface: Catalog

Update the statistics of a table.

Parameters:

tablePath - path of the table

tableStatistics - new statistics to update

ignoreIfNotExists - flag to specify behavior if the table does not exist: if set to false, throw an exception, if set to true, nothing happens.

Throws:

TableNotExistException - if the table does not exist in the catalog

CatalogException - in case of any runtime exception

alterTableColumnStatistics

public void alterTableColumnStatistics(ObjectPath tablePath,
                                       CatalogColumnStatistics columnStatistics,
                                       boolean ignoreIfNotExists)
                                throws TableNotExistException,
                                       CatalogException,
                                       TablePartitionedException

Description copied from interface: Catalog

Update the column statistics of a table.

Parameters:

tablePath - path of the table

columnStatistics - new column statistics to update

ignoreIfNotExists - flag to specify behavior if the table does not exist: if set to false, throw an exception, if set to true, nothing happens.

Throws:

TableNotExistException - if the table does not exist in the catalog

CatalogException - in case of any runtime exception

TablePartitionedException

alterPartitionStatistics

public void alterPartitionStatistics(ObjectPath tablePath,
                                     CatalogPartitionSpec partitionSpec,
                                     CatalogTableStatistics partitionStatistics,
                                     boolean ignoreIfNotExists)
                              throws PartitionNotExistException,
                                     CatalogException

Description copied from interface: Catalog

Update the statistics of a table partition.

Parameters:

tablePath - path of the table

partitionSpec - partition spec of the partition

partitionStatistics - new statistics to update

ignoreIfNotExists - flag to specify behavior if the partition does not exist: if set to false, throw an exception, if set to true, nothing happens.

Throws:

PartitionNotExistException - if the partition does not exist

CatalogException - in case of any runtime exception

alterPartitionColumnStatistics

public void alterPartitionColumnStatistics(ObjectPath tablePath,
                                           CatalogPartitionSpec partitionSpec,
                                           CatalogColumnStatistics columnStatistics,
                                           boolean ignoreIfNotExists)
                                    throws PartitionNotExistException,
                                           CatalogException

Description copied from interface: Catalog

Update the column statistics of a table partition.

Parameters:

tablePath - path of the table

partitionSpec - partition spec of the partition @@param columnStatistics new column statistics to update

ignoreIfNotExists - flag to specify behavior if the partition does not exist: if set to false, throw an exception, if set to true, nothing happens.

Throws:

PartitionNotExistException - if the partition does not exist

CatalogException - in case of any runtime exception

getTableStatistics

public CatalogTableStatistics getTableStatistics(ObjectPath tablePath)
                                          throws TableNotExistException,
                                                 CatalogException

Description copied from interface: Catalog

Get the statistics of a table.

Parameters:

tablePath - path of the table

Returns:

statistics of the given table

Throws:

TableNotExistException - if the table does not exist in the catalog

CatalogException - in case of any runtime exception

getTableColumnStatistics

public CatalogColumnStatistics getTableColumnStatistics(ObjectPath tablePath)
                                                 throws TableNotExistException,
                                                        CatalogException

Description copied from interface: Catalog

Get the column statistics of a table.

Parameters:

tablePath - path of the table

Returns:

column statistics of the given table

Throws:

TableNotExistException - if the table does not exist in the catalog

CatalogException - in case of any runtime exception

getPartitionStatistics

public CatalogTableStatistics getPartitionStatistics(ObjectPath tablePath,
                                                     CatalogPartitionSpec partitionSpec)
                                              throws PartitionNotExistException,
                                                     CatalogException

Description copied from interface: Catalog

Get the statistics of a partition.

Parameters:

tablePath - path of the table

partitionSpec - partition spec of the partition

Returns:

statistics of the given partition

Throws:

PartitionNotExistException - if the partition does not exist

CatalogException - in case of any runtime exception

bulkGetPartitionStatistics

public List<CatalogTableStatistics> bulkGetPartitionStatistics(ObjectPath tablePath,
                                                               List<CatalogPartitionSpec> partitionSpecs)
                                                        throws PartitionNotExistException,
                                                               CatalogException

Description copied from interface: Catalog

Get a list of statistics of given partitions.

Parameters:

tablePath - path of the table

partitionSpecs - partition specs of partitions that will be used to filter out all other unrelated statistics, i.e. the statistics fetch will be limited within the given partitions

Returns:

list of statistics of given partitions

Throws:

PartitionNotExistException - if one partition does not exist

CatalogException - in case of any runtime exception

getPartitionColumnStatistics

public CatalogColumnStatistics getPartitionColumnStatistics(ObjectPath tablePath,
                                                            CatalogPartitionSpec partitionSpec)
                                                     throws PartitionNotExistException,
                                                            CatalogException

Description copied from interface: Catalog

Get the column statistics of a partition.

Parameters:

tablePath - path of the table

partitionSpec - partition spec of the partition

Returns:

column statistics of the given partition

Throws:

PartitionNotExistException - if the partition does not exist

CatalogException - in case of any runtime exception

bulkGetPartitionColumnStatistics

public List<CatalogColumnStatistics> bulkGetPartitionColumnStatistics(ObjectPath tablePath,
                                                                      List<CatalogPartitionSpec> partitionSpecs)
                                                               throws PartitionNotExistException,
                                                                      CatalogException

Description copied from interface: Catalog

Get a list of column statistics for given partitions.

Parameters:

tablePath - path of the table

partitionSpecs - partition specs of partitions that will be used to filter out all other unrelated statistics, i.e. the statistics fetch will be limited within the given partitions

Returns:

list of column statistics for given partitions

Throws:

PartitionNotExistException - if one partition does not exist

CatalogException - in case of any runtime exception

supportsManagedTable

public boolean supportsManagedTable()

Description copied from interface: Catalog

If true, tables which do not specify a connector will be translated to managed tables.

isHiveTable

@Internal
public static boolean isHiveTable(Map<String,String> properties)

isHiveTable

@Internal
public static boolean isHiveTable(org.apache.hadoop.hive.metastore.api.Table table)

loadTable

@Internal
public void loadTable(org.apache.hadoop.fs.Path loadPath,
                                ObjectPath tablePath,
                                boolean isOverwrite,
                                boolean isSrcLocal)

loadPartition

@Internal
public void loadPartition(org.apache.hadoop.fs.Path loadPath,
                                    ObjectPath tablePath,
                                    Map<String,String> partSpec,
                                    boolean isOverwrite,
                                    boolean isSrcLocal)

isEmbeddedMetastore

@Internal
public static boolean isEmbeddedMetastore(HiveConf hiveConf)