Catalog#

Catalog#

Catalog is responsible for reading and writing metadata such as database/table/views/UDFs from a registered catalog. It connects a registered catalog and Flink’s Table API.

Catalog.get_default_database()

Get the name of the default database for this catalog.

Catalog.list_databases()

Get the names of all databases in this catalog.

Catalog.get_database(database_name)

Get a database from this catalog.

Catalog.database_exists(database_name)

Check if a database exists in this catalog.

Catalog.create_database(name, database, ...)

Create a database.

Catalog.drop_database(name, ignore_if_exists)

Drop a database.

Catalog.alter_database(name, new_database, ...)

Modify an existing database.

Catalog.list_tables(database_name)

Get names of all tables and views under this database.

Catalog.list_views(database_name)

Get names of all views under this database.

Catalog.get_table(table_path)

Get a CatalogTable or CatalogView identified by tablePath.

Catalog.table_exists(table_path)

Check if a table or view exists in this catalog.

Catalog.drop_table(table_path, ...)

Drop a table or view.

Catalog.rename_table(table_path, ...)

Rename an existing table or view.

Catalog.create_table(table_path, table, ...)

Create a new table or view.

Catalog.alter_table(table_path, new_table, ...)

Modify an existing table or view.

Catalog.list_partitions(table_path[, ...])

Get CatalogPartitionSpec of all partitions of the table.

Catalog.get_partition(table_path, partition_spec)

Get a partition of the given table.

Catalog.partition_exists(table_path, ...)

Check whether a partition exists or not.

Catalog.create_partition(table_path, ...)

Create a partition.

Catalog.drop_partition(table_path, ...)

Drop a partition.

Catalog.alter_partition(table_path, ...)

Alter a partition.

Catalog.list_functions(database_name)

List the names of all functions in the given database.

Catalog.get_function(function_path)

Get the function.

Catalog.function_exists(function_path)

Check whether a function exists or not.

Catalog.create_function(function_path, ...)

Create a function.

Catalog.alter_function(function_path, ...)

Modify an existing function.

Catalog.drop_function(function_path, ...)

Drop a function.

Catalog.get_table_statistics(table_path)

Get the statistics of a table.

Catalog.get_table_column_statistics(table_path)

Get the column statistics of a table.

Catalog.get_partition_statistics(table_path, ...)

Get the statistics of a partition.

Catalog.bulk_get_partition_statistics(...)

Get a list of statistics of given partitions.

Catalog.get_partition_column_statistics(...)

Get the column statistics of a partition.

Catalog.bulk_get_partition_column_statistics(...)

Get a list of the column statistics for the given partitions.

Catalog.alter_table_statistics(table_path, ...)

Update the statistics of a table.

Catalog.alter_table_column_statistics(...)

Update the column statistics of a table.

Catalog.alter_partition_statistics(...)

Update the statistics of a table partition.

Catalog.alter_partition_column_statistics(...)

Update the column statistics of a table partition.

CatalogDatabase#

Represents a database object in a catalog.

CatalogDatabase.create_instance(properties)

Creates an instance of CatalogDatabase.

CatalogDatabase.get_properties()

Get a map of properties associated with the database.

CatalogDatabase.get_comment()

Get comment of the database.

CatalogDatabase.copy()

Get a deep copy of the CatalogDatabase instance.

CatalogDatabase.get_description()

Get a brief description of the database.

CatalogDatabase.get_detailed_description()

Get a detailed description of the database.

CatalogBaseTable#

CatalogBaseTable is the common parent of table and view. It has a map of key-value pairs defining the properties of the table.

CatalogBaseTable.create_table(schema[, ...])

Create an instance of CatalogBaseTable for the catalog table.

CatalogBaseTable.create_view(original_query, ...)

Create an instance of CatalogBaseTable for the catalog view.

CatalogBaseTable.get_options()

Returns a map of string-based options.

CatalogBaseTable.get_unresolved_schema()

Returns the schema of the table or view.

CatalogBaseTable.get_comment()

Get comment of the table or view.

CatalogBaseTable.copy()

Get a deep copy of the CatalogBaseTable instance.

CatalogBaseTable.get_description()

Get a brief description of the table or view.

CatalogBaseTable.get_detailed_description()

Get a detailed description of the table or view.

CatalogPartition#

Represents a partition object in catalog.

CatalogPartition.create_instance(properties)

Creates an instance of CatalogPartition.

CatalogPartition.get_properties()

Get a map of properties associated with the partition.

CatalogPartition.copy()

Get a deep copy of the CatalogPartition instance.

CatalogPartition.get_description()

Get a brief description of the partition object.

CatalogPartition.get_detailed_description()

Get a detailed description of the partition object.

CatalogPartition.get_comment()

Get comment of the partition.

CatalogFunction#

Represents a partition object in catalog.

CatalogFunction.create_instance(class_name)

Creates an instance of CatalogDatabase.

CatalogFunction.get_class_name()

Get the full name of the class backing the function.

CatalogFunction.copy()

Create a deep copy of the function.

CatalogFunction.get_description()

Get a brief description of the function.

CatalogFunction.get_detailed_description()

Get a detailed description of the function.

CatalogFunction.is_generic()

Whether or not is the function a flink UDF.

CatalogFunction.get_function_language()

Get the language used for the function definition.

ObjectPath#

A database name and object (table/view/function) name combo in a catalog.

ObjectPath.from_string(full_name)

ObjectPath.get_database_name()

ObjectPath.get_object_name()

ObjectPath.get_full_name()

CatalogPartitionSpec#

Represents a partition spec object in catalog. Partition columns and values are NOT of strict order, and they need to be re-arranged to the correct order by comparing with a list of strictly ordered partition keys.

CatalogPartitionSpec.get_partition_spec()

Get the partition spec as key-value map.

CatalogTableStatistics#

Statistics for a non-partitioned table or a partition of a partitioned table.

CatalogTableStatistics.get_row_count()

The number of rows in the table or partition.

CatalogTableStatistics.get_field_count()

The number of files on disk.

CatalogTableStatistics.get_total_size()

The total size in bytes.

CatalogTableStatistics.get_raw_data_size()

The raw data size (size when loaded in memory) in bytes.

CatalogTableStatistics.get_properties()

CatalogTableStatistics.copy()

Create a deep copy of "this" instance.

CatalogColumnStatistics#

Column statistics of a table or partition.

CatalogColumnStatistics.get_column_statistics_data()

CatalogColumnStatistics.get_properties()

CatalogColumnStatistics.copy()

HiveCatalog#

A catalog implementation for Hive.

HiveCatalog(catalog_name[, ...])

A catalog implementation for Hive.

JdbcCatalog#

A catalog implementation for Jdbc.

JdbcCatalog(catalog_name, default_database, ...)

A catalog implementation for Jdbc.