org.apache.flink.connector.hbase.source

Class AbstractHBaseTableSource

java.lang.Object

org.apache.flink.connector.hbase.source.AbstractHBaseTableSource

All Implemented Interfaces:

BatchTableSource<Row>, LookupableTableSource<Row>, ProjectableTableSource<Row>, StreamTableSource<Row>, TableSource<Row>

Direct Known Subclasses:

HBaseTableSource, HBaseTableSource

@Internal
public abstract class AbstractHBaseTableSource
extends Object
implements BatchTableSource<Row>, ProjectableTableSource<Row>, StreamTableSource<Row>, LookupableTableSource<Row>

Creates a TableSource to scan an HBase table.

The table name and required HBase configuration is passed during AbstractHBaseTableSource construction. Use addColumn(String, String, Class) to specify the family, qualifier, and type of columns to scan.

The TableSource returns Row with nested Rows for each column family.

The HBaseTableSource is used as shown in the example below.

 HBaseTableSource hSrc = new HBaseTableSource(conf, "hTable");
 hSrc.setRowKey("rowkey", String.class);
 hSrc.addColumn("fam1", "col1", byte[].class);
 hSrc.addColumn("fam1", "col2", Integer.class);
 hSrc.addColumn("fam2", "col1", String.class);

 tableEnv.registerTableSourceInternal("hTable", hSrc);
 Table res = tableEnv.sqlQuery(
   "SELECT t.fam2.col1, SUM(t.fam1.col2) FROM hTable AS t " +
   "WHERE t.rowkey LIKE 'flink%' GROUP BY t.fam2.col1");
 

Field Summary

Fields

Modifier and Type	Field and Description
protected Configuration	conf
protected HBaseTableSchema	hbaseSchema
protected String	tableName

Constructor Summary

Constructors

Constructor and Description
AbstractHBaseTableSource(Configuration conf, String tableName, HBaseTableSchema hbaseSchema, int[] projectFields)

Method Summary

Modifier and Type	Method and Description
void	addColumn(String family, String qualifier, Class<?> clazz) Adds a column defined by family, qualifier, and type to the table schema.
String	explainSource() Describes the table source.
AsyncTableFunction<Row>	getAsyncLookupFunction(String[] lookupKeys) Gets the AsyncTableFunction which supports async lookup one key at a time.
DataSet<Row>	getDataSet(ExecutionEnvironment execEnv) Returns the data of the table as a DataSet.
DataStream<Row>	getDataStream(StreamExecutionEnvironment execEnv) Returns the data of the table as a DataStream.
HBaseTableSchema	getHBaseTableSchema()
protected abstract InputFormat<Row,?>	getInputFormat(HBaseTableSchema projectedSchema)
TableFunction<Row>	getLookupFunction(String[] lookupKeys) Gets the TableFunction which supports lookup one key at a time.
TypeInformation<Row>	getReturnType()
TableSchema	getTableSchema() Returns the schema of the produced table.
boolean	isAsyncEnabled() Returns true if async lookup is enabled.
boolean	isBounded() Returns true if this is a bounded source, false if this is an unbounded source.
void	setCharset(String charset) Specifies the charset to parse Strings to HBase byte[] keys and String values.
void	setRowKey(String rowKeyName, Class<?> clazz) Sets row key information in the table schema.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.sources.ProjectableTableSource

projectFields

Methods inherited from interface org.apache.flink.table.sources.TableSource

getProducedDataType

Field Detail

conf

protected final Configuration conf

tableName

protected final String tableName

hbaseSchema

protected final HBaseTableSchema hbaseSchema

Constructor Detail

AbstractHBaseTableSource

public AbstractHBaseTableSource(Configuration conf,
                                String tableName,
                                HBaseTableSchema hbaseSchema,
                                int[] projectFields)

Method Detail

addColumn

public void addColumn(String family,
                      String qualifier,
                      Class<?> clazz)

Adds a column defined by family, qualifier, and type to the table schema.

Parameters:

family - the family name

qualifier - the qualifier name

clazz - the data type of the qualifier

setRowKey

public void setRowKey(String rowKeyName,
                      Class<?> clazz)

Sets row key information in the table schema.

Parameters:

rowKeyName - the row key field name

clazz - the data type of the row key

setCharset

public void setCharset(String charset)

Specifies the charset to parse Strings to HBase byte[] keys and String values.

Parameters:

charset - Name of the charset to use.

getReturnType

public TypeInformation<Row> getReturnType()

Specified by:

getReturnType in interface TableSource<Row>

getTableSchema

public TableSchema getTableSchema()

Description copied from interface: TableSource

Returns the schema of the produced table.

Specified by:

getTableSchema in interface TableSource<Row>

Returns:

The TableSchema of the produced table.

getDataSet

public DataSet<Row> getDataSet(ExecutionEnvironment execEnv)

Description copied from interface: BatchTableSource

Returns the data of the table as a DataSet.

NOTE: This method is for internal use only for defining a TableSource. Do not use it in Table API programs.

Specified by:

getDataSet in interface BatchTableSource<Row>

explainSource

public String explainSource()

Description copied from interface: TableSource

Describes the table source.

Specified by:

explainSource in interface TableSource<Row>

Returns:

A String explaining the TableSource.

getLookupFunction

public TableFunction<Row> getLookupFunction(String[] lookupKeys)

Description copied from interface: LookupableTableSource

Gets the TableFunction which supports lookup one key at a time.

Specified by:

getLookupFunction in interface LookupableTableSource<Row>

Parameters:

lookupKeys - the chosen field names as lookup keys, it is in the defined order

getAsyncLookupFunction

public AsyncTableFunction<Row> getAsyncLookupFunction(String[] lookupKeys)

Description copied from interface: LookupableTableSource

Gets the AsyncTableFunction which supports async lookup one key at a time.

Specified by:

getAsyncLookupFunction in interface LookupableTableSource<Row>

Parameters:

lookupKeys - the chosen field names as lookup keys, it is in the defined order

isAsyncEnabled

public boolean isAsyncEnabled()

Description copied from interface: LookupableTableSource

Returns true if async lookup is enabled.

The lookup function returned by LookupableTableSource.getAsyncLookupFunction(String[]) will be used if returns true. Otherwise, the lookup function returned by LookupableTableSource.getLookupFunction(String[]) will be used.

Specified by:

isAsyncEnabled in interface LookupableTableSource<Row>

isBounded

public boolean isBounded()

Description copied from interface: StreamTableSource

Returns true if this is a bounded source, false if this is an unbounded source. Default is unbounded for compatibility.

Specified by:

isBounded in interface StreamTableSource<Row>

getDataStream

public DataStream<Row> getDataStream(StreamExecutionEnvironment execEnv)

Description copied from interface: StreamTableSource

Returns the data of the table as a DataStream.

NOTE: This method is for internal use only for defining a TableSource. Do not use it in Table API programs.

Specified by:

getDataStream in interface StreamTableSource<Row>

getInputFormat

protected abstract InputFormat<Row,?> getInputFormat(HBaseTableSchema projectedSchema)

getHBaseTableSchema

@VisibleForTesting
public HBaseTableSchema getHBaseTableSchema()