Spark2 #

Table Store supports reading table store tables through Spark.

Version #

Table Store supports Spark 2.4+. It is highly recommended to use Spark 2.4+ version with many improvements.

Install #

Download flink-table-store-spark2-0.2.1.jar.

Use --jars in spark-shell:

spark-shell ... --jars flink-table-store-spark2-0.2.1.jar

Alternatively, you can copy flink-table-store-spark2-0.2.1.jar under spark/jars in your Spark installation.

Note: If you are using HDFS, make sure that the environment variable HADOOP_HOME or HADOOP_CONF_DIR is set.

Read #

Table store with Spark 2.4 does not support DDL, you can use the Dataset reader and register the Dataset as a temporary table. In spark shell:

val dataset = spark.read.format("tablestore").load("file:/tmp/warehouse/default.db/myTable")
dataset.createOrReplaceTempView("myTable")
spark.sql("SELECT * FROM myTable")