Spark2
This documentation is for an unreleased version of Apache Flink Table Store. We recommend you use the latest stable version.

Spark2 #

Table Store supports reading table store tables through Spark.

Version #

Table Store supports Spark 2.4+. It is highly recommended to use Spark 2.4+ version with many improvements.

Install #

You are using an unreleased version of Table Store, you need to manually Build Spark Bundled Jar from the source code.

Use --jars in spark-shell:

spark-shell ... --jars flink-table-store-spark2-0.3-SNAPSHOT.jar

Alternatively, you can copy flink-table-store-spark2-0.3-SNAPSHOT.jar under spark/jars in your Spark installation.

Note: If you are using HDFS, make sure that the environment variable HADOOP_HOME or HADOOP_CONF_DIR is set.

Read #

Table store with Spark 2.4 does not support DDL, you can use the Dataset reader and register the Dataset as a temporary table. In spark shell:

val dataset = spark.read.format("tablestore").load("file:/tmp/warehouse/default.db/myTable")
dataset.createOrReplaceTempView("myTable")
spark.sql("SELECT * FROM myTable")