org.apache.flink.table.api.bridge.java

Interface StreamTableEnvironment

All Superinterfaces:

TableEnvironment

All Known Implementing Classes:

StreamTableEnvironmentImpl

@PublicEvolving
public interface StreamTableEnvironment
extends TableEnvironment

This table environment is the entry point and central context for creating Table and SQL API programs that integrate with the Java-specific DataStream API.

It is unified for bounded and unbounded data processing.

A stream table environment is responsible for:

• Convert a DataStream into Table and vice-versa.
• Connecting to external systems.
• Registering and retrieving Tables and other meta objects from a catalog.
• Executing SQL statements.
• Offering further configuration options.

Note: If you don't intend to use the DataStream API, TableEnvironment is meant for pure table programs.

Method Summary

Modifier and Type	Method and Description
StreamTableDescriptor	connect(ConnectorDescriptor connectorDescriptor) Deprecated. The SQL CREATE TABLE DDL is richer than this part of the API. This method might be refactored in the next versions. Please use executeSql(ddl) to register a table instead.
static StreamTableEnvironment	create(StreamExecutionEnvironment executionEnvironment) Creates a table environment that is the entry point and central context for creating Table and SQL API programs that integrate with the Java-specific DataStream API.
static StreamTableEnvironment	create(StreamExecutionEnvironment executionEnvironment, EnvironmentSettings settings) Creates a table environment that is the entry point and central context for creating Table and SQL API programs that integrate with the Java-specific DataStream API.
static StreamTableEnvironment	create(StreamExecutionEnvironment executionEnvironment, TableConfig tableConfig) Deprecated. Use create(StreamExecutionEnvironment) and TableEnvironment.getConfig() for manipulating TableConfig.
<T> void	createTemporaryView(String path, DataStream<T> dataStream) Creates a view from the given DataStream in a given path.
<T> void	createTemporaryView(String path, DataStream<T> dataStream, Expression... fields) Creates a view from the given DataStream in a given path with specified field names.
<T> void	createTemporaryView(String path, DataStream<T> dataStream, String fields) Deprecated. use createTemporaryView(String, DataStream, Expression...)
JobExecutionResult	execute(String jobName) Triggers the program execution.
<T> Table	fromDataStream(DataStream<T> dataStream) Converts the given DataStream into a Table.
<T> Table	fromDataStream(DataStream<T> dataStream, Expression... fields) Converts the given DataStream into a Table with specified field names.
<T> Table	fromDataStream(DataStream<T> dataStream, String fields) Deprecated. use fromDataStream(DataStream, Expression...)
<T> void	registerDataStream(String name, DataStream<T> dataStream) Deprecated. use createTemporaryView(String, DataStream)
<T> void	registerDataStream(String name, DataStream<T> dataStream, String fields) Deprecated. use createTemporaryView(String, DataStream, Expression...)
<T,ACC> void	registerFunction(String name, AggregateFunction<T,ACC> aggregateFunction) Deprecated. Use TableEnvironment.createTemporarySystemFunction(String, UserDefinedFunction) instead. Please note that the new method also uses the new type system and reflective extraction logic. It might be necessary to update the function implementation as well. See the documentation of AggregateFunction for more information on the new function design.
<T,ACC> void	registerFunction(String name, TableAggregateFunction<T,ACC> tableAggregateFunction) Deprecated. Use TableEnvironment.createTemporarySystemFunction(String, UserDefinedFunction) instead. Please note that the new method also uses the new type system and reflective extraction logic. It might be necessary to update the function implementation as well. See the documentation of TableAggregateFunction for more information on the new function design.
<T> void	registerFunction(String name, TableFunction<T> tableFunction) Deprecated. Use TableEnvironment.createTemporarySystemFunction(String, UserDefinedFunction) instead. Please note that the new method also uses the new type system and reflective extraction logic. It might be necessary to update the function implementation as well. See the documentation of TableFunction for more information on the new function design.
<T> DataStream<T>	toAppendStream(Table table, Class<T> clazz) Converts the given Table into an append DataStream of a specified type.
<T> DataStream<T>	toAppendStream(Table table, TypeInformation<T> typeInfo) Converts the given Table into an append DataStream of a specified type.
<T> DataStream<Tuple2<Boolean,T>>	toRetractStream(Table table, Class<T> clazz) Converts the given Table into a DataStream of add and retract messages.
<T> DataStream<Tuple2<Boolean,T>>	toRetractStream(Table table, TypeInformation<T> typeInfo) Converts the given Table into a DataStream of add and retract messages.

Methods inherited from interface org.apache.flink.table.api.TableEnvironment

create, createFunction, createFunction, createStatementSet, createTemporaryFunction, createTemporaryFunction, createTemporarySystemFunction, createTemporarySystemFunction, createTemporaryView, dropFunction, dropTemporaryFunction, dropTemporarySystemFunction, dropTemporaryTable, dropTemporaryView, executeSql, explain, explain, explain, explainSql, from, fromTableSource, fromValues, fromValues, fromValues, fromValues, fromValues, fromValues, getCatalog, getCompletionHints, getConfig, getCurrentCatalog, getCurrentDatabase, insertInto, insertInto, listCatalogs, listDatabases, listFunctions, listModules, listTables, listTemporaryTables, listTemporaryViews, listUserDefinedFunctions, listViews, loadModule, registerCatalog, registerFunction, registerTable, scan, sqlQuery, sqlUpdate, unloadModule, useCatalog, useDatabase

Method Detail

create

static StreamTableEnvironment create(StreamExecutionEnvironment executionEnvironment)

Creates a table environment that is the entry point and central context for creating Table and SQL API programs that integrate with the Java-specific DataStream API.

It is unified for bounded and unbounded data processing.

A stream table environment is responsible for:

• Convert a DataStream into Table and vice-versa.
• Connecting to external systems.
• Registering and retrieving Tables and other meta objects from a catalog.
• Executing SQL statements.
• Offering further configuration options.

Note: If you don't intend to use the DataStream API, TableEnvironment is meant for pure table programs.

Parameters:

executionEnvironment - The Java StreamExecutionEnvironment of the TableEnvironment.

create

static StreamTableEnvironment create(StreamExecutionEnvironment executionEnvironment,
                                     EnvironmentSettings settings)

Creates a table environment that is the entry point and central context for creating Table and SQL API programs that integrate with the Java-specific DataStream API.

It is unified for bounded and unbounded data processing.

A stream table environment is responsible for:

• Convert a DataStream into Table and vice-versa.
• Connecting to external systems.
• Registering and retrieving Tables and other meta objects from a catalog.
• Executing SQL statements.
• Offering further configuration options.

Note: If you don't intend to use the DataStream API, TableEnvironment is meant for pure table programs.

Parameters:

executionEnvironment - The Java StreamExecutionEnvironment of the TableEnvironment.

settings - The environment settings used to instantiate the TableEnvironment.

create

@Deprecated
static StreamTableEnvironment create(StreamExecutionEnvironment executionEnvironment,
                                                 TableConfig tableConfig)

Deprecated. Use create(StreamExecutionEnvironment) and TableEnvironment.getConfig() for manipulating TableConfig.

Creates a table environment that is the entry point and central context for creating Table and SQL API programs that integrate with the Java-specific DataStream API.

It is unified for bounded and unbounded data processing.

A stream table environment is responsible for:

• Convert a DataStream into Table and vice-versa.
• Connecting to external systems.
• Registering and retrieving Tables and other meta objects from a catalog.
• Executing SQL statements.
• Offering further configuration options.

Note: If you don't intend to use the DataStream API, TableEnvironment is meant for pure table programs.

Parameters:

executionEnvironment - The Java StreamExecutionEnvironment of the TableEnvironment.

tableConfig - The configuration of the TableEnvironment.

registerFunction

@Deprecated
<T> void registerFunction(String name,
                                      TableFunction<T> tableFunction)

Deprecated. Use TableEnvironment.createTemporarySystemFunction(String, UserDefinedFunction) instead. Please note that the new method also uses the new type system and reflective extraction logic. It might be necessary to update the function implementation as well. See the documentation of TableFunction for more information on the new function design.

Registers a TableFunction under a unique name in the TableEnvironment's catalog. Registered functions can be referenced in Table API and SQL queries.

Type Parameters:

T - The type of the output row.

Parameters:

name - The name under which the function is registered.

tableFunction - The TableFunction to register.

registerFunction

@Deprecated
<T,ACC> void registerFunction(String name,
                                          AggregateFunction<T,ACC> aggregateFunction)

Deprecated. Use TableEnvironment.createTemporarySystemFunction(String, UserDefinedFunction) instead. Please note that the new method also uses the new type system and reflective extraction logic. It might be necessary to update the function implementation as well. See the documentation of AggregateFunction for more information on the new function design.

Registers an AggregateFunction under a unique name in the TableEnvironment's catalog. Registered functions can be referenced in Table API and SQL queries.

Type Parameters:

T - The type of the output value.

ACC - The type of aggregate accumulator.

Parameters:

name - The name under which the function is registered.

aggregateFunction - The AggregateFunction to register.

registerFunction

@Deprecated
<T,ACC> void registerFunction(String name,
                                          TableAggregateFunction<T,ACC> tableAggregateFunction)

Deprecated. Use TableEnvironment.createTemporarySystemFunction(String, UserDefinedFunction) instead. Please note that the new method also uses the new type system and reflective extraction logic. It might be necessary to update the function implementation as well. See the documentation of TableAggregateFunction for more information on the new function design.

Registers an TableAggregateFunction under a unique name in the TableEnvironment's catalog. Registered functions can only be referenced in Table API.

Type Parameters:

T - The type of the output value.

ACC - The type of aggregate accumulator.

Parameters:

name - The name under which the function is registered.

tableAggregateFunction - The TableAggregateFunction to register.

fromDataStream

<T> Table fromDataStream(DataStream<T> dataStream)

Converts the given DataStream into a Table.

The field names of the Table are automatically derived from the type of the DataStream.

Type Parameters:

T - The type of the DataStream.

Parameters:

dataStream - The DataStream to be converted.

Returns:

The converted Table.

fromDataStream

@Deprecated
<T> Table fromDataStream(DataStream<T> dataStream,
                                     String fields)

Deprecated. use fromDataStream(DataStream, Expression...)

Converts the given DataStream into a Table with specified field names.

There are two modes for mapping original fields to the fields of the Table:

1. Reference input fields by name: All fields in the schema definition are referenced by name (and possibly renamed using an alias (as). Moreover, we can define proctime and rowtime attributes at arbitrary positions using arbitrary names (except those that exist in the result schema). In this mode, fields can be reordered and projected out. This mode can be used for any input type, including POJOs.

Example:

 DataStream<Tuple2<String, Long>> stream = ...
 // reorder the fields, rename the original 'f0' field to 'name' and add event-time
 // attribute named 'rowtime'
 Table table = tableEnv.fromDataStream(stream, "f1, rowtime.rowtime, f0 as 'name'");
 

2. Reference input fields by position: In this mode, fields are simply renamed. Event-time attributes can replace the field on their position in the input data (if it is of correct type) or be appended at the end. Proctime attributes must be appended at the end. This mode can only be used if the input type has a defined field order (tuple, case class, Row) and none of the fields references a field of the input type.

Example:

 DataStream<Tuple2<String, Long>> stream = ...
 // rename the original fields to 'a' and 'b' and extract the internally attached timestamp into an event-time
 // attribute named 'rowtime'
 Table table = tableEnv.fromDataStream(stream, "a, b, rowtime.rowtime");
 

Type Parameters:

T - The type of the DataStream.

Parameters:

dataStream - The DataStream to be converted.

fields - The fields expressions to map original fields of the DataStream to the fields of the Table.

Returns:

The converted Table.

fromDataStream

<T> Table fromDataStream(DataStream<T> dataStream,
                         Expression... fields)

Converts the given DataStream into a Table with specified field names.

There are two modes for mapping original fields to the fields of the Table:

1. Reference input fields by name: All fields in the schema definition are referenced by name (and possibly renamed using an alias (as). Moreover, we can define proctime and rowtime attributes at arbitrary positions using arbitrary names (except those that exist in the result schema). In this mode, fields can be reordered and projected out. This mode can be used for any input type, including POJOs.

Example:

 DataStream<Tuple2<String, Long>> stream = ...
 Table table = tableEnv.fromDataStream(
    stream,
    $("f1"), // reorder and use the original field
    $("rowtime").rowtime(), // extract the internally attached timestamp into an event-time
                            // attribute named 'rowtime'
    $("f0").as("name") // reorder and give the original field a better name
 );
 

2. Reference input fields by position: In this mode, fields are simply renamed. Event-time attributes can replace the field on their position in the input data (if it is of correct type) or be appended at the end. Proctime attributes must be appended at the end. This mode can only be used if the input type has a defined field order (tuple, case class, Row) and none of the fields references a field of the input type.

Example:

 DataStream<Tuple2<String, Long>> stream = ...
 Table table = tableEnv.fromDataStream(
    stream,
    $("a"), // rename the first field to 'a'
    $("b"), // rename the second field to 'b'
    $("rowtime").rowtime() // extract the internally attached timestamp into an event-time
                           // attribute named 'rowtime'
 );
 

Type Parameters:

T - The type of the DataStream.

Parameters:

dataStream - The DataStream to be converted.

fields - The fields expressions to map original fields of the DataStream to the fields of the Table.

Returns:

The converted Table.

registerDataStream

@Deprecated
<T> void registerDataStream(String name,
                                        DataStream<T> dataStream)

Deprecated. use createTemporaryView(String, DataStream)

Creates a view from the given DataStream. Registered views can be referenced in SQL queries.

The field names of the Table are automatically derived from the type of the DataStream.

The view is registered in the namespace of the current catalog and database. To register the view in a different catalog use createTemporaryView(String, DataStream).

Temporary objects can shadow permanent ones. If a permanent object in a given path exists, it will be inaccessible in the current session. To make the permanent object available again you can drop the corresponding temporary object.

Type Parameters:

T - The type of the DataStream to register.

Parameters:

name - The name under which the DataStream is registered in the catalog.

dataStream - The DataStream to register.

createTemporaryView

<T> void createTemporaryView(String path,
                             DataStream<T> dataStream)

Creates a view from the given DataStream in a given path. Registered views can be referenced in SQL queries.

The field names of the Table are automatically derived from the type of the DataStream.

Temporary objects can shadow permanent ones. If a permanent object in a given path exists, it will be inaccessible in the current session. To make the permanent object available again you can drop the corresponding temporary object.

Type Parameters:

T - The type of the DataStream.

Parameters:

path - The path under which the DataStream is created. See also the TableEnvironment class description for the format of the path.

dataStream - The DataStream out of which to create the view.

registerDataStream

@Deprecated
<T> void registerDataStream(String name,
                                        DataStream<T> dataStream,
                                        String fields)

Deprecated. use createTemporaryView(String, DataStream, Expression...)

Creates a view from the given DataStream in a given path with specified field names. Registered views can be referenced in SQL queries.

There are two modes for mapping original fields to the fields of the View:

1. Reference input fields by name: All fields in the schema definition are referenced by name (and possibly renamed using an alias (as). Moreover, we can define proctime and rowtime attributes at arbitrary positions using arbitrary names (except those that exist in the result schema). In this mode, fields can be reordered and projected out. This mode can be used for any input type, including POJOs.

Example:

 DataStream<Tuple2<String, Long>> stream = ...
 // reorder the fields, rename the original 'f0' field to 'name' and add event-time
 // attribute named 'rowtime'
 tableEnv.registerDataStream("myTable", stream, "f1, rowtime.rowtime, f0 as 'name'");
 

2. Reference input fields by position: In this mode, fields are simply renamed. Event-time attributes can replace the field on their position in the input data (if it is of correct type) or be appended at the end. Proctime attributes must be appended at the end. This mode can only be used if the input type has a defined field order (tuple, case class, Row) and none of the fields references a field of the input type.

Example:

 DataStream<Tuple2<String, Long>> stream = ...
 // rename the original fields to 'a' and 'b' and extract the internally attached timestamp into an event-time
 // attribute named 'rowtime'
 tableEnv.registerDataStream("myTable", stream, "a, b, rowtime.rowtime");
 

The view is registered in the namespace of the current catalog and database. To register the view in a different catalog use createTemporaryView(String, DataStream).

Temporary objects can shadow permanent ones. If a permanent object in a given path exists, it will be inaccessible in the current session. To make the permanent object available again you can drop the corresponding temporary object.

Type Parameters:

T - The type of the DataStream to register.

Parameters:

name - The name under which the DataStream is registered in the catalog.

dataStream - The DataStream to register.

fields - The fields expressions to map original fields of the DataStream to the fields of the View.

createTemporaryView

@Deprecated
<T> void createTemporaryView(String path,
                                         DataStream<T> dataStream,
                                         String fields)

Deprecated. use createTemporaryView(String, DataStream, Expression...)

Creates a view from the given DataStream in a given path with specified field names. Registered views can be referenced in SQL queries.

There are two modes for mapping original fields to the fields of the View:

1. Reference input fields by name: All fields in the schema definition are referenced by name (and possibly renamed using an alias (as). Moreover, we can define proctime and rowtime attributes at arbitrary positions using arbitrary names (except those that exist in the result schema). In this mode, fields can be reordered and projected out. This mode can be used for any input type, including POJOs.

Example:

 DataStream<Tuple2<String, Long>> stream = ...
 // reorder the fields, rename the original 'f0' field to 'name' and add event-time
 // attribute named 'rowtime'
 tableEnv.createTemporaryView("cat.db.myTable", stream, "f1, rowtime.rowtime, f0 as 'name'");
 

2. Reference input fields by position: In this mode, fields are simply renamed. Event-time attributes can replace the field on their position in the input data (if it is of correct type) or be appended at the end. Proctime attributes must be appended at the end. This mode can only be used if the input type has a defined field order (tuple, case class, Row) and none of the fields references a field of the input type.

Example:

 DataStream<Tuple2<String, Long>> stream = ...
 // rename the original fields to 'a' and 'b' and extract the internally attached timestamp into an event-time
 // attribute named 'rowtime'
 tableEnv.createTemporaryView("cat.db.myTable", stream, "a, b, rowtime.rowtime");
 

Temporary objects can shadow permanent ones. If a permanent object in a given path exists, it will be inaccessible in the current session. To make the permanent object available again you can drop the corresponding temporary object.

Type Parameters:

T - The type of the DataStream.

Parameters:

path - The path under which the DataStream is created. See also the TableEnvironment class description for the format of the path.

dataStream - The DataStream out of which to create the view.

fields - The fields expressions to map original fields of the DataStream to the fields of the View.

createTemporaryView

<T> void createTemporaryView(String path,
                             DataStream<T> dataStream,
                             Expression... fields)

Creates a view from the given DataStream in a given path with specified field names. Registered views can be referenced in SQL queries.

There are two modes for mapping original fields to the fields of the View:

1. Reference input fields by name: All fields in the schema definition are referenced by name (and possibly renamed using an alias (as). Moreover, we can define proctime and rowtime attributes at arbitrary positions using arbitrary names (except those that exist in the result schema). In this mode, fields can be reordered and projected out. This mode can be used for any input type, including POJOs.

Example:

 DataStream<Tuple2<String, Long>> stream = ...
 tableEnv.createTemporaryView(
    "cat.db.myTable",
    stream,
    $("f1"), // reorder and use the original field
    $("rowtime").rowtime(), // extract the internally attached timestamp into an event-time
                            // attribute named 'rowtime'
    $("f0").as("name") // reorder and give the original field a better name
 );
 

2. Reference input fields by position: In this mode, fields are simply renamed. Event-time attributes can replace the field on their position in the input data (if it is of correct type) or be appended at the end. Proctime attributes must be appended at the end. This mode can only be used if the input type has a defined field order (tuple, case class, Row) and none of the fields references a field of the input type.

Example:

 DataStream<Tuple2<String, Long>> stream = ...
 tableEnv.createTemporaryView(
    "cat.db.myTable",
    stream,
    $("a"), // rename the first field to 'a'
    $("b"), // rename the second field to 'b'
    $("rowtime").rowtime() // adds an event-time attribute named 'rowtime'
 );
 

Temporary objects can shadow permanent ones. If a permanent object in a given path exists, it will be inaccessible in the current session. To make the permanent object available again you can drop the corresponding temporary object.

Type Parameters:

T - The type of the DataStream.

Parameters:

path - The path under which the DataStream is created. See also the TableEnvironment class description for the format of the path.

dataStream - The DataStream out of which to create the view.

fields - The fields expressions to map original fields of the DataStream to the fields of the View.

toAppendStream

<T> DataStream<T> toAppendStream(Table table,
                                 Class<T> clazz)

Converts the given Table into an append DataStream of a specified type.

The Table must only have insert (append) changes. If the Table is also modified by update or delete changes, the conversion will fail.

The fields of the Table are mapped to DataStream fields as follows:

• Row and Tuple types: Fields are mapped by position, field types must match.
• POJO DataStream types: Fields are mapped by field name, field types must match.

Type Parameters:

T - The type of the resulting DataStream.

Parameters:

table - The Table to convert.

clazz - The class of the type of the resulting DataStream.

Returns:

The converted DataStream.

toAppendStream

<T> DataStream<T> toAppendStream(Table table,
                                 TypeInformation<T> typeInfo)

Converts the given Table into an append DataStream of a specified type.

The Table must only have insert (append) changes. If the Table is also modified by update or delete changes, the conversion will fail.

The fields of the Table are mapped to DataStream fields as follows:

• Row and Tuple types: Fields are mapped by position, field types must match.
• POJO DataStream types: Fields are mapped by field name, field types must match.

Type Parameters:

T - The type of the resulting DataStream.

Parameters:

table - The Table to convert.

typeInfo - The TypeInformation that specifies the type of the DataStream.

Returns:

The converted DataStream.

toRetractStream

<T> DataStream<Tuple2<Boolean,T>> toRetractStream(Table table,
                                                  Class<T> clazz)

Converts the given Table into a DataStream of add and retract messages. The message will be encoded as Tuple2. The first field is a Boolean flag, the second field holds the record of the specified type T.

A true Boolean flag indicates an add message, a false flag indicates a retract message.

The fields of the Table are mapped to DataStream fields as follows:

• Row and Tuple types: Fields are mapped by position, field types must match.
• POJO DataStream types: Fields are mapped by field name, field types must match.

Type Parameters:

T - The type of the requested record type.

Parameters:

table - The Table to convert.

clazz - The class of the requested record type.

Returns:

The converted DataStream.

toRetractStream

<T> DataStream<Tuple2<Boolean,T>> toRetractStream(Table table,
                                                  TypeInformation<T> typeInfo)

Converts the given Table into a DataStream of add and retract messages. The message will be encoded as Tuple2. The first field is a Boolean flag, the second field holds the record of the specified type T.

A true Boolean flag indicates an add message, a false flag indicates a retract message.

The fields of the Table are mapped to DataStream fields as follows:

• Row and Tuple types: Fields are mapped by position, field types must match.
• POJO DataStream types: Fields are mapped by field name, field types must match.

Type Parameters:

T - The type of the requested record type.

Parameters:

table - The Table to convert.

typeInfo - The TypeInformation of the requested record type.

Returns:

The converted DataStream.

connect

@Deprecated
StreamTableDescriptor connect(ConnectorDescriptor connectorDescriptor)

Deprecated. The SQL CREATE TABLE DDL is richer than this part of the API. This method might be refactored in the next versions. Please use executeSql(ddl) to register a table instead.

Creates a table source and/or table sink from a descriptor.

Descriptors allow for declaring the communication to external systems in an implementation-agnostic way. The classpath is scanned for suitable table factories that match the desired configuration.

The following example shows how to read from a Kafka connector using a JSON format and registering a table source "MyTable" in append mode:

 tableEnv
   .connect(
     new Kafka()
       .version("0.11")
       .topic("clicks")
       .property("group.id", "click-group")
       .startFromEarliest())
   .withFormat(
     new Json()
       .jsonSchema("{...}")
       .failOnMissingField(false))
   .withSchema(
     new Schema()
       .field("user-name", "VARCHAR").from("u_name")
       .field("count", "DECIMAL")
       .field("proc-time", "TIMESTAMP").proctime())
   .inAppendMode()
   .createTemporaryTable("MyTable")
 

Specified by:

connect in interface TableEnvironment

Parameters:

connectorDescriptor - connector descriptor describing the external system

execute

JobExecutionResult execute(String jobName)
                    throws Exception

Triggers the program execution. The environment will execute all parts of the program.

The program execution will be logged and displayed with the provided name

It calls the StreamExecutionEnvironment.execute(String) on the underlying StreamExecutionEnvironment. In contrast to the TableEnvironment this environment translates queries eagerly.

Specified by:

execute in interface TableEnvironment

Parameters:

jobName - Desired name of the job

Returns:

The result of the job execution, containing elapsed time and accumulators.

Throws:

Exception - which occurs during job execution.