@PublicEvolving public final class TableConfig extends Object implements WritableConfig, ReadableConfig
TableEnvironment
session to adjust Table & SQL API
programs.
This class is a pure API class that abstracts configuration from various sources. Currently, configuration can be set in any of the following layers (in the given order):
config.yaml
,
StreamExecutionEnvironment
when bridging to DataStream API,
EnvironmentSettings.Builder.withConfiguration(Configuration)
/ TableEnvironment.create(Configuration)
,
set(ConfigOption, Object)
/ set(String,
String)
.
The latter two represent the application-specific part of the configuration. They initialize
and directly modify getConfiguration()
. Other layers represent the
configuration of the execution context and are immutable.
The getters get(ConfigOption)
and getOptional(ConfigOption)
give read-only
access to the full configuration. However, application-specific configuration has precedence.
Configuration of outer layers is used for defaults and fallbacks. The setters set(ConfigOption, Object)
and set(String, String)
will only affect
application-specific configuration.
For common or important configuration options, this class provides getters and setters methods with detailed inline documentation.
For more advanced configuration, users can directly access the underlying key-value map via
getConfiguration()
. Users can configure also underlying execution parameters via this
object.
For example:
tEnv.getConfig().addConfiguration(
new Configuration()
.set(CoreOptions.DEFAULT_PARALLELISM, 128)
.set(PipelineOptions.AUTO_WATERMARK_INTERVAL, Duration.ofMillis(800))
.set(CheckpointingOptions.CHECKPOINTING_INTERVAL, Duration.ofSeconds(30))
);
Note: Because options are read at different point in time when performing operations, it is recommended to set configuration options early after instantiating a table environment.
Constructor and Description |
---|
TableConfig()
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
void |
addConfiguration(Configuration configuration)
Adds the given key-value configuration to the underlying application-specific configuration.
|
void |
addJobParameter(String key,
String value)
Sets a custom user parameter that can be accessed via
FunctionContext.getJobParameter(String, String) . |
<T> T |
get(ConfigOption<T> option)
Reads a value using the metadata included in
ConfigOption . |
Configuration |
getConfiguration()
Gives direct access to the underlying application-specific key-value map for advanced
configuration.
|
static TableConfig |
getDefault() |
Duration |
getIdleStateRetention() |
ZoneId |
getLocalTimeZone()
Returns the current session time zone id.
|
Integer |
getMaxGeneratedCodeLength()
Returns the current threshold where generated code will be split into sub-function calls.
|
long |
getMaxIdleStateRetentionTime()
Deprecated.
use
getIdleStateRetention instead. |
long |
getMinIdleStateRetentionTime()
Deprecated.
use
getIdleStateRetention instead. |
<T> Optional<T> |
getOptional(ConfigOption<T> option)
Reads a value using the metadata included in
ConfigOption . |
PlannerConfig |
getPlannerConfig()
Returns the current configuration of Planner for Table API and SQL queries.
|
ReadableConfig |
getRootConfiguration()
Gives direct access to the underlying environment-specific key-value map for advanced
configuration.
|
SqlDialect |
getSqlDialect()
Returns the current SQL dialect.
|
<T> TableConfig |
set(ConfigOption<T> option,
T value)
Sets an application-specific value for the given
ConfigOption . |
TableConfig |
set(String key,
String value)
Sets an application-specific string-based value for the given string-based key.
|
void |
setIdleStateRetention(Duration duration)
Specifies a retention time interval for how long idle state, i.e., state which was not
updated, will be retained.
|
void |
setIdleStateRetentionTime(Time minTime,
Time maxTime)
Deprecated.
use
setIdleStateRetention(Duration) instead. |
void |
setLocalTimeZone(ZoneId zoneId)
Sets the current session time zone id.
|
void |
setMaxGeneratedCodeLength(Integer maxGeneratedCodeLength)
Sets current threshold where generated code will be split into sub-function calls.
|
void |
setPlannerConfig(PlannerConfig plannerConfig)
Sets the configuration of Planner for Table API and SQL queries.
|
void |
setRootConfiguration(ReadableConfig rootConfiguration)
Sets the given configuration as
rootConfiguration , which contains any configuration
set in the execution context. |
void |
setSqlDialect(SqlDialect sqlDialect)
Sets the current SQL dialect to parse a SQL query.
|
Map<String,String> |
toMap()
Converts the configuration items into a map of string key-value pairs.
|
@Deprecated public TableConfig()
getDefault()
instead.public <T> TableConfig set(ConfigOption<T> option, T value)
ConfigOption
.
This method should be preferred over set(String, String)
as it is type-safe,
avoids unnecessary parsing of the value, and provides inline documentation.
Note: Scala users might need to convert the value into a boxed type. E.g. by using Int.box(1)
or Boolean.box(false)
.
set
in interface WritableConfig
T
- type of the value to be storedoption
- metadata informationvalue
- value to be storedTableConfigOptions
,
ExecutionConfigOptions
,
OptimizerConfigOptions
public TableConfig set(String key, String value)
The value will be parsed by the framework on access.
This method exists for convenience when configuring a session with string-based
properties. Use set(ConfigOption, Object)
for more type-safety and inline
documentation.
public <T> T get(ConfigOption<T> option)
ConfigOption
. Returns the ConfigOption.defaultValue()
if value key not present in the configuration.
This method gives read-only access to the full configuration. However,
application-specific configuration has precedence. Configuration of outer layers is used for
defaults and fallbacks. See the docs of TableConfig
for more information.
get
in interface ReadableConfig
T
- type of the value to readoption
- metadata of the option to readConfigOption.defaultValue()
if not foundReadableConfig.getOptional(ConfigOption)
public <T> Optional<T> getOptional(ConfigOption<T> option)
ConfigOption
. In contrast to ReadableConfig.get(ConfigOption)
returns Optional.empty()
if value not present.
This method gives read-only access to the full configuration. However,
application-specific configuration has precedence. Configuration of outer layers is used for
defaults and fallbacks. See the docs of TableConfig
for more information.
getOptional
in interface ReadableConfig
T
- type of the value to readoption
- metadata of the option to readOptional.empty()
if not foundReadableConfig.get(ConfigOption)
@Internal public Map<String,String> toMap()
ReadableConfig
toMap
in interface ReadableConfig
public Configuration getConfiguration()
@Internal public ReadableConfig getRootConfiguration()
public void addConfiguration(Configuration configuration)
configuration
- key-value configuration to be addedpublic SqlDialect getSqlDialect()
public void setSqlDialect(SqlDialect sqlDialect)
public ZoneId getLocalTimeZone()
TIMESTAMP
WITH LOCAL TIME ZONE
. See setLocalTimeZone(ZoneId)
for more details.LocalZonedTimestampType
public void setLocalTimeZone(ZoneId zoneId)
DataTypes.TIMESTAMP_WITH_LOCAL_TIME_ZONE()
. Internally, timestamps with local time zone are
always represented in the UTC time zone. However, when converting to data types that don't
include a time zone (e.g. TIMESTAMP, TIME, or simply STRING), the session time zone is used
during conversion.
Example:
TableConfig config = tEnv.getConfig();
config.setLocalTimeZone(ZoneOffset.ofHours(2));
tEnv.executeSql("CREATE TABLE testTable (id BIGINT, tmstmp TIMESTAMP WITH LOCAL TIME ZONE)");
tEnv.executeSql("INSERT INTO testTable VALUES ((1, '2000-01-01 2:00:00'), (2, TIMESTAMP '2000-01-01 2:00:00'))");
tEnv.executeSql("SELECT * FROM testTable"); // query with local time zone set to UTC+2
should produce:
============================= id | tmstmp ============================= 1 | 2000-01-01 2:00:00' 2 | 2000-01-01 2:00:00'
If we change the local time zone and query the same table:
config.setLocalTimeZone(ZoneOffset.ofHours(0));
tEnv.executeSql("SELECT * FROM testTable"); // query with local time zone set to UTC+0
we should get:
============================= id | tmstmp ============================= 1 | 2000-01-01 0:00:00' 2 | 2000-01-01 0:00:00'
LocalZonedTimestampType
public PlannerConfig getPlannerConfig()
public void setPlannerConfig(PlannerConfig plannerConfig)
public Integer getMaxGeneratedCodeLength()
Default value is 4000 instead of 64KB as by default JIT refuses to work on methods with more than 8K byte code.
public void setMaxGeneratedCodeLength(Integer maxGeneratedCodeLength)
Default value is 4000 instead of 64KB as by default JIT refuses to work on methods with more than 8K byte code.
@Deprecated public void setIdleStateRetentionTime(Time minTime, Time maxTime)
setIdleStateRetention(Duration)
instead.When new data arrives for previously cleaned-up state, the new data will be handled as if it was the first data. This can result in previous results being overwritten.
Set to 0 (zero) to never clean-up the state.
NOTE: Cleaning up state requires additional bookkeeping which becomes less expensive for larger differences of minTime and maxTime. The difference between minTime and maxTime must be at least 5 minutes.
NOTE: Currently maxTime will be ignored and it will automatically derived from minTime as 1.5 x minTime.
minTime
- The minimum time interval for which idle state is retained. Set to 0 (zero) to
never clean-up the state.maxTime
- The maximum time interval for which idle state is retained. Must be at least 5
minutes greater than minTime. Set to 0 (zero) to never clean-up the state.public void setIdleStateRetention(Duration duration)
When new data arrives for previously cleaned-up state, the new data will be handled as if it was the first data. This can result in previous results being overwritten.
Set to 0 (zero) to never clean-up the state.
duration
- The retention time interval for which idle state is retained. Set to 0 (zero)
to never clean-up the state.StateTtlConfig
@Deprecated public long getMinIdleStateRetentionTime()
getIdleStateRetention
instead.@Deprecated public long getMaxIdleStateRetentionTime()
getIdleStateRetention
instead.public Duration getIdleStateRetention()
public void addJobParameter(String key, String value)
FunctionContext.getJobParameter(String, String)
.
This will add an entry to the current value of PipelineOptions.GLOBAL_JOB_PARAMETERS
.
It is also possible to set multiple parameters at once, which will override any previously set parameters:
Map<String, String> params = ...
TableConfig config = tEnv.getConfig();
config.set(PipelineOptions.GLOBAL_JOB_PARAMETERS, params);
@Internal public void setRootConfiguration(ReadableConfig rootConfiguration)
rootConfiguration
, which contains any configuration
set in the execution context. See the docs of TableConfig
for more information.rootConfiguration
- root configuration to be setpublic static TableConfig getDefault()
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.