pyflink.common package¶

Module contents¶

Common classes used by both Flink DataStream API and Table API:

Configuration: Lightweight configuration object which stores key/value pairs.

ExecutionConfig: A config to define the behavior of the program execution.

ExecutionMode: Specifies how a batch program is executed in terms of data exchange: pipelining or batched.

TypeInformation: TypeInformation is the core class of Flink’s type system. FLink requires a type information for all types that are used as input or return type of a user function.

Types: Contains utilities to access the TypeInformation of the most common types for which Flink has provided built-in implementation.

WatermarkStrategy: Defines how to generate Watermarks in the stream sources.

Row: A row is a fixed-length, null-aware composite type for storing multiple values in a deterministic field order.

SerializationSchema: Base class to describes how to turn a data object into a different serialized representation. Most data sinks (for example Apache Kafka) require the data to be handed to them in a specific format (for example as byte strings). See JsonRowSerializationSchema, JsonRowDeserializationSchema, CsvRowSerializationSchema, CsvRowDeserializationSchema, AvroRowSerializationSchema, AvroRowDeserializationSchema and SimpleStringSchema for more details.

class pyflink.common.AssignerWithPeriodicWatermarksWrapper(j_assigner_with_periodic_watermarks)[source]¶

Bases: object

The AssignerWithPeriodicWatermarks assigns event time timestamps to elements, and generates low watermarks that signal event time progress within the stream. These timestamps and watermarks are used by functions and operators that operate on event time, for example event time windows.

class pyflink.common.CompletableFuture(j_completable_future, py_class=None)[source]¶

Bases: object

A Future that may be explicitly completed (setting its value and status), supporting dependent functions and actions that trigger upon its completion.

When two or more threads attempt to set_result, set_exception, or cancel a CompletableFuture, only one of them succeeds.

New in version 1.11.0.

cancel() → bool[source]¶

Completes this CompletableFuture if not already completed.

Returns: true if this task is now cancelled

New in version 1.11.0.

cancelled() → bool[source]¶: Returns true if this CompletableFuture was cancelled before it completed normally.

New in version 1.11.0.

done() → bool[source]¶: Returns true if completed in any fashion: normally, exceptionally, or via cancellation.

New in version 1.11.0.

exception()[source]¶: Returns the exception that was set on this future or None if no exception was set.

New in version 1.11.0.

result()[source]¶

Waits if necessary for this future to complete, and then returns its result.

Returns: the result value

New in version 1.11.0.

class pyflink.common.ConfigOption(*args, **kwds)[source]¶

Bases: typing.Generic

A {@code ConfigOption} describes a configuration parameter. It encapsulates the configuration key, deprecated older versions of the key, and an optional default value for the configuration parameter.

{@code ConfigOptions} are built via the ConfigOptions class. Once created, a config option is immutable.

class pyflink.common.ConfigOptions(j_config_options)[source]¶

Bases: object

{@code ConfigOptions} are used to build a ConfigOption. The option is typically built in one of the following patterns:

Example:

# simple string-valued option with a default value
>>> ConfigOptions.key("tmp.dir").string_type().default_value("/tmp")
# simple integer-valued option with a default value
>>> ConfigOptions.key("application.parallelism").int_type().default_value(100)
# option with no default value
>>> ConfigOptions.key("user.name").string_type().no_default_value()

class OptionBuilder(j_option_builder)[source]¶

Bases: object

boolean_type() → pyflink.common.config_options.ConfigOptions.TypedConfigOptionBuilder[bool][source]¶: Defines that the value of the option should be of bool type.

double_type() → pyflink.common.config_options.ConfigOptions.TypedConfigOptionBuilder[float][source]¶: Defines that the value of the option should be of float Double} type (8-byte double precision floating point number).

float_type() → pyflink.common.config_options.ConfigOptions.TypedConfigOptionBuilder[float][source]¶: Defines that the value of the option should be of float type (4-byte single precision floating point number).

int_type() → pyflink.common.config_options.ConfigOptions.TypedConfigOptionBuilder[int][source]¶: Defines that the value of the option should be of int type (from -2,147,483,648 to 2,147,483,647).

long_type() → pyflink.common.config_options.ConfigOptions.TypedConfigOptionBuilder[int][source]¶: Defines that the value of the option should be of int type (from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807).

string_type() → pyflink.common.config_options.ConfigOptions.TypedConfigOptionBuilder[str][source]¶: Defines that the value of the option should be of str type.

class TypedConfigOptionBuilder(*args, **kwds)[source]¶

Bases: typing.Generic

default_value(value: T) → pyflink.common.config_options.ConfigOption[T][source]¶

no_default_value() → pyflink.common.config_options.ConfigOption[str][source]¶

static key(key: str)[source]¶

Starts building a new ConfigOption.

Parameters: key – The key for the config option.
Returns: The builder for the config option with the given key.

class pyflink.common.Configuration(other: Optional[pyflink.common.configuration.Configuration] = None, j_configuration: py4j.java_gateway.JavaObject = None)[source]¶

Bases: object

Lightweight configuration object which stores key/value pairs.

add_all(other: pyflink.common.configuration.Configuration, prefix: str = None) → pyflink.common.configuration.Configuration[source]¶

Adds all entries from the given configuration into this configuration. The keys are prepended with the given prefix if exist.

Parameters

other – The configuration whose entries are added to this configuration.
prefix – Optional, the prefix to prepend.

add_all_to_dict(target_dict: Dict)[source]¶

Adds all entries in this configuration to the given dict.

Parameters: target_dict – The dict to be updated.

contains_key(key: str) → bool[source]¶

Checks whether there is an entry with the specified key.

Parameters: key – Key of entry.
Returns: True if the key is stored, false otherwise.

get_boolean(key: str, default_value: bool) → bool[source]¶

Returns the value associated with the given key as a boolean.

Parameters

key – The key pointing to the associated value.
default_value – The default value which is returned in case there is no value associated with the given key.

Returns