pyflink.common.execution_config.ExecutionConfig

class ExecutionConfig(j_execution_config)

A config to define the behavior of the program execution. It allows to define (among other options) the following settings:

• The default parallelism of the program, i.e., how many parallel tasks to use for all functions that do not define a specific value directly.

• The number of retries in the case of failed executions.

• The delay between execution retries.

• The ExecutionMode of the program: Batch or Pipelined. The default execution mode is ExecutionMode.PIPELINED

• Enabling or disabling the “closure cleaner”. The closure cleaner pre-processes the implementations of functions. In case they are (anonymous) inner classes, it removes unused references to the enclosing class to fix certain serialization-related problems and to reduce the size of the closure.

• The config allows to register types and serializers to increase the efficiency of handling generic types and POJOs. This is usually only needed when the functions return not only the types declared in their signature, but also subclasses of those types.

PARALLELISM_DEFAULT:

The flag value indicating use of the default parallelism. This value can be used to reset the parallelism back to the default state.

PARALLELISM_UNKNOWN:

The flag value indicating an unknown or unset parallelism. This value is not a valid parallelism and indicates that the parallelism should remain unchanged.

Methods

add_default_kryo_serializer(type_class_name, ...)	Adds a new Kryo default serializer to the Runtime.
disable_auto_generated_uids()	Disables auto-generated UIDs.
disable_auto_type_registration()	Control whether Flink is automatically registering all types in the user programs with Kryo.
disable_closure_cleaner()	Disables the ClosureCleaner.
disable_force_avro()	Disables the Apache Avro serializer as the forced serializer for POJOs.
disable_force_kryo()	Disable use of Kryo serializer for all POJOs.
disable_generic_types()	Disables the use of generic types (types that would be serialized via Kryo).
disable_object_reuse()	Disables reusing objects that Flink internally uses for deserialization and passing data to user-code functions.
enable_auto_generated_uids()	Enables the Flink runtime to auto-generate UID's for operators.
enable_closure_cleaner()	Enables the ClosureCleaner.
enable_force_avro()	Forces Flink to use the Apache Avro serializer for POJOs.
enable_force_kryo()	Force TypeExtractor to use Kryo serializer for POJOS even though we could analyze as POJO.
enable_generic_types()	Enables the use generic types which are serialized via Kryo.
enable_object_reuse()	Enables reusing objects that Flink internally uses for deserialization and passing data to user-code functions.
get_auto_watermark_interval()	Returns the interval of the automatic watermark emission.
get_default_input_dependency_constraint()	Gets the default input dependency constraint for vertex scheduling.
get_default_kryo_serializer_classes()	Returns the registered default Kryo Serializer classes.
get_execution_mode()	Gets the execution mode used to execute the program.
get_global_job_parameters()	Gets current configuration dict.
get_latency_tracking_interval()	Returns the latency tracking interval.
get_max_parallelism()	Gets the maximum degree of parallelism defined for the program.
get_parallelism()	Gets the parallelism with which operation are executed by default.
get_registered_kryo_types()	Returns the registered Kryo types.
get_registered_pojo_types()	Returns the registered POJO types.
get_registered_types_with_kryo_serializer_classes()	Returns the registered types with their Kryo Serializer classes.
get_restart_strategy()	Returns the restart strategy which has been set for the current job.
get_task_cancellation_interval()	Gets the interval (in milliseconds) between consecutive attempts to cancel a running task.
get_task_cancellation_timeout()	Returns the timeout (in milliseconds) after which an ongoing task cancellation leads to a fatal TaskManager error.
has_auto_generated_uids_enabled()	Checks whether auto generated UIDs are supported.
has_generic_types_disabled()	Checks whether generic types are supported.
is_auto_type_registration_disabled()	Returns whether Flink is automatically registering all types in the user programs with Kryo.
is_closure_cleaner_enabled()	Returns whether the ClosureCleaner is enabled.
is_force_avro_enabled()	Returns whether the Apache Avro is the default serializer for POJOs.
is_force_kryo_enabled()	return Boolean value that represent whether the usage of Kryo serializer for all POJOs
is_object_reuse_enabled()	Returns whether object reuse has been enabled or disabled.
is_use_snapshot_compression()	Returns whether he compression (snappy) for keyed state in full checkpoints and savepoints is enabled.
register_kryo_type(type_class_name)	Registers the given type with the serialization stack.
register_pojo_type(type_class_name)	Registers the given type with the serialization stack.
register_type_with_kryo_serializer(...)	Registers the given Serializer via its class as a serializer for the given type at the KryoSerializer.
set_auto_watermark_interval(interval)	Sets the interval of the automatic watermark emission.
set_default_input_dependency_constraint(...)	Sets the default input dependency constraint for vertex scheduling.
set_execution_mode(execution_mode)	Sets the execution mode to execute the program.
set_global_job_parameters(...)	Register a custom, serializable user configuration dict.
set_latency_tracking_interval(interval)	Interval for sending latency tracking marks from the sources to the sinks.
set_max_parallelism(max_parallelism)	Sets the maximum degree of parallelism defined for the program.
set_parallelism(parallelism)	Sets the parallelism for operations executed through this environment.
set_restart_strategy(...)	Sets the restart strategy to be used for recovery.
set_task_cancellation_interval(interval)	Sets the configuration parameter specifying the interval (in milliseconds) between consecutive attempts to cancel a running task.
set_task_cancellation_timeout(timeout)	Sets the timeout (in milliseconds) after which an ongoing task cancellation is considered failed, leading to a fatal TaskManager error.
set_use_snapshot_compression(...)	Control whether the compression (snappy) for keyed state in full checkpoints and savepoints is enabled.

Attributes

PARALLELISM_DEFAULT
PARALLELISM_UNKNOWN