A factory for components of general purpose test jobs for Flink's DataStream API operators and
primitives.
The components can be configured for different state backends, including memory, file, and
RocksDB state backends. It also allows specifying the processing guarantee semantics, which will
also be verified by the job itself according to the specified semantic.
Program parameters:
- test.semantics (String, default - 'exactly-once'): This configures the semantics to test.
Can be 'exactly-once' or 'at-least-once'.
- test.simulate_failure (boolean, default - false): This configures whether or not to
simulate failures by throwing exceptions within the job.
- test.simulate_failure.num_records (long, default - 100L): The number of records to process
before throwing an exception, per job execution attempt. Only relevant if configured to
simulate failures.
- test.simulate_failure.num_checkpoints (long, default - 1L): The number of complete
checkpoints before throwing an exception, per job execution attempt. Only relevant if
configured to simulate failures.
- test.simulate_failure.max_failures (int, default - 1): The maximum number of times to fail
the job. This also takes into account failures that were not triggered by the job's own
failure simulation, e.g. TaskManager or JobManager failures. Only relevant if configured to
simulate failures.
- environment.checkpoint_interval (long, default - 1000): the checkpoint interval.
- environment.externalize_checkpoint (boolean, default - false): whether or not checkpoints
should be externalized.
- environment.externalize_checkpoint.cleanup (String, default - 'retain'): Configures the
cleanup mode for externalized checkpoints. Can be 'retain' or 'delete'.
- environment.tolerable_checkpoint_failure_number (int, default - 0): Sets the expected
behaviour for the job manager in case that it received declined checkpoints from tasks.
- environment.parallelism (int, default - 1): parallelism to use for the job.
- environment.max_parallelism (int, default - 128): max parallelism to use for the job
- environment.restart_strategy (String, default - 'fixed_delay'): The failure restart
strategy to use. Can be 'fixed_delay' or 'no_restart'.
- environment.restart_strategy.fixed_delay.attempts (Integer, default - Integer.MAX_VALUE):
The number of allowed attempts to restart the job, when using 'fixed_delay' restart.
- environment.restart_strategy.fixed_delay.delay (long, default - 0): delay between restart
attempts, in milliseconds, when using 'fixed_delay' restart.
- state_backend (String, default - 'file'): Supported values are 'file' for FsStateBackend
and 'rocks' for RocksDBStateBackend.
- state_backend.checkpoint_directory (String): The checkpoint directory.
- state_backend.rocks.incremental (boolean, default - false): Activate or deactivate
incremental snapshots if RocksDBStateBackend is selected.
- state_backend.file.async (boolean, default - true): Activate or deactivate asynchronous
snapshots if FileStateBackend is selected.
- sequence_generator_source.keyspace (int, default - 1000): Number of different keys for
events emitted by the sequence generator.
- sequence_generator_source.payload_size (int, default - 20): Length of message payloads
emitted by the sequence generator.
- sequence_generator_source.sleep_time (long, default - 0): Milliseconds to sleep after
emitting events in the sequence generator. Set to 0 to disable sleeping.
- sequence_generator_source.sleep_after_elements (long, default - 0): Number of elements to
emit before sleeping in the sequence generator. Set to 0 to disable sleeping.
- sequence_generator_source.event_time.max_out_of_order (long, default - 500): Max event time
out-of-orderness for events emitted by the sequence generator.
- sequence_generator_source.event_time.clock_progress (long, default - 100): The amount of
event time to progress per event generated by the sequence generator.
- tumbling_window_operator.num_events (long, default - 20L): The duration of the window,
indirectly determined by the target number of events in each window. Total duration is
(sliding_window_operator.num_events) *
(sequence_generator_source.event_time.clock_progress).
- test_slide_factor (int, default - 3): test_slide_factor (int, default - 3): how many slides
there are in a single window (in other words at most how many windows may be opened at time
for a given key) The length of a window will be calculated as (test_slide_size) *
(test_slide_factor)
- test_slide_size (long, default - 250): length of a slide of sliding window in milliseconds.
The length of a window will be calculated as (test_slide_size) * (test_slide_factor)