org.apache.flink.api.common.state

Class StateTtlConfig.Builder

java.lang.Object

org.apache.flink.api.common.state.StateTtlConfig.Builder

Enclosing class:

StateTtlConfig

public static class StateTtlConfig.Builder
extends Object

Builder for the StateTtlConfig.

Constructor Summary

Constructors

Constructor and Description
Builder(Time ttl)

Method Summary

Modifier and Type	Method and Description
StateTtlConfig	build()
StateTtlConfig.Builder	cleanupFullSnapshot() Cleanup expired state in full snapshot on checkpoint.
StateTtlConfig.Builder	cleanupIncrementally(int cleanupSize, boolean runCleanupForEveryRecord) Cleanup expired state incrementally cleanup local state.
StateTtlConfig.Builder	cleanupInRocksdbCompactFilter(long queryTimeAfterNumEntries) Cleanup expired state while Rocksdb compaction is running.
StateTtlConfig.Builder	disableCleanupInBackground() Disable default cleanup of expired state in background (enabled by default).
StateTtlConfig.Builder	neverReturnExpired()
StateTtlConfig.Builder	returnExpiredIfNotCleanedUp()
StateTtlConfig.Builder	setStateVisibility(StateTtlConfig.StateVisibility stateVisibility) Sets the state visibility.
StateTtlConfig.Builder	setTtl(Time ttl) Sets the ttl time.
StateTtlConfig.Builder	setTtlTimeCharacteristic(StateTtlConfig.TtlTimeCharacteristic ttlTimeCharacteristic) Sets the time characteristic.
StateTtlConfig.Builder	setUpdateType(StateTtlConfig.UpdateType updateType) Sets the ttl update type.
StateTtlConfig.Builder	updateTtlOnCreateAndWrite()
StateTtlConfig.Builder	updateTtlOnReadAndWrite()
StateTtlConfig.Builder	useProcessingTime()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

public Builder(@Nonnull
               Time ttl)

Method Detail

setUpdateType

@Nonnull
public StateTtlConfig.Builder setUpdateType(StateTtlConfig.UpdateType updateType)

Sets the ttl update type.

Parameters:

updateType - The ttl update type configures when to update last access timestamp which prolongs state TTL.

updateTtlOnCreateAndWrite

@Nonnull
public StateTtlConfig.Builder updateTtlOnCreateAndWrite()

updateTtlOnReadAndWrite

@Nonnull
public StateTtlConfig.Builder updateTtlOnReadAndWrite()

setStateVisibility

@Nonnull
public StateTtlConfig.Builder setStateVisibility(@Nonnull
                                                          StateTtlConfig.StateVisibility stateVisibility)

Sets the state visibility.

Parameters:

stateVisibility - The state visibility configures whether expired user value can be returned or not.

returnExpiredIfNotCleanedUp

@Nonnull
public StateTtlConfig.Builder returnExpiredIfNotCleanedUp()

neverReturnExpired

@Nonnull
public StateTtlConfig.Builder neverReturnExpired()

setTtlTimeCharacteristic

@Nonnull
public StateTtlConfig.Builder setTtlTimeCharacteristic(@Nonnull
                                                                StateTtlConfig.TtlTimeCharacteristic ttlTimeCharacteristic)

Sets the time characteristic.

Parameters:

ttlTimeCharacteristic - The time characteristic configures time scale to use for ttl.

useProcessingTime

@Nonnull
public StateTtlConfig.Builder useProcessingTime()

cleanupFullSnapshot

@Nonnull
public StateTtlConfig.Builder cleanupFullSnapshot()

Cleanup expired state in full snapshot on checkpoint.

cleanupIncrementally

@Nonnull
public StateTtlConfig.Builder cleanupIncrementally(@Nonnegative
                                                            int cleanupSize,
                                                            boolean runCleanupForEveryRecord)

Cleanup expired state incrementally cleanup local state.

Upon every state access this cleanup strategy checks a bunch of state keys for expiration and cleans up expired ones. It keeps a lazy iterator through all keys with relaxed consistency if backend supports it. This way all keys should be regularly checked and cleaned eventually over time if any state is constantly being accessed.

Additionally to the incremental cleanup upon state access, it can also run per every record. Caution: if there are a lot of registered states using this option, they all will be iterated for every record to check if there is something to cleanup.

Note: if no access happens to this state or no records are processed in case of runCleanupForEveryRecord, expired state will persist.

Note: Time spent for the incremental cleanup increases record processing latency.

Note: At the moment incremental cleanup is implemented only for Heap state backend. Setting it for RocksDB will have no effect.

Note: If heap state backend is used with synchronous snapshotting, the global iterator keeps a copy of all keys while iterating because of its specific implementation which does not support concurrent modifications. Enabling of this feature will increase memory consumption then. Asynchronous snapshotting does not have this problem.

Parameters:

cleanupSize - max number of keys pulled from queue for clean up upon state touch for any key

runCleanupForEveryRecord - run incremental cleanup per each processed record

cleanupInRocksdbCompactFilter

@Nonnull
public StateTtlConfig.Builder cleanupInRocksdbCompactFilter(long queryTimeAfterNumEntries)

Cleanup expired state while Rocksdb compaction is running.

RocksDB compaction filter will query current timestamp, used to check expiration, from Flink every time after processing queryTimeAfterNumEntries number of state entries. Updating the timestamp more often can improve cleanup speed but it decreases compaction performance because it uses JNI call from native code.

Parameters:

queryTimeAfterNumEntries - number of state entries to process by compaction filter before updating current timestamp

disableCleanupInBackground

@Nonnull
public StateTtlConfig.Builder disableCleanupInBackground()

Disable default cleanup of expired state in background (enabled by default).

If some specific cleanup is configured, e.g. cleanupIncrementally(int, boolean) or cleanupInRocksdbCompactFilter(long), this setting does not disable it.

setTtl

@Nonnull
public StateTtlConfig.Builder setTtl(@Nonnull
                                              Time ttl)

Sets the ttl time.

Parameters:

ttl - The ttl time.

build

@Nonnull
public StateTtlConfig build()