pyflink.datastream.state.StateTtlConfig.Builder.cleanup_in_rocksdb_compact_filter#
- Builder.cleanup_in_rocksdb_compact_filter(query_time_after_num_entries, periodic_compaction_time=None) pyflink.datastream.state.StateTtlConfig.Builder #
Cleanup expired state while Rocksdb compaction is running.
RocksDB compaction filter will query current timestamp, used to check expiration, from Flink every time after processing {@code queryTimeAfterNumEntries} number of state entries. Updating the timestamp more often can improve cleanup speed but it decreases compaction performance because it uses JNI call from native code.
Periodic compaction could speed up expired state entries cleanup, especially for state entries rarely accessed. Files older than this value will be picked up for compaction, and re-written to the same level as they were before. It makes sure a file goes through compaction filters periodically.
- Parameters
query_time_after_num_entries – number of state entries to process by compaction filter before updating current timestamp
periodic_compaction_time – periodic compaction which could speed up expired state cleanup. 0 means turning off periodic compaction.
- Returns