|Modifier and Type||Method and Description|
A complementary method to
Opens a stream to persist checkpoint state data that is owned strictly by tasks and not attached to the life cycle of a specific checkpoint.
Resolves a storage location reference into a CheckpointStreamFactory.
CheckpointStreamFactory resolveCheckpointStorageLocation(long checkpointId, CheckpointStorageLocationReference reference) throws IOException
The reference may be the
default reference, in which case the method should return the default location, taking
existing configuration and checkpoint ID into account.
checkpointId- The ID of the checkpoint that the location is initialized for.
reference- The checkpoint location reference.
IOException- Thrown, if the storage location cannot be initialized from the reference.
CheckpointStateOutputStream createTaskOwnedStateStream() throws IOException
This method should be used when the persisted data cannot be immediately dropped once the checkpoint that created it is dropped. Examples are write-ahead-logs. For those, the state can only be dropped once the data has been moved to the target system, which may sometimes take longer than one checkpoint (if the target system is temporarily unable to keep up).
The fact that the job manager does not own the life cycle of this type of state means also that it is strictly the responsibility of the tasks to handle the cleanup of this data.
Developer note: In the future, we may be able to make this a special case of "shared state", where the task re-emits the shared state reference as long as it needs to hold onto the persisted state data.
IOException- Thrown, if the stream cannot be opened.
createTaskOwnedStateStream(). Creates a toolset that gives access to additional operations that can be performed in the task owned state location.
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.