public class DuplicatingStateChangeFsUploader extends AbstractStateChangeFsUploader
LocalChangelogRegistry
, TaskChangelogRegistry
and ChangelogTaskLocalStateStore
are responsible for managing
them.
TaskChangelogRegistry
is
responsible for TM side, and SharedStateRegistry
is responsible for JM side.
The total discard logic of local dstl files is:
TaskChangelogRegistry.startTracking(org.apache.flink.runtime.state.StreamStateHandle, long)
on AbstractStateChangeFsUploader.upload(java.util.Collection<org.apache.flink.changelog.fs.StateChangeUploadScheduler.UploadTask>)
.
ChangelogTaskLocalStateStore
by
AsyncCheckpointRunnable#reportCompletedSnapshotStates().
LocalChangelogRegistry.register(org.apache.flink.runtime.state.StreamStateHandle, long)
when
FsStateChangelogWriter#persist , files of the previous checkpoint will be deleted by
LocalChangelogRegistry.discardUpToCheckpoint(long)
when the checkpoint is confirmed.
TaskChangelogRegistry.release(org.apache.flink.runtime.state.StreamStateHandle)
is responsible for deleting local files.
StateChangeUploader.UploadTasksResult
bufferSize, compression, metrics
Constructor and Description |
---|
DuplicatingStateChangeFsUploader(JobID jobID,
Path basePath,
FileSystem fileSystem,
boolean compression,
int bufferSize,
ChangelogStorageMetricGroup metrics,
TaskChangelogRegistry changelogRegistry,
LocalSnapshotDirectoryProvider localSnapshotDirectoryProvider) |
Modifier and Type | Method and Description |
---|---|
void |
close() |
org.apache.flink.changelog.fs.OutputStreamWithPos |
prepareStream() |
generateFileName, upload
public DuplicatingStateChangeFsUploader(JobID jobID, Path basePath, FileSystem fileSystem, boolean compression, int bufferSize, ChangelogStorageMetricGroup metrics, TaskChangelogRegistry changelogRegistry, LocalSnapshotDirectoryProvider localSnapshotDirectoryProvider)
public org.apache.flink.changelog.fs.OutputStreamWithPos prepareStream() throws IOException
IOException
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.