public class PerJobCheckpointRecoveryFactory<T extends CompletedCheckpointStore> extends Object implements CheckpointRecoveryFactory
CheckpointRecoveryFactory
which creates and keeps separate CompletedCheckpointStore
and CheckpointIDCounter
for each JobID
.Constructor and Description |
---|
PerJobCheckpointRecoveryFactory(java.util.function.BiFunction<Integer,T,T> completedCheckpointStorePerJobFactory) |
PerJobCheckpointRecoveryFactory(java.util.function.BiFunction<Integer,T,T> completedCheckpointStorePerJobFactory,
java.util.function.Supplier<CheckpointIDCounter> checkpointIDCounterPerJobFactory) |
Modifier and Type | Method and Description |
---|---|
CheckpointIDCounter |
createCheckpointIDCounter(JobID jobId)
Creates a
CheckpointIDCounter instance for a job. |
CompletedCheckpointStore |
createRecoveredCompletedCheckpointStore(JobID jobId,
int maxNumberOfCheckpointsToRetain,
ClassLoader userClassLoader)
Creates a RECOVERED
CompletedCheckpointStore instance for a job. |
static <T extends CompletedCheckpointStore> |
withoutCheckpointStoreRecovery(java.util.function.IntFunction<T> storeFn) |
public PerJobCheckpointRecoveryFactory(java.util.function.BiFunction<Integer,T,T> completedCheckpointStorePerJobFactory)
public PerJobCheckpointRecoveryFactory(java.util.function.BiFunction<Integer,T,T> completedCheckpointStorePerJobFactory, java.util.function.Supplier<CheckpointIDCounter> checkpointIDCounterPerJobFactory)
@VisibleForTesting public static <T extends CompletedCheckpointStore> CheckpointRecoveryFactory withoutCheckpointStoreRecovery(java.util.function.IntFunction<T> storeFn)
public CompletedCheckpointStore createRecoveredCompletedCheckpointStore(JobID jobId, int maxNumberOfCheckpointsToRetain, ClassLoader userClassLoader)
CheckpointRecoveryFactory
CompletedCheckpointStore
instance for a job. In this context,
RECOVERED means, that if we already have completed checkpoints from previous runs, we should
use them as the initial state.createRecoveredCompletedCheckpointStore
in interface CheckpointRecoveryFactory
jobId
- Job ID to recover checkpoints formaxNumberOfCheckpointsToRetain
- Maximum number of checkpoints to retainuserClassLoader
- User code class loader of the jobCompletedCheckpointStore
instance for the jobpublic CheckpointIDCounter createCheckpointIDCounter(JobID jobId)
CheckpointRecoveryFactory
CheckpointIDCounter
instance for a job.createCheckpointIDCounter
in interface CheckpointRecoveryFactory
jobId
- Job ID to recover checkpoints forCheckpointIDCounter
instance for the jobCopyright © 2014–2023 The Apache Software Foundation. All rights reserved.