@Internal public class SplitAssignmentTracker<SplitT extends SourceSplit> extends Object
SplitEnumerator
.Constructor and Description |
---|
SplitAssignmentTracker() |
Modifier and Type | Method and Description |
---|---|
List<SplitT> |
getAndRemoveUncheckpointedAssignment(int subtaskId,
long restoredCheckpointId)
This method is invoked when a source reader fails over.
|
void |
onCheckpointComplete(long checkpointId)
when a checkpoint has been successfully made, this method is invoked to clean up the
assignment history before this successful checkpoint.
|
void |
recordSplitAssignment(SplitsAssignment<SplitT> splitsAssignment)
Record a new split assignment.
|
void |
restoreState(SimpleVersionedSerializer<SplitT> splitSerializer,
DataInputStream in)
Restore the state of the SplitAssignmentTracker.
|
void |
snapshotState(long checkpointId,
SimpleVersionedSerializer<SplitT> splitSerializer,
DataOutputStream out)
Take a snapshot of the uncheckpointed split assignments.
|
public void snapshotState(long checkpointId, SimpleVersionedSerializer<SplitT> splitSerializer, DataOutputStream out) throws Exception
checkpointId
- the id of the ongoing checkpointException
public void restoreState(SimpleVersionedSerializer<SplitT> splitSerializer, DataInputStream in) throws Exception
splitSerializer
- The serializer of the splits.in
- The ObjectInput that contains the state of the SplitAssignmentTracker.Exception
- when the state deserialization fails.public void onCheckpointComplete(long checkpointId)
checkpointId
- the id of the successful checkpoint.public void recordSplitAssignment(SplitsAssignment<SplitT> splitsAssignment)
splitsAssignment
- the new split assignment.public List<SplitT> getAndRemoveUncheckpointedAssignment(int subtaskId, long restoredCheckpointId)
subtaskId
- the subtask id of the reader that failed over.restoredCheckpointId
- the ID of the checkpoint that the reader was restored to.SplitEnumerator
.Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.