CommT
- The committable type of the GlobalCommitter
.GlobalCommT
- The global committable type of the GlobalCommitter
.@Internal public final class GlobalStreamingCommitterHandler<CommT,GlobalCommT> extends Object
CommitterHandler
for executing GlobalCommitter
in the streaming execution mode.Constructor and Description |
---|
GlobalStreamingCommitterHandler(GlobalCommitter<CommT,GlobalCommT> globalCommitter,
SimpleVersionedSerializer<GlobalCommT> committableSerializer) |
Modifier and Type | Method and Description |
---|---|
void |
close() |
protected List<CommT> |
commitUpTo(long checkpointId) |
List<GlobalCommT> |
endOfInput() |
void |
initializeState(StateInitializationContext context) |
boolean |
needsRetry() |
Collection<GlobalCommT> |
notifyCheckpointCompleted(long checkpointId) |
protected List<InputT> |
pollCommittables() |
protected List<RecoverT> |
prependRecoveredCommittables(List<RecoverT> committables) |
List<OutputT> |
processCommittables(List<InputT> committables)
Processes the committables by either directly transforming them or by adding them to the
internal state of this handler.
|
protected void |
recoveredCommittables(List<GlobalCommT> committables)
Notifies a list of committables that might need to be committed again after recovering from a
failover.
|
void |
retry()
Retries all recovered committables.
|
protected void |
retry(List<CommT> recoveredCommittables) |
void |
snapshotState(StateSnapshotContext context) |
public GlobalStreamingCommitterHandler(GlobalCommitter<CommT,GlobalCommT> globalCommitter, SimpleVersionedSerializer<GlobalCommT> committableSerializer)
protected void recoveredCommittables(List<GlobalCommT> committables) throws IOException
committables
- A list of committablesIOException
public List<GlobalCommT> endOfInput()
public Collection<GlobalCommT> notifyCheckpointCompleted(long checkpointId) throws IOException, InterruptedException
IOException
InterruptedException
protected void retry(List<CommT> recoveredCommittables) throws IOException, InterruptedException
IOException
InterruptedException
public void initializeState(StateInitializationContext context) throws Exception
Exception
public void snapshotState(StateSnapshotContext context) throws Exception
Exception
protected List<CommT> commitUpTo(long checkpointId) throws IOException, InterruptedException
IOException
InterruptedException
protected List<RecoverT> prependRecoveredCommittables(List<RecoverT> committables)
public boolean needsRetry()
public void retry() throws IOException, InterruptedException
initializeState(StateInitializationContext)
and have been re-added in any of the committing
functions.IOException
InterruptedException
public List<OutputT> processCommittables(List<InputT> committables)
protected List<InputT> pollCommittables()
Copyright © 2014–2023 The Apache Software Foundation. All rights reserved.