CommT
- type of committable@Internal public class CommittableCollector<CommT> extends Object
CommittableSummary
.Constructor and Description |
---|
CommittableCollector(int subtaskId,
int numberOfSubtasks,
SinkCommitterMetricGroup metricGroup) |
Modifier and Type | Method and Description |
---|---|
void |
addMessage(CommittableMessage<CommT> message)
Adds a
CommittableMessage to the collector to hold it until emission. |
CommittableCollector<CommT> |
copy()
Returns a new committable collector that deep copies all internals.
|
Collection<? extends CheckpointCommittableManager<CommT>> |
getCheckpointCommittablesUpTo(long checkpointId)
Returns all
CheckpointCommittableManager until the requested checkpoint id. |
CommittableManager<CommT> |
getEndOfInputCommittable()
Returns
CheckpointCommittableManager that is currently hold by the collector and
associated with the EOI checkpoint id. |
int |
getNumberOfSubtasks()
Returns number of subtasks.
|
int |
getSubtaskId()
Returns subtask id.
|
boolean |
isFinished()
Returns whether all
CheckpointCommittableManager currently hold by the collector are
either committed or failed. |
void |
merge(CommittableCollector<CommT> cc)
Merges all information from an external collector into this collector.
|
static <CommT> CommittableCollector<CommT> |
of(RuntimeContext context,
SinkCommitterMetricGroup metricGroup)
Creates a
CommittableCollector based on the current runtime information. |
public CommittableCollector(int subtaskId, int numberOfSubtasks, SinkCommitterMetricGroup metricGroup)
public static <CommT> CommittableCollector<CommT> of(RuntimeContext context, SinkCommitterMetricGroup metricGroup)
CommittableCollector
based on the current runtime information. This method
should be used for to instantiate a collector for all Sink V2.CommT
- type of the committablecontext
- holding runtime of informationmetricGroup
- storing the committable metricsCommittableCollector
public void addMessage(CommittableMessage<CommT> message)
CommittableMessage
to the collector to hold it until emission.message
- either CommittableSummary
or CommittableWithLineage
public Collection<? extends CheckpointCommittableManager<CommT>> getCheckpointCommittablesUpTo(long checkpointId)
CheckpointCommittableManager
until the requested checkpoint id.checkpointId
- counterCheckpointCommittableManager
@Nullable public CommittableManager<CommT> getEndOfInputCommittable()
CheckpointCommittableManager
that is currently hold by the collector and
associated with the EOI
checkpoint id.CheckpointCommittableManager
public boolean isFinished()
CheckpointCommittableManager
currently hold by the collector are
either committed or failed.CheckpointCommittableManager
public void merge(CommittableCollector<CommT> cc)
This method is important during recovery from existing state.
cc
- other CommittableCollector
public int getNumberOfSubtasks()
public int getSubtaskId()
public CommittableCollector<CommT> copy()
CommittableCollector
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.