Class CommittableCollector<CommT>
- java.lang.Object
-
- org.apache.flink.streaming.runtime.operators.sink.committables.CommittableCollector<CommT>
-
- Type Parameters:
CommT
- type of committable
@Internal public class CommittableCollector<CommT> extends Object
This class is responsible to book-keep the committing progress across checkpoints and upstream subtasks.Each checkpoint in turn is handled by a
CheckpointCommittableManager
.
-
-
Constructor Summary
Constructors Constructor Description CommittableCollector(SinkCommitterMetricGroup metricGroup)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
addMessage(CommittableMessage<CommT> message)
Adds aCommittableMessage
to the collector to hold it until emission.CommittableCollector<CommT>
copy()
Returns a new committable collector that deep copies all internals.boolean
equals(Object o)
Collection<? extends CheckpointCommittableManager<CommT>>
getCheckpointCommittablesUpTo(long checkpointId)
Returns allCheckpointCommittableManager
until the requested checkpoint id.Optional<CheckpointCommittableManager<CommT>>
getEndOfInputCommittable()
ReturnsCheckpointCommittableManager
belonging to the last input.int
hashCode()
boolean
isFinished()
Returns whether allCheckpointCommittableManager
currently hold by the collector are either committed or failed.void
merge(CommittableCollector<CommT> cc)
Merges all information from an external collector into this collector.static <CommT> CommittableCollector<CommT>
of(SinkCommitterMetricGroup metricGroup)
Creates aCommittableCollector
based on the current runtime information.void
remove(CheckpointCommittableManager<CommT> manager)
Removes the manager for a specific checkpoint and all it's metadata.String
toString()
-
-
-
Constructor Detail
-
CommittableCollector
public CommittableCollector(SinkCommitterMetricGroup metricGroup)
-
-
Method Detail
-
of
public static <CommT> CommittableCollector<CommT> of(SinkCommitterMetricGroup metricGroup)
Creates aCommittableCollector
based on the current runtime information. This method should be used for to instantiate a collector for all Sink V2.- Type Parameters:
CommT
- type of the committable- Parameters:
metricGroup
- storing the committable metrics- Returns:
CommittableCollector
-
addMessage
public void addMessage(CommittableMessage<CommT> message)
Adds aCommittableMessage
to the collector to hold it until emission.- Parameters:
message
- eitherCommittableSummary
orCommittableWithLineage
-
getCheckpointCommittablesUpTo
public Collection<? extends CheckpointCommittableManager<CommT>> getCheckpointCommittablesUpTo(long checkpointId)
Returns allCheckpointCommittableManager
until the requested checkpoint id.- Parameters:
checkpointId
- counter- Returns:
- collection of
CheckpointCommittableManager
-
getEndOfInputCommittable
public Optional<CheckpointCommittableManager<CommT>> getEndOfInputCommittable()
ReturnsCheckpointCommittableManager
belonging to the last input.- Returns:
CheckpointCommittableManager
-
isFinished
public boolean isFinished()
Returns whether allCheckpointCommittableManager
currently hold by the collector are either committed or failed.- Returns:
- state of the
CheckpointCommittableManager
-
merge
public void merge(CommittableCollector<CommT> cc)
Merges all information from an external collector into this collector.This method is important during recovery from existing state.
- Parameters:
cc
- otherCommittableCollector
-
copy
public CommittableCollector<CommT> copy()
Returns a new committable collector that deep copies all internals.- Returns:
CommittableCollector
-
remove
public void remove(CheckpointCommittableManager<CommT> manager)
Removes the manager for a specific checkpoint and all it's metadata.
-
-