InputT
- The type of the sink's inputCommT
- The type of the committables.@PublicEvolving public interface TwoPhaseCommittingSink<InputT,CommT> extends Sink<InputT>
Sink
for exactly-once semantics using a two-phase commit protocol. The Sink
consists of a SinkWriter
that performs the precommits and a Committer
that
actually commits the data. To facilitate the separation the SinkWriter
creates
committables on checkpoint or end of input and the sends it to the Committer
.
The TwoPhaseCommittingSink
needs to be serializable. All configuration should be
validated eagerly. The respective sink writers and committers are transient and will only be
created in the subtasks on the taskmanagers.
Modifier and Type | Interface and Description |
---|---|
static interface |
TwoPhaseCommittingSink.PrecommittingSinkWriter<InputT,CommT>
A
SinkWriter that performs the first part of a two-phase commit protocol. |
Sink.InitContext
Modifier and Type | Method and Description |
---|---|
Committer<CommT> |
createCommitter()
Creates a
Committer that permanently makes the previously written data visible
through Committer.commit(Collection) . |
TwoPhaseCommittingSink.PrecommittingSinkWriter<InputT,CommT> |
createWriter(Sink.InitContext context)
Creates a
TwoPhaseCommittingSink.PrecommittingSinkWriter that creates committables on checkpoint or end of
input. |
SimpleVersionedSerializer<CommT> |
getCommittableSerializer()
Returns the serializer of the committable type.
|
TwoPhaseCommittingSink.PrecommittingSinkWriter<InputT,CommT> createWriter(Sink.InitContext context) throws IOException
TwoPhaseCommittingSink.PrecommittingSinkWriter
that creates committables on checkpoint or end of
input.createWriter
in interface Sink<InputT>
context
- the runtime context.IOException
- for any failure during creation.Committer<CommT> createCommitter() throws IOException
Committer
that permanently makes the previously written data visible
through Committer.commit(Collection)
.IOException
- for any failure during creation.SimpleVersionedSerializer<CommT> getCommittableSerializer()
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.