InputT
- The type of the sink's inputCommT
- The type of information needed to commit data staged by the sinkWriterStateT
- The type of the sink writer's stateGlobalCommT
- The type of the aggregated committable@Experimental public interface Sink<InputT,CommT,WriterStateT,GlobalCommT> extends Serializable
Committer
or
GlobalCommitter
. 1. The SinkWriter
is responsible for producing the committable.
2. The Committer
is responsible for committing a single committable. 3. The GlobalCommitter
is responsible for committing an aggregated committable, which we call the
global committable. The GlobalCommitter
is always executed with a parallelism of 1. Note:
Developers need to ensure the idempotence of Committer
and GlobalCommitter
.Modifier and Type | Interface and Description |
---|---|
static interface |
Sink.InitContext
The interface exposes some runtime info for creating a
SinkWriter . |
static interface |
Sink.ProcessingTimeService
A service that allows to get the current processing time and register timers that will
execute the given
Sink.ProcessingTimeService.ProcessingTimeCallback when firing. |
Modifier and Type | Method and Description |
---|---|
Optional<Committer<CommT>> |
createCommitter()
Creates a
Committer . |
Optional<GlobalCommitter<CommT,GlobalCommT>> |
createGlobalCommitter()
Creates a
GlobalCommitter . |
SinkWriter<InputT,CommT,WriterStateT> |
createWriter(Sink.InitContext context,
List<WriterStateT> states)
Create a
SinkWriter . |
Optional<SimpleVersionedSerializer<CommT>> |
getCommittableSerializer()
Returns the serializer of the committable type.
|
Optional<SimpleVersionedSerializer<GlobalCommT>> |
getGlobalCommittableSerializer()
Returns the serializer of the aggregated committable type.
|
Optional<SimpleVersionedSerializer<WriterStateT>> |
getWriterStateSerializer()
Return the serializer of the writer's state type.
|
SinkWriter<InputT,CommT,WriterStateT> createWriter(Sink.InitContext context, List<WriterStateT> states) throws IOException
SinkWriter
.context
- the runtime context.states
- the writer's state.IOException
- if fail to create a writer.Optional<Committer<CommT>> createCommitter() throws IOException
Committer
.IOException
- if fail to create a committer.Optional<GlobalCommitter<CommT,GlobalCommT>> createGlobalCommitter() throws IOException
GlobalCommitter
.IOException
- if fail to create a global committer.Optional<SimpleVersionedSerializer<CommT>> getCommittableSerializer()
Optional<SimpleVersionedSerializer<GlobalCommT>> getGlobalCommittableSerializer()
Optional<SimpleVersionedSerializer<WriterStateT>> getWriterStateSerializer()
Copyright © 2014–2021 The Apache Software Foundation. All rights reserved.