org.apache.flink.api.connector.sink

Interface Sink<InputT,CommT,WriterStateT,GlobalCommT>

Type Parameters:

InputT - The type of the sink's input

CommT - The type of information needed to commit data staged by the sink

WriterStateT - The type of the sink writer's state

GlobalCommT - The type of the aggregated committable

All Superinterfaces:

Serializable

Deprecated.

Please use Sink or a derivative.

@Deprecated
 @PublicEvolving
public interface Sink<InputT,CommT,WriterStateT,GlobalCommT>
extends Serializable

This interface lets the sink developer build a simple sink topology, which could guarantee the exactly once semantics in both batch and stream execution mode if there is a Committer or GlobalCommitter. 1. The SinkWriter is responsible for producing the committable. 2. The Committer is responsible for committing a single committable. 3. The GlobalCommitter is responsible for committing an aggregated committable, which we call the global committable. The GlobalCommitter is always executed with a parallelism of 1. Note: Developers need to ensure the idempotence of Committer and GlobalCommitter.

A sink must always have a writer, but committer and global committer are each optional and all combinations are valid.

The Sink needs to be serializable. All configuration should be validated eagerly. The respective sink parts are transient and will only be created in the subtasks on the taskmanagers.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	Sink.InitContext Deprecated. Please migrate to Sink and use Sink.InitContext.
static interface	Sink.ProcessingTimeService Deprecated. Please migrate to Sink and use ProcessingTimeService.

Method Summary

Modifier and Type	Method and Description
Optional<Committer<CommT>>	createCommitter() Deprecated. Creates a Committer which is part of a 2-phase-commit protocol.
Optional<GlobalCommitter<CommT,GlobalCommT>>	createGlobalCommitter() Deprecated. Creates a GlobalCommitter which is part of a 2-phase-commit protocol.
SinkWriter<InputT,CommT,WriterStateT>	createWriter(Sink.InitContext context, List<WriterStateT> states) Deprecated. Create a SinkWriter.
Optional<SimpleVersionedSerializer<CommT>>	getCommittableSerializer() Deprecated. Returns the serializer of the committable type.
default Collection<String>	getCompatibleStateNames() Deprecated. A list of state names of sinks from which the state can be restored.
Optional<SimpleVersionedSerializer<GlobalCommT>>	getGlobalCommittableSerializer() Deprecated. Returns the serializer of the aggregated committable type.
Optional<SimpleVersionedSerializer<WriterStateT>>	getWriterStateSerializer() Deprecated. Any stateful sink needs to provide this state serializer and implement SinkWriter.snapshotState(long) properly.

Method Detail

createWriter

SinkWriter<InputT,CommT,WriterStateT> createWriter(Sink.InitContext context,
                                                   List<WriterStateT> states)
                                            throws IOException

Deprecated.

Create a SinkWriter. If the application is resumed from a checkpoint or savepoint and the sink is stateful, it will receive the corresponding state obtained with SinkWriter.snapshotState(long) and serialized with getWriterStateSerializer(). If no state exists, the first existing, compatible state specified in getCompatibleStateNames() will be loaded and passed.

Parameters:

context - the runtime context.

states - the writer's previous state.

Returns:

A sink writer.

Throws:

IOException - for any failure during creation.

See Also:

SinkWriter.snapshotState(long), getWriterStateSerializer(), getCompatibleStateNames()

getWriterStateSerializer

Optional<SimpleVersionedSerializer<WriterStateT>> getWriterStateSerializer()

Deprecated.

Any stateful sink needs to provide this state serializer and implement SinkWriter.snapshotState(long) properly. The respective state is used in createWriter(InitContext, List) on recovery.

Returns:

the serializer of the writer's state type.

createCommitter

Optional<Committer<CommT>> createCommitter()
                                    throws IOException

Deprecated.

Creates a Committer which is part of a 2-phase-commit protocol. The SinkWriter creates committables through SinkWriter.prepareCommit(boolean) in the first phase. The committables are then passed to this committer and persisted with Committer.commit(List). If a committer is returned, the sink must also return a getCommittableSerializer().

Returns:

A committer for the 2-phase-commit protocol.

Throws:

IOException - for any failure during creation.

createGlobalCommitter

Optional<GlobalCommitter<CommT,GlobalCommT>> createGlobalCommitter()
                                                            throws IOException

Deprecated.

Creates a GlobalCommitter which is part of a 2-phase-commit protocol. The SinkWriter creates committables through SinkWriter.prepareCommit(boolean) in the first phase. The committables are then passed to the Committer and persisted with Committer.commit(List). The committables are also passed to this GlobalCommitter of which only a single instance exists. If a global committer is returned, the sink must also return a getCommittableSerializer() and getGlobalCommittableSerializer().

Returns:

A global committer for the 2-phase-commit protocol.

Throws:

IOException - for any failure during creation.

getCommittableSerializer

Optional<SimpleVersionedSerializer<CommT>> getCommittableSerializer()

Deprecated.

Returns the serializer of the committable type. The serializer is required iff the sink has a Committer or GlobalCommitter.

getGlobalCommittableSerializer

Optional<SimpleVersionedSerializer<GlobalCommT>> getGlobalCommittableSerializer()

Deprecated.

Returns the serializer of the aggregated committable type. The serializer is required iff the sink has a GlobalCommitter.

getCompatibleStateNames

default Collection<String> getCompatibleStateNames()

Deprecated.

A list of state names of sinks from which the state can be restored. For example, the new FileSink can resume from the state of an old StreamingFileSink as a drop-in replacement when resuming from a checkpoint/savepoint.