org.apache.flink.streaming.api.datastream

Class DataStreamSink<T>

java.lang.Object

org.apache.flink.streaming.api.datastream.DataStreamSink<T>

Type Parameters:

T - The type of the elements in the Stream

Direct Known Subclasses:

CollectStreamSink

@Public
public class DataStreamSink<T>
extends Object

A Stream Sink. This is used for emitting elements from a streaming topology.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	DataStreamSink(PhysicalTransformation<T> transformation)

Method Summary

Modifier and Type	Method and Description
DataStreamSink<T>	disableChaining() Turns off chaining for this operator so thread co-location will not be used as an optimization.
static <T> DataStreamSink<T>	forSink(DataStream<T> inputStream, Sink<T> sink, CustomSinkOperatorUidHashes customSinkOperatorUidHashes)
static <T> DataStreamSink<T>	forSinkV1(DataStream<T> inputStream, Sink<T,?,?,?> sink, CustomSinkOperatorUidHashes customSinkOperatorUidHashes)
LegacySinkTransformation<T>	getLegacyTransformation()
Transformation<T>	getTransformation() Returns the transformation that contains the actual sink operator of this sink.
DataStreamSink<T>	name(String name) Sets the name of this sink.
DataStreamSink<T>	setDescription(String description) Sets the description for this sink.
DataStreamSink<T>	setParallelism(int parallelism) Sets the parallelism for this sink.
DataStreamSink<T>	setUidHash(String uidHash) Sets an user provided hash for this operator.
DataStreamSink<T>	slotSharingGroup(SlotSharingGroup slotSharingGroup) Sets the slot sharing group of this operation.
DataStreamSink<T>	slotSharingGroup(String slotSharingGroup) Sets the slot sharing group of this operation.
DataStreamSink<T>	uid(String uid) Sets an ID for this operator.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DataStreamSink

protected DataStreamSink(PhysicalTransformation<T> transformation)

Method Detail

forSink

@Internal
public static <T> DataStreamSink<T> forSink(DataStream<T> inputStream,
                                                      Sink<T> sink,
                                                      CustomSinkOperatorUidHashes customSinkOperatorUidHashes)

forSinkV1

@Internal
public static <T> DataStreamSink<T> forSinkV1(DataStream<T> inputStream,
                                                        Sink<T,?,?,?> sink,
                                                        CustomSinkOperatorUidHashes customSinkOperatorUidHashes)

getTransformation

@Internal
public Transformation<T> getTransformation()

Returns the transformation that contains the actual sink operator of this sink.

getLegacyTransformation

@Internal
public LegacySinkTransformation<T> getLegacyTransformation()

name

public DataStreamSink<T> name(String name)

Sets the name of this sink. This name is used by the visualization and logging during runtime.

Returns:

The named sink.

uid

@PublicEvolving
public DataStreamSink<T> uid(String uid)

Sets an ID for this operator.

The specified ID is used to assign the same operator ID across job submissions (for example when starting a job from a savepoint).

Important: this ID needs to be unique per transformation and job. Otherwise, job submission will fail.

Parameters:

uid - The unique user-specified ID of this transformation.

Returns:

The operator with the specified ID.

setUidHash

@PublicEvolving
public DataStreamSink<T> setUidHash(String uidHash)

Sets an user provided hash for this operator. This will be used AS IS the create the JobVertexID.

The user provided hash is an alternative to the generated hashes, that is considered when identifying an operator through the default hash mechanics fails (e.g. because of changes between Flink versions).

Important: this should be used as a workaround or for trouble shooting. The provided hash needs to be unique per transformation and job. Otherwise, job submission will fail. Furthermore, you cannot assign user-specified hash to intermediate nodes in an operator chain and trying so will let your job fail.

A use case for this is in migration between Flink versions or changing the jobs in a way that changes the automatically generated hashes. In this case, providing the previous hashes directly through this method (e.g. obtained from old logs) can help to reestablish a lost mapping from states to their target operator.

Parameters:

uidHash - The user provided hash for this operator. This will become the JobVertexID, which is shown in the logs and web ui.

Returns:

The operator with the user provided hash.

setParallelism

public DataStreamSink<T> setParallelism(int parallelism)

Sets the parallelism for this sink. The degree must be higher than zero.

Parameters:

parallelism - The parallelism for this sink.

Returns:

The sink with set parallelism.

setDescription

@PublicEvolving
public DataStreamSink<T> setDescription(String description)

Sets the description for this sink.

Description is used in json plan and web ui, but not in logging and metrics where only name is available. Description is expected to provide detailed information about the sink, while name is expected to be more simple, providing summary information only, so that we can have more user-friendly logging messages and metric tags without losing useful messages for debugging.

Parameters:

description - The description for this sink.

Returns:

The sink with new description.

disableChaining

@PublicEvolving
public DataStreamSink<T> disableChaining()

Turns off chaining for this operator so thread co-location will not be used as an optimization.

Chaining can be turned off for the whole job by StreamExecutionEnvironment.disableOperatorChaining() however it is not advised for performance considerations.

Returns:

The sink with chaining disabled

slotSharingGroup

@PublicEvolving
public DataStreamSink<T> slotSharingGroup(String slotSharingGroup)

Sets the slot sharing group of this operation. Parallel instances of operations that are in the same slot sharing group will be co-located in the same TaskManager slot, if possible.

Operations inherit the slot sharing group of input operations if all input operations are in the same slot sharing group and no slot sharing group was explicitly specified.

Initially an operation is in the default slot sharing group. An operation can be put into the default group explicitly by setting the slot sharing group to "default".

Parameters:

slotSharingGroup - The slot sharing group name.

slotSharingGroup

@PublicEvolving
public DataStreamSink<T> slotSharingGroup(SlotSharingGroup slotSharingGroup)

Sets the slot sharing group of this operation. Parallel instances of operations that are in the same slot sharing group will be co-located in the same TaskManager slot, if possible.

Operations inherit the slot sharing group of input operations if all input operations are in the same slot sharing group and no slot sharing group was explicitly specified.

Initially an operation is in the default slot sharing group. An operation can be put into the default group explicitly by setting the slot sharing group with name "default".

Parameters:

slotSharingGroup - which contains name and its resource spec.