org.apache.flink.streaming.runtime.watermarkstatus

Class StatusWatermarkValve

java.lang.Object

org.apache.flink.streaming.runtime.watermarkstatus.StatusWatermarkValve

@Internal
public class StatusWatermarkValve
extends Object

A StatusWatermarkValve embodies the logic of how Watermark and WatermarkStatus are propagated to downstream outputs, given a set of one or multiple subpartitions that continuously receive them. Usages of this class need to define the number of subpartitions that the valve needs to handle, as well as provide a implementation of PushingAsyncDataInput.DataOutput, which is called by the valve only when it determines a new watermark or watermark status can be propagated.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	StatusWatermarkValve.SubpartitionStatus An SubpartitionStatus keeps track of a subpartition's last watermark, stream status, and whether or not the subpartition's current watermark is aligned with the overall watermark output from the valve.

Constructor Summary

Constructors

Constructor and Description
StatusWatermarkValve(CheckpointedInputGate inputGate)
StatusWatermarkValve(int numInputChannels) Returns a new StatusWatermarkValve.
StatusWatermarkValve(ResultSubpartitionIndexSet[] subpartitionIndexSets)

Method Summary

Modifier and Type	Method and Description
protected StatusWatermarkValve.SubpartitionStatus	getSubpartitionStatus(int subpartitionIndex)
void	inputWatermark(Watermark watermark, int channelIndex, PushingAsyncDataInput.DataOutput<?> output) Feed a Watermark into the valve.
void	inputWatermarkStatus(WatermarkStatus watermarkStatus, int channelIndex, PushingAsyncDataInput.DataOutput<?> output) Feed a WatermarkStatus into the valve.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StatusWatermarkValve

@VisibleForTesting
public StatusWatermarkValve(int numInputChannels)

Returns a new StatusWatermarkValve.

Parameters:

numInputChannels - the number of input channels that this valve will need to handle

StatusWatermarkValve

public StatusWatermarkValve(CheckpointedInputGate inputGate)

StatusWatermarkValve

public StatusWatermarkValve(ResultSubpartitionIndexSet[] subpartitionIndexSets)

Method Detail

inputWatermark

public void inputWatermark(Watermark watermark,
                           int channelIndex,
                           PushingAsyncDataInput.DataOutput<?> output)
                    throws Exception

Feed a Watermark into the valve. If the input triggers the valve to output a new Watermark, PushingAsyncDataInput.DataOutput.emitWatermark(Watermark) will be called to process the new Watermark.

Parameters:

watermark - the watermark to feed to the valve

channelIndex - the index of the channel that the fed watermark belongs to (index starting from 0)

Throws:

Exception

inputWatermarkStatus

public void inputWatermarkStatus(WatermarkStatus watermarkStatus,
                                 int channelIndex,
                                 PushingAsyncDataInput.DataOutput<?> output)
                          throws Exception

Feed a WatermarkStatus into the valve. This may trigger the valve to output either a new Watermark Status, for which PushingAsyncDataInput.DataOutput.emitWatermarkStatus(WatermarkStatus) will be called, or a new Watermark, for which PushingAsyncDataInput.DataOutput.emitWatermark(Watermark) will be called.

Parameters:

watermarkStatus - the watermark status to feed to the valve

channelIndex - the index of the channel that the fed watermark status belongs to (index starting from 0)

Throws:

Exception

getSubpartitionStatus

@VisibleForTesting
protected StatusWatermarkValve.SubpartitionStatus getSubpartitionStatus(int subpartitionIndex)