org.apache.flink.streaming.connectors.gcp.pubsub.common

Class AcknowledgeOnCheckpoint<ACKID extends Serializable>

java.lang.Object

org.apache.flink.streaming.connectors.gcp.pubsub.common.AcknowledgeOnCheckpoint<ACKID>

Type Parameters:

ACKID - Type of Ids to acknowledge

All Implemented Interfaces:

CheckpointListener, ListCheckpointed<AcknowledgeIdsForCheckpoint<ACKID>>

public class AcknowledgeOnCheckpoint<ACKID extends Serializable>
extends Object
implements CheckpointListener, ListCheckpointed<AcknowledgeIdsForCheckpoint<ACKID>>

Helper class for SourceFunctions to acknowledge messages to external systems after a successful checkpoint.

The mechanism for this source assumes that messages are identified by a unique ID. When messages are taken from the message queue, the message must not be dropped immediately from the external system, but must be retained until acknowledged. Messages that are not acknowledged within a certain time interval will be served again (to a different connection, established by the recovered source).

Note that this source can give no guarantees about message order in the case of failures, because messages that were retrieved but not yet acknowledged will be returned later again, after a set of messages that was not retrieved before the failure.

Internally, this class gathers the IDs of elements it emits. Per checkpoint, the IDs are stored and acknowledged when the checkpoint is complete. That way, no message is acknowledged unless it is certain that it has been successfully processed throughout the topology and the updates to any state caused by that message are persistent.

Constructor Summary

Constructors

Constructor and Description
AcknowledgeOnCheckpoint(Acknowledger<ACKID> acknowledger)

Method Summary

Modifier and Type	Method and Description
void	addAcknowledgeId(ACKID id)
void	notifyCheckpointAborted(long checkpointId) This method is called as a notification once a distributed checkpoint has been aborted.
void	notifyCheckpointComplete(long checkpointId) This method is called as a notification once a distributed checkpoint has been completed.
int	numberOfOutstandingAcknowledgements()
void	restoreState(List<AcknowledgeIdsForCheckpoint<ACKID>> state) Restores the state of the function or operator to that of a previous checkpoint.
List<AcknowledgeIdsForCheckpoint<ACKID>>	snapshotState(long checkpointId, long timestamp) Gets the current state of the function.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AcknowledgeOnCheckpoint

public AcknowledgeOnCheckpoint(Acknowledger<ACKID> acknowledger)

Method Detail

addAcknowledgeId

public void addAcknowledgeId(ACKID id)

notifyCheckpointComplete

public void notifyCheckpointComplete(long checkpointId)
                              throws Exception

Description copied from interface: CheckpointListener

This method is called as a notification once a distributed checkpoint has been completed.

Note that any exception during this method will not cause the checkpoint to fail any more.

Specified by:

notifyCheckpointComplete in interface CheckpointListener

Parameters:

checkpointId - The ID of the checkpoint that has been completed.

Throws:

Exception - This method can propagate exceptions, which leads to a failure/recovery for the task. Not that this will NOT lead to the checkpoint being revoked.

notifyCheckpointAborted

public void notifyCheckpointAborted(long checkpointId)

Description copied from interface: CheckpointListener

This method is called as a notification once a distributed checkpoint has been aborted.

Specified by:

notifyCheckpointAborted in interface CheckpointListener

Parameters:

checkpointId - The ID of the checkpoint that has been aborted.

snapshotState

public List<AcknowledgeIdsForCheckpoint<ACKID>> snapshotState(long checkpointId,
                                                              long timestamp)
                                                       throws Exception

Description copied from interface: ListCheckpointed

Gets the current state of the function. The state must reflect the result of all prior invocations to this function.

The returned list should contain one entry for redistributable unit of state. See the class docs for an illustration how list-style state redistribution works.

As special case, the returned list may be null or empty (if the operator has no state) or it may contain a single element (if the operator state is indivisible).

Specified by:

snapshotState in interface ListCheckpointed<AcknowledgeIdsForCheckpoint<ACKID extends Serializable>>

Parameters:

checkpointId - The ID of the checkpoint - a unique and monotonously increasing value.

timestamp - The wall clock timestamp when the checkpoint was triggered by the master.

Returns:

The operator state in a list of redistributable, atomic sub-states. Should not return null, but empty list instead.

Throws:

Exception - Thrown if the creation of the state object failed. This causes the checkpoint to fail. The system may decide to fail the operation (and trigger recovery), or to discard this checkpoint attempt and to continue running and to try again with the next checkpoint attempt.

restoreState

public void restoreState(List<AcknowledgeIdsForCheckpoint<ACKID>> state)
                  throws Exception

Description copied from interface: ListCheckpointed

Restores the state of the function or operator to that of a previous checkpoint. This method is invoked when the function is executed after a failure recovery. The state list may be empty if no state is to be recovered by the particular parallel instance of the function.

The given state list will contain all the sub states that this parallel instance of the function needs to handle. Refer to the class docs for an illustration how list-style state redistribution works.

Important: When implementing this interface together with RichFunction, then the restoreState() method is called before RichFunction.open(Configuration).

Specified by:

restoreState in interface ListCheckpointed<AcknowledgeIdsForCheckpoint<ACKID extends Serializable>>

Parameters:

state - The state to be restored as a list of atomic sub-states.

Throws:

Exception - Throwing an exception in this method causes the recovery to fail. The exact consequence depends on the configured failure handling strategy, but typically the system will re-attempt the recovery, or try recovering from a different checkpoint.

numberOfOutstandingAcknowledgements

public int numberOfOutstandingAcknowledgements()