Package org.apache.flink.streaming.tests

Class FailureMapper<T>

  • All Implemented Interfaces:
    Serializable, Function, MapFunction<T,​T>, RichFunction, CheckpointListener
    public class FailureMapper<T>
extends RichMapFunction<T,​T>
implements CheckpointListener
    This mapper simulates failure by throwing exceptions. The timing to throw an exception by configuring the number of records to process, and number of complete checkpoints to be acknowledged before throwing the exception.

    The total times to simulate a failure across multiple execution attempts of the operator can also be configured. Note that this also takes into account failures that were not triggered by this mapper, e.g. TaskManager failures.

    See Also:
    Serialized Form

    • Constructor Detail

      • FailureMapper

        public FailureMapper​(long numProcessedRecordsFailureThreshold,
                     long numCompleteCheckpointsFailureThreshold,
                     int maxNumFailures)

    • Method Detail

      • map

        public T map​(T value)
      throws Exception
        Description copied from interface: MapFunction
        The mapping method. Takes an element from the input data set and transforms it into exactly one element.
        Specified by:
        map in interface MapFunction<T,​T>
        Specified by:
        map in class RichMapFunction<T,​T>
        Parameters:
        value - The input value.
        Returns:
        The transformed value
        Throws:
        Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.

      • notifyCheckpointComplete

        public void notifyCheckpointComplete​(long checkpointId)
                              throws Exception
        Description copied from interface: CheckpointListener
        Notifies the listener that the checkpoint with the given checkpointId completed and was committed.

        These notifications are "best effort", meaning they can sometimes be skipped. To behave properly, implementers need to follow the "Checkpoint Subsuming Contract". Please see the class-level JavaDocs for details.

        Please note that checkpoints may generally overlap, so you cannot assume that the notifyCheckpointComplete() call is always for the latest prior checkpoint (or snapshot) that was taken on the function/operator implementing this interface. It might be for a checkpoint that was triggered earlier. Implementing the "Checkpoint Subsuming Contract" (see above) properly handles this situation correctly as well.

        Please note that throwing exceptions from this method will not cause the completed checkpoint to be revoked. Throwing exceptions will typically cause task/job failure and trigger recovery.

        Specified by:
        notifyCheckpointComplete in interface CheckpointListener
        Parameters:
        checkpointId - The ID of the checkpoint that has been completed.
        Throws:
        Exception - This method can propagate exceptions, which leads to a failure/recovery for the task. Note that this will NOT lead to the checkpoint being revoked.

      • notifyCheckpointAborted

        public void notifyCheckpointAborted​(long checkpointId)
        Description copied from interface: CheckpointListener
        This method is called as a notification once a distributed checkpoint has been aborted.

        Important: The fact that a checkpoint has been aborted does NOT mean that the data and artifacts produced between the previous checkpoint and the aborted checkpoint are to be discarded. The expected behavior is as if this checkpoint was never triggered in the first place, and the next successful checkpoint simply covers a longer time span. See the "Checkpoint Subsuming Contract" in the class-level JavaDocs for details.

        These notifications are "best effort", meaning they can sometimes be skipped.

        This method is very rarely necessary to implement. The "best effort" guarantee, together with the fact that this method should not result in discarding any data (per the "Checkpoint Subsuming Contract") means it is mainly useful for earlier cleanups of auxiliary resources. One example is to pro-actively clear a local per-checkpoint state cache upon checkpoint failure.

        Specified by:
        notifyCheckpointAborted in interface CheckpointListener
        Parameters:
        checkpointId - The ID of the checkpoint that has been aborted.