org.apache.flink.streaming.tests.artificialstate

Class ArtificalOperatorStateMapper<IN,OUT>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.api.common.functions.RichMapFunction<IN,OUT>

org.apache.flink.streaming.tests.artificialstate.ArtificalOperatorStateMapper<IN,OUT>

All Implemented Interfaces:

Serializable, Function, MapFunction<IN,OUT>, RichFunction, CheckpointedFunction

public class ArtificalOperatorStateMapper<IN,OUT>
extends RichMapFunction<IN,OUT>
implements CheckpointedFunction

A self-verifiable RichMapFunction used to verify checkpointing and restore semantics for various kinds of operator state.

For verifying broadcast state, the each subtask stores as broadcast state a map of (Integer, String) entries, key being the subtask index, and value being a String that corresponds to the subtask index. The total number of subtasks is also stored as broadcast state. On restore, each subtask should be restored with exactly the same broadcast state, with one entry for each subtask in the previous run.

For verifying union state, each subtask of this operator stores its own subtask index as a subset of the whole union state. On restore, each subtask's restored union state should have one entry for each subtask in the previous run.

All input elements to the operator arre simply passed through a user-provided map function and emitted.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ArtificalOperatorStateMapper(MapFunction<IN,OUT> mapFunction)

Method Summary

Modifier and Type	Method and Description
void	initializeState(FunctionInitializationContext context) This method is called when the parallel function instance is created during distributed execution.
OUT	map(IN value) The mapping method.
void	snapshotState(FunctionSnapshotContext context) This method is called when a snapshot for a checkpoint is requested.

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

close, getIterationRuntimeContext, getRuntimeContext, open, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ArtificalOperatorStateMapper

public ArtificalOperatorStateMapper(MapFunction<IN,OUT> mapFunction)

Method Detail

map

public OUT map(IN value)
        throws Exception

Description copied from interface: MapFunction

The mapping method. Takes an element from the input data set and transforms it into exactly one element.

Specified by:

map in interface MapFunction<IN,OUT>

Specified by:

map in class RichMapFunction<IN,OUT>

Parameters:

value - The input value.

Returns:

The transformed value

Throws:

Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.

initializeState

public void initializeState(FunctionInitializationContext context)
                     throws Exception

Description copied from interface: CheckpointedFunction

This method is called when the parallel function instance is created during distributed execution. Functions typically set up their state storing data structures in this method.

Specified by:

initializeState in interface CheckpointedFunction

Parameters:

context - the context for initializing the operator

Throws:

Exception - Thrown, if state could not be created ot restored.

snapshotState

public void snapshotState(FunctionSnapshotContext context)
                   throws Exception

Description copied from interface: CheckpointedFunction

This method is called when a snapshot for a checkpoint is requested. This acts as a hook to the function to ensure that all state is exposed by means previously offered through FunctionInitializationContext when the Function was initialized, or offered now by FunctionSnapshotContext itself.

Specified by:

snapshotState in interface CheckpointedFunction

Parameters:

context - the context for drawing a snapshot of the operator

Throws:

Exception - Thrown, if state could not be created ot restored.