Package org.apache.flink.state.api.runtime

Class SavepointRuntimeContext

  • All Implemented Interfaces:
    RuntimeContext
    @Internal
public final class SavepointRuntimeContext
extends Object
implements RuntimeContext
    A streaming RuntimeContext which delegates to the underlying batch RuntimeContext along with a specified KeyedStateStore.

    This RuntimeContext has the ability to force eager state registration by throwing an exception if state is registered outside of open.

    • Method Detail

      • createSerializer

        public <T> TypeSerializer<T> createSerializer​(TypeInformation<T> typeInformation)
        Description copied from interface: RuntimeContext
        Create a serializer for a given type.
        Specified by:
        createSerializer in interface RuntimeContext
        Parameters:
        typeInformation - the type information of the object to be serialized
        Returns:
        the serializer for the given type

      • isObjectReuseEnabled

        public boolean isObjectReuseEnabled()
        Description copied from interface: RuntimeContext
        Check if object reuse is enabled.
        Specified by:
        isObjectReuseEnabled in interface RuntimeContext
        Returns:
        true if object reuse is enabled, false otherwise

      • getUserCodeClassLoader

        public ClassLoader getUserCodeClassLoader()
        Description copied from interface: RuntimeContext
        Gets the ClassLoader to load classes that are not in system's classpath, but are part of the jar file of a user job.
        Specified by:
        getUserCodeClassLoader in interface RuntimeContext
        Returns:
        The ClassLoader for user code classes.

      • registerUserCodeClassLoaderReleaseHookIfAbsent

        public void registerUserCodeClassLoaderReleaseHookIfAbsent​(String releaseHookName,
                                                           Runnable releaseHook)
        Description copied from interface: RuntimeContext
        Registers a custom hook for the user code class loader release.

        The release hook is executed just before the user code class loader is being released. Registration only happens if no hook has been registered under this name already.

        Specified by:
        registerUserCodeClassLoaderReleaseHookIfAbsent in interface RuntimeContext
        Parameters:
        releaseHookName - name of the release hook.
        releaseHook - release hook which is executed just before the user code class loader is being released

      • addAccumulator

        public <V,​A extends Serializable> void addAccumulator​(String name,
                                                            Accumulator<V,​A> accumulator)
        Description copied from interface: RuntimeContext
        Add this accumulator. Throws an exception if the accumulator already exists in the same Task. Note that the Accumulator name must have an unique name across the Flink job. Otherwise you will get an error when incompatible accumulators from different Tasks are combined at the JobManager upon job completion.
        Specified by:
        addAccumulator in interface RuntimeContext

      • getAccumulator

        public <V,​A extends Serializable> Accumulator<V,​A> getAccumulator​(String name)
        Description copied from interface: RuntimeContext
        Get an existing accumulator object. The accumulator must have been added previously in this local runtime context.

        Throws an exception if the accumulator does not exist or if the accumulator exists, but with different type.

        Specified by:
        getAccumulator in interface RuntimeContext

      • getExternalResourceInfos

        public Set<ExternalResourceInfo> getExternalResourceInfos​(String resourceName)
        Description copied from interface: RuntimeContext
        Get the specific external resource information by the resourceName.
        Specified by:
        getExternalResourceInfos in interface RuntimeContext
        Parameters:
        resourceName - of the required external resource
        Returns:
        information set of the external resource identified by the resourceName

      • hasBroadcastVariable

        public boolean hasBroadcastVariable​(String name)
        Description copied from interface: RuntimeContext
        Tests for the existence of the broadcast variable identified by the given name.
        Specified by:
        hasBroadcastVariable in interface RuntimeContext
        Parameters:
        name - The name under which the broadcast variable is registered;
        Returns:
        Whether a broadcast variable exists for the given name.

      • getBroadcastVariable

        public <RT> List<RT> getBroadcastVariable​(String name)
        Description copied from interface: RuntimeContext
        Returns the result bound to the broadcast variable identified by the given name.

        IMPORTANT: The broadcast variable data structure is shared between the parallel tasks on one machine. Any access that modifies its internal state needs to be manually synchronized by the caller.

        Specified by:
        getBroadcastVariable in interface RuntimeContext
        Parameters:
        name - The name under which the broadcast variable is registered;
        Returns:
        The broadcast variable, materialized as a list of elements.

      • getBroadcastVariableWithInitializer

        public <T,​C> C getBroadcastVariableWithInitializer​(String name,
                                                         BroadcastVariableInitializer<T,​C> initializer)
        Description copied from interface: RuntimeContext
        Returns the result bound to the broadcast variable identified by the given name. The broadcast variable is returned as a shared data structure that is initialized with the given BroadcastVariableInitializer.

        IMPORTANT: The broadcast variable data structure is shared between the parallel tasks on one machine. Any access that modifies its internal state needs to be manually synchronized by the caller.

        Specified by:
        getBroadcastVariableWithInitializer in interface RuntimeContext
        Parameters:
        name - The name under which the broadcast variable is registered;
        initializer - The initializer that creates the shared data structure of the broadcast variable from the sequence of elements.
        Returns:
        The broadcast variable, materialized as a list of elements.

      • getState

        public <T> ValueState<T> getState​(ValueStateDescriptor<T> stateProperties)
        Description copied from interface: RuntimeContext
        Gets a handle to the system's key/value state. The key/value state is only accessible if the function is executed on a KeyedStream. On each access, the state exposes the value for the key of the element currently processed by the function. Each function may have multiple partitioned states, addressed with different names.

        Because the scope of each value is the key of the currently processed element, and the elements are distributed by the Flink runtime, the system can transparently scale out and redistribute the state and KeyedStream.

        The following code example shows how to implement a continuous counter that counts how many times elements of a certain key occur, and emits an updated count for that element on each occurrence. 

        
 DataStream<MyType> stream = ...;
 KeyedStream<MyType> keyedStream = stream.keyBy("id");

 keyedStream.map(new RichMapFunction<MyType, Tuple2<MyType, Long>>() {

     private ValueState<Long> state;

     public void open(OpenContext ctx) {
         state = getRuntimeContext().getState(
                 new ValueStateDescriptor<Long>("count", LongSerializer.INSTANCE, 0L));
     }

     public Tuple2<MyType, Long> map(MyType value) {
         long count = state.value() + 1;
         state.update(count);
         return new Tuple2<>(value, count);
     }
 });
        Specified by:
        getState in interface RuntimeContext
        Type Parameters:
        T - The type of value stored in the state.
        Parameters:
        stateProperties - The descriptor defining the properties of the stats.
        Returns:
        The partitioned state object.

      • getListState

        public <T> ListState<T> getListState​(ListStateDescriptor<T> stateProperties)
        Description copied from interface: RuntimeContext
        Gets a handle to the system's key/value list state. This state is similar to the state accessed via RuntimeContext.getState(ValueStateDescriptor), but is optimized for state that holds lists. One can add elements to the list, or retrieve the list as a whole.

        This state is only accessible if the function is executed on a KeyedStream. 

        
 DataStream<MyType> stream = ...;
 KeyedStream<MyType> keyedStream = stream.keyBy("id");

 keyedStream.map(new RichFlatMapFunction<MyType, List<MyType>>() {

     private ListState<MyType> state;

     public void open(OpenContext ctx) {
         state = getRuntimeContext().getListState(
                 new ListStateDescriptor<>("myState", MyType.class));
     }

     public void flatMap(MyType value, Collector<MyType> out) {
         if (value.isDivider()) {
             for (MyType t : state.get()) {
                 out.collect(t);
             }
         } else {
             state.add(value);
         }
     }
 });
        Specified by:
        getListState in interface RuntimeContext
        Type Parameters:
        T - The type of value stored in the state.
        Parameters:
        stateProperties - The descriptor defining the properties of the stats.
        Returns:
        The partitioned state object.

      • getReducingState

        public <T> ReducingState<T> getReducingState​(ReducingStateDescriptor<T> stateProperties)
        Description copied from interface: RuntimeContext
        Gets a handle to the system's key/value reducing state. This state is similar to the state accessed via RuntimeContext.getState(ValueStateDescriptor), but is optimized for state that aggregates values.

        This state is only accessible if the function is executed on a KeyedStream. 

        
 DataStream<MyType> stream = ...;
 KeyedStream<MyType> keyedStream = stream.keyBy("id");

 keyedStream.map(new RichMapFunction<MyType, List<MyType>>() {

     private ReducingState<Long> state;

     public void open(OpenContext ctx) {
         state = getRuntimeContext().getReducingState(
                 new ReducingStateDescriptor<>("sum", (a, b) -> a + b, Long.class));
     }

     public Tuple2<MyType, Long> map(MyType value) {
         state.add(value.count());
         return new Tuple2<>(value, state.get());
     }
 });
        Specified by:
        getReducingState in interface RuntimeContext
        Type Parameters:
        T - The type of value stored in the state.
        Parameters:
        stateProperties - The descriptor defining the properties of the stats.
        Returns:
        The partitioned state object.

      • getAggregatingState

        public <IN,​ACC,​OUT> AggregatingState<IN,​OUT> getAggregatingState​(AggregatingStateDescriptor<IN,​ACC,​OUT> stateProperties)
        Description copied from interface: RuntimeContext
        Gets a handle to the system's key/value aggregating state. This state is similar to the state accessed via RuntimeContext.getState(ValueStateDescriptor), but is optimized for state that aggregates values with different types.

        This state is only accessible if the function is executed on a KeyedStream. 

        
 DataStream<MyType> stream = ...;
 KeyedStream<MyType> keyedStream = stream.keyBy("id");
 AggregateFunction<...> aggregateFunction = ...

 keyedStream.map(new RichMapFunction<MyType, List<MyType>>() {

     private AggregatingState<MyType, Long> state;

     public void open(OpenContext ctx) {
         state = getRuntimeContext().getAggregatingState(
                 new AggregatingStateDescriptor<>("sum", aggregateFunction, Long.class));
     }

     public Tuple2<MyType, Long> map(MyType value) {
         state.add(value);
         return new Tuple2<>(value, state.get());
     }
 });
        Specified by:
        getAggregatingState in interface RuntimeContext
        Type Parameters:
        IN - The type of the values that are added to the state.
        ACC - The type of the accumulator (intermediate aggregation state).
        OUT - The type of the values that are returned from the state.
        Parameters:
        stateProperties - The descriptor defining the properties of the stats.
        Returns:
        The partitioned state object.

      • getMapState

        public <UK,​UV> MapState<UK,​UV> getMapState​(MapStateDescriptor<UK,​UV> stateProperties)
        Description copied from interface: RuntimeContext
        Gets a handle to the system's key/value map state. This state is similar to the state accessed via RuntimeContext.getState(ValueStateDescriptor), but is optimized for state that is composed of user-defined key-value pairs

        This state is only accessible if the function is executed on a KeyedStream. 

        
 DataStream<MyType> stream = ...;
 KeyedStream<MyType> keyedStream = stream.keyBy("id");

 keyedStream.map(new RichMapFunction<MyType, List<MyType>>() {

     private MapState<MyType, Long> state;

     public void open(OpenContext ctx) {
         state = getRuntimeContext().getMapState(
                 new MapStateDescriptor<>("sum", MyType.class, Long.class));
     }

     public Tuple2<MyType, Long> map(MyType value) {
         return new Tuple2<>(value, state.get(value));
     }
 });
        Specified by:
        getMapState in interface RuntimeContext
        Type Parameters:
        UK - The type of the user keys stored in the state.
        UV - The type of the user values stored in the state.
        Parameters:
        stateProperties - The descriptor defining the properties of the stats.
        Returns:
        The partitioned state object.

      • disableStateRegistration

        public void disableStateRegistration()
                              throws Exception
        Throws:
        Exception