org.apache.flink.api.common.functions

Interface RuntimeContext

All Known Subinterfaces:

IterationRuntimeContext

All Known Implementing Classes:

AbstractRuntimeUDFContext, DistributedRuntimeUDFContext, RuntimeUDFContext, StreamingRuntimeContext

@Public
public interface RuntimeContext

A RuntimeContext contains information about the context in which functions are executed. Each parallel instance of the function will have a context through which it can access static contextual information (such as the current parallelism) and other constructs like accumulators and broadcast variables.

A function can, during runtime, obtain the RuntimeContext via a call to AbstractRichFunction.getRuntimeContext().

Method Summary

Modifier and Type	Method and Description
<V,A extends Serializable> void	addAccumulator(String name, Accumulator<V,A> accumulator) Add this accumulator.
<V,A extends Serializable> Accumulator<V,A>	getAccumulator(String name) Get an existing accumulator object.
Map<String,Accumulator<?,?>>	getAllAccumulators() Deprecated. Use getAccumulator(..) to obtain the value of an accumulator.
int	getAttemptNumber() Gets the attempt number of this parallel subtask.
<RT> List<RT>	getBroadcastVariable(String name) Returns the result bound to the broadcast variable identified by the given name.
<T,C> C	getBroadcastVariableWithInitializer(String name, BroadcastVariableInitializer<T,C> initializer) Returns the result bound to the broadcast variable identified by the given name.
DistributedCache	getDistributedCache() Returns the DistributedCache to get the local temporary file copies of files otherwise not locally accessible.
DoubleCounter	getDoubleCounter(String name) Convenience function to create a counter object for doubles.
ExecutionConfig	getExecutionConfig() Returns the ExecutionConfig for the currently executing job.
Histogram	getHistogram(String name) Convenience function to create a counter object for histograms.
int	getIndexOfThisSubtask() Gets the number of this parallel subtask.
IntCounter	getIntCounter(String name) Convenience function to create a counter object for integers.
<S> OperatorState<S>	getKeyValueState(String name, Class<S> stateType, S defaultState) Deprecated. Use the more expressive getState(ValueStateDescriptor) instead.
<S> OperatorState<S>	getKeyValueState(String name, TypeInformation<S> stateType, S defaultState) Deprecated. Use the more expressive getState(ValueStateDescriptor) instead.
<T> ListState<T>	getListState(ListStateDescriptor<T> stateProperties) Gets a handle to the system's key/value list state.
LongCounter	getLongCounter(String name) Convenience function to create a counter object for longs.
int	getNumberOfParallelSubtasks() Gets the parallelism with which the parallel task runs.
<T> ReducingState<T>	getReducingState(ReducingStateDescriptor<T> stateProperties) Gets a handle to the system's key/value list state.
<T> ValueState<T>	getState(ValueStateDescriptor<T> stateProperties) Gets a handle to the system's key/value state.
String	getTaskName() Returns the name of the task in which the UDF runs, as assigned during plan construction.
String	getTaskNameWithSubtasks() Returns the name of the task, appended with the subtask indicator, such as "MyTask (3/6)", where 3 would be (getIndexOfThisSubtask() + 1), and 6 would be getNumberOfParallelSubtasks().
ClassLoader	getUserCodeClassLoader() Gets the ClassLoader to load classes that were are not in system's classpath, but are part of the jar file of a user job.

Method Detail

getTaskName

String getTaskName()

Returns the name of the task in which the UDF runs, as assigned during plan construction.

Returns:

The name of the task in which the UDF runs.

getNumberOfParallelSubtasks

int getNumberOfParallelSubtasks()

Gets the parallelism with which the parallel task runs.

Returns:

The parallelism with which the parallel task runs.

getIndexOfThisSubtask

int getIndexOfThisSubtask()

Gets the number of this parallel subtask. The numbering starts from 0 and goes up to parallelism-1 (parallelism as returned by getNumberOfParallelSubtasks()).

Returns:

The index of the parallel subtask.

getAttemptNumber

int getAttemptNumber()

Gets the attempt number of this parallel subtask. First attempt is numbered 0.

Returns:

Attempt number of the subtask.

getTaskNameWithSubtasks

String getTaskNameWithSubtasks()

Returns the name of the task, appended with the subtask indicator, such as "MyTask (3/6)", where 3 would be (getIndexOfThisSubtask() + 1), and 6 would be getNumberOfParallelSubtasks().

Returns:

The name of the task, with subtask indicator.

getExecutionConfig

ExecutionConfig getExecutionConfig()

Returns the ExecutionConfig for the currently executing job.

getUserCodeClassLoader

ClassLoader getUserCodeClassLoader()

Gets the ClassLoader to load classes that were are not in system's classpath, but are part of the jar file of a user job.

Returns:

The ClassLoader for user code classes.

addAccumulator

<V,A extends Serializable> void addAccumulator(String name,
                                               Accumulator<V,A> accumulator)

Add this accumulator. Throws an exception if the accumulator already exists in the same Task. Note that the Accumulator name must have an unique name across the Flink job. Otherwise you will get an error when incompatible accumulators from different Tasks are combined at the JobManager upon job completion.

getAccumulator

<V,A extends Serializable> Accumulator<V,A> getAccumulator(String name)

Get an existing accumulator object. The accumulator must have been added previously in this local runtime context. Throws an exception if the accumulator does not exist or if the accumulator exists, but with different type.

getAllAccumulators

@Deprecated
 @PublicEvolving
Map<String,Accumulator<?,?>> getAllAccumulators()

Deprecated. Use getAccumulator(..) to obtain the value of an accumulator.

Returns a map of all registered accumulators for this task. The returned map must not be modified.

getIntCounter

@PublicEvolving
IntCounter getIntCounter(String name)

Convenience function to create a counter object for integers.

getLongCounter

@PublicEvolving
LongCounter getLongCounter(String name)

Convenience function to create a counter object for longs.

getDoubleCounter

@PublicEvolving
DoubleCounter getDoubleCounter(String name)

Convenience function to create a counter object for doubles.

getHistogram

@PublicEvolving
Histogram getHistogram(String name)

Convenience function to create a counter object for histograms.

getBroadcastVariable

<RT> List<RT> getBroadcastVariable(String name)

Returns the result bound to the broadcast variable identified by the given name.

IMPORTANT: The broadcast variable data structure is shared between the parallel tasks on one machine. Any access that modifies its internal state needs to be manually synchronized by the caller.

Parameters:

name - The name under which the broadcast variable is registered;

Returns:

The broadcast variable, materialized as a list of elements.

getBroadcastVariableWithInitializer

<T,C> C getBroadcastVariableWithInitializer(String name,
                                            BroadcastVariableInitializer<T,C> initializer)

Returns the result bound to the broadcast variable identified by the given name. The broadcast variable is returned as a shared data structure that is initialized with the given BroadcastVariableInitializer.

IMPORTANT: The broadcast variable data structure is shared between the parallel tasks on one machine. Any access that modifies its internal state needs to be manually synchronized by the caller.

Parameters:

name - The name under which the broadcast variable is registered;

initializer - The initializer that creates the shared data structure of the broadcast variable from the sequence of elements.

Returns:

The broadcast variable, materialized as a list of elements.

getDistributedCache

DistributedCache getDistributedCache()

Returns the DistributedCache to get the local temporary file copies of files otherwise not locally accessible.

Returns:

The distributed cache of the worker executing this instance.

getState

@PublicEvolving
<T> ValueState<T> getState(ValueStateDescriptor<T> stateProperties)

Gets a handle to the system's key/value state. The key/value state is only accessible if the function is executed on a KeyedStream. On each access, the state exposes the value for the the key of the element currently processed by the function. Each function may have multiple partitioned states, addressed with different names.

Because the scope of each value is the key of the currently processed element, and the elements are distributed by the Flink runtime, the system can transparently scale out and redistribute the state and KeyedStream.

The following code example shows how to implement a continuous counter that counts how many times elements of a certain key occur, and emits an updated count for that element on each occurrence.

 DataStream<MyType> stream = ...;
 KeyedStream<MyType> keyedStream = stream.keyBy("id");

 keyedStream.map(new RichMapFunction<MyType, Tuple2<MyType, Long>>() {

     private ValueState<Long> count;

     public void open(Configuration cfg) {
         state = getRuntimeContext().getState(
                 new ValueStateDescriptor<Long>("count", LongSerializer.INSTANCE, 0L));
     }

     public Tuple2<MyType, Long> map(MyType value) {
         long count = state.value() + 1;
         state.update(value);
         return new Tuple2<>(value, count);
     }
 });
 

Type Parameters:

T - The type of value stored in the state.

Parameters:

stateProperties - The descriptor defining the properties of the stats.

Returns:

The partitioned state object.

Throws:

UnsupportedOperationException - Thrown, if no partitioned state is available for the function (function is not part of a KeyedStream).

getListState

@PublicEvolving
<T> ListState<T> getListState(ListStateDescriptor<T> stateProperties)

Gets a handle to the system's key/value list state. This state is similar to the state accessed via getState(ValueStateDescriptor), but is optimized for state that holds lists. One can adds elements to the list, or retrieve the list as a whole.

This state is only accessible if the function is executed on a KeyedStream.

 DataStream<MyType> stream = ...;
 KeyedStream<MyType> keyedStream = stream.keyBy("id");

 keyedStream.map(new RichFlatMapFunction<MyType, List<MyType>>() {

     private ListState<MyType> state;

     public void open(Configuration cfg) {
         state = getRuntimeContext().getListState(
                 new ListStateDescriptor<>("myState", MyType.class));
     }

     public void flatMap(MyType value, Collector<MyType> out) {
         if (value.isDivider()) {
             for (MyType t : state.get()) {
                 out.collect(t);
             }
         } else {
             state.add(value);
         }
     }
 });
 

Type Parameters:

T - The type of value stored in the state.

Parameters:

stateProperties - The descriptor defining the properties of the stats.

Returns:

The partitioned state object.

Throws:

UnsupportedOperationException - Thrown, if no partitioned state is available for the function (function is not part os a KeyedStream).

getReducingState

@PublicEvolving
<T> ReducingState<T> getReducingState(ReducingStateDescriptor<T> stateProperties)

Gets a handle to the system's key/value list state. This state is similar to the state accessed via getState(ValueStateDescriptor), but is optimized for state that aggregates values.

This state is only accessible if the function is executed on a KeyedStream.

 DataStream<MyType> stream = ...;
 KeyedStream<MyType> keyedStream = stream.keyBy("id");

 keyedStream.map(new RichMapFunction<MyType, List<MyType>>() {

     private ReducingState<Long> sum;

     public void open(Configuration cfg) {
         state = getRuntimeContext().getReducingState(
                 new ReducingStateDescriptor<>("sum", MyType.class, 0L, (a, b) -> a + b));
     }

     public Tuple2<MyType, Long> map(MyType value) {
         sum.add(value.count());
         return new Tuple2<>(value, sum.get());
     }
 });
 
 

Type Parameters:

T - The type of value stored in the state.

Parameters:

stateProperties - The descriptor defining the properties of the stats.

Returns:

The partitioned state object.

Throws:

UnsupportedOperationException - Thrown, if no partitioned state is available for the function (function is not part of a KeyedStream).

getKeyValueState

@Deprecated
 @PublicEvolving
<S> OperatorState<S> getKeyValueState(String name,
                                                                   Class<S> stateType,
                                                                   S defaultState)

Deprecated. Use the more expressive getState(ValueStateDescriptor) instead.

Gets the key/value state, which is only accessible if the function is executed on a KeyedStream. Upon calling ValueState.value(), the key/value state will return the value bound to the key of the element currently processed by the function. Each operator may maintain multiple key/value states, addressed with different names.

Because the scope of each value is the key of the currently processed element, and the elements are distributed by the Flink runtime, the system can transparently scale out and redistribute the state and KeyedStream.

The following code example shows how to implement a continuous counter that counts how many times elements of a certain key occur, and emits an updated count for that element on each occurrence.

 DataStream<MyType> stream = ...;
 KeyedStream<MyType> keyedStream = stream.keyBy("id");     

 keyedStream.map(new RichMapFunction<MyType, Tuple2<MyType, Long>>() {

     private State<Long> state;

     public void open(Configuration cfg) {
         state = getRuntimeContext().getKeyValueState(Long.class, 0L);
     }

     public Tuple2<MyType, Long> map(MyType value) {
         long count = state.value();
         state.update(value + 1);
         return new Tuple2<>(value, count);
     }
 });

 

This method attempts to deduce the type information from the given type class. If the full type cannot be determined from the class (for example because of generic parameters), the TypeInformation object must be manually passed via getKeyValueState(String, TypeInformation, Object).

Type Parameters:

S - The type of the state.

Parameters:

name - The name of the key/value state.

stateType - The class of the type that is stored in the state. Used to generate serializers for managed memory and checkpointing.

defaultState - The default state value, returned when the state is accessed and no value has yet been set for the key. May be null.

Returns:

The key/value state access.

Throws:

UnsupportedOperationException - Thrown, if no key/value state is available for the function (function is not part os a KeyedStream).

getKeyValueState

@Deprecated
 @PublicEvolving
<S> OperatorState<S> getKeyValueState(String name,
                                                                   TypeInformation<S> stateType,
                                                                   S defaultState)

Deprecated. Use the more expressive getState(ValueStateDescriptor) instead.

Gets the key/value state, which is only accessible if the function is executed on a KeyedStream. Upon calling ValueState.value(), the key/value state will return the value bound to the key of the element currently processed by the function. Each operator may maintain multiple key/value states, addressed with different names.

Because the scope of each value is the key of the currently processed element, and the elements are distributed by the Flink runtime, the system can transparently scale out and redistribute the state and KeyedStream.

The following code example shows how to implement a continuous counter that counts how many times elements of a certain key occur, and emits an updated count for that element on each occurrence.

 DataStream<MyType> stream = ...;
 KeyedStream<MyType> keyedStream = stream.keyBy("id");     
 
 keyedStream.map(new RichMapFunction<MyType, Tuple2<MyType, Long>>() {
 
     private State<Long> state;
     
     public void open(Configuration cfg) {
         state = getRuntimeContext().getKeyValueState(Long.class, 0L);
     }
     
     public Tuple2<MyType, Long> map(MyType value) {
         long count = state.value();
         state.update(value + 1);
         return new Tuple2<>(value, count);
     }
 });
     
 

Type Parameters:

S - The type of the state.

Parameters:

name - The name of the key/value state.

stateType - The type information for the type that is stored in the state. Used to create serializers for managed memory and checkpoints.

defaultState - The default state value, returned when the state is accessed and no value has yet been set for the key. May be null.

Returns:

The key/value state access.

Throws:

UnsupportedOperationException - Thrown, if no key/value state is available for the function (function is not part os a KeyedStream).