@PublicEvolving public class StreamingRuntimeContext extends AbstractRuntimeUDFContext
RuntimeContext
,
for streaming operators.Constructor and Description |
---|
StreamingRuntimeContext(AbstractStreamOperator<?> operator,
Environment env,
Map<String,Accumulator<?,?>> accumulators) |
Modifier and Type | Method and Description |
---|---|
<RT> List<RT> |
getBroadcastVariable(String name)
Returns the result bound to the broadcast variable identified by the
given
name . |
<T,C> C |
getBroadcastVariableWithInitializer(String name,
BroadcastVariableInitializer<T,C> initializer)
Returns the result bound to the broadcast variable identified by the
given
name . |
long |
getBufferTimeout()
Returns the buffer timeout of the job
|
CheckpointingMode |
getCheckpointMode()
Returns the checkpointing mode
|
long |
getCurrentProcessingTime()
Returns the current processing time as defined by the task's
TimeServiceProvider |
InputSplitProvider |
getInputSplitProvider()
Returns the input split provider associated with the operator.
|
<S> OperatorState<S> |
getKeyValueState(String name,
Class<S> stateType,
S defaultState)
Deprecated.
|
<S> OperatorState<S> |
getKeyValueState(String name,
TypeInformation<S> stateType,
S defaultState)
Deprecated.
|
<T> ListState<T> |
getListState(ListStateDescriptor<T> stateProperties)
Gets a handle to the system's key/value list state.
|
<T> ReducingState<T> |
getReducingState(ReducingStateDescriptor<T> stateProperties)
Gets a handle to the system's key/value list state.
|
<T> ValueState<T> |
getState(ValueStateDescriptor<T> stateProperties)
Gets a handle to the system's key/value state.
|
boolean |
hasBroadcastVariable(String name)
Tests for the existence of the broadcast variable identified by the
given
name . |
boolean |
isCheckpointingEnabled()
Returns true if checkpointing is enabled for the running job.
|
ScheduledFuture<?> |
registerTimer(long time,
Triggerable target)
Register a timer callback.
|
addAccumulator, getAccumulator, getAllAccumulators, getAttemptNumber, getDistributedCache, getDoubleCounter, getExecutionConfig, getHistogram, getIndexOfThisSubtask, getIntCounter, getLongCounter, getMetricGroup, getNumberOfParallelSubtasks, getTaskName, getTaskNameWithSubtasks, getUserCodeClassLoader
public StreamingRuntimeContext(AbstractStreamOperator<?> operator, Environment env, Map<String,Accumulator<?,?>> accumulators)
public InputSplitProvider getInputSplitProvider()
public ScheduledFuture<?> registerTimer(long time, Triggerable target)
Triggerable
will be invoked.
This call is guaranteed to not happen concurrently with method calls on the operator.time
- The absolute time in milliseconds.target
- The target to be triggered.public long getCurrentProcessingTime()
TimeServiceProvider
public boolean hasBroadcastVariable(String name)
RuntimeContext
name
.name
- The name under which the broadcast variable is registered;public <RT> List<RT> getBroadcastVariable(String name)
RuntimeContext
name
.
IMPORTANT: The broadcast variable data structure is shared between the parallel tasks on one machine. Any access that modifies its internal state needs to be manually synchronized by the caller.
name
- The name under which the broadcast variable is registered;public <T,C> C getBroadcastVariableWithInitializer(String name, BroadcastVariableInitializer<T,C> initializer)
RuntimeContext
name
. The broadcast variable is returned as a shared data structure
that is initialized with the given BroadcastVariableInitializer
.
IMPORTANT: The broadcast variable data structure is shared between the parallel tasks on one machine. Any access that modifies its internal state needs to be manually synchronized by the caller.
name
- The name under which the broadcast variable is registered;initializer
- The initializer that creates the shared data structure of the broadcast
variable from the sequence of elements.public <T> ValueState<T> getState(ValueStateDescriptor<T> stateProperties)
RuntimeContext
Because the scope of each value is the key of the currently processed element, and the elements are distributed by the Flink runtime, the system can transparently scale out and redistribute the state and KeyedStream.
The following code example shows how to implement a continuous counter that counts how many times elements of a certain key occur, and emits an updated count for that element on each occurrence.
DataStream<MyType> stream = ...;
KeyedStream<MyType> keyedStream = stream.keyBy("id");
keyedStream.map(new RichMapFunction<MyType, Tuple2<MyType, Long>>() {
private ValueState<Long> count;
public void open(Configuration cfg) {
state = getRuntimeContext().getState(
new ValueStateDescriptor<Long>("count", LongSerializer.INSTANCE, 0L));
}
public Tuple2<MyType, Long> map(MyType value) {
long count = state.value() + 1;
state.update(value);
return new Tuple2<>(value, count);
}
});
getState
in interface RuntimeContext
getState
in class AbstractRuntimeUDFContext
T
- The type of value stored in the state.stateProperties
- The descriptor defining the properties of the stats.public <T> ListState<T> getListState(ListStateDescriptor<T> stateProperties)
RuntimeContext
RuntimeContext.getState(ValueStateDescriptor)
, but is optimized for state that
holds lists. One can adds elements to the list, or retrieve the list as a whole.
This state is only accessible if the function is executed on a KeyedStream.
DataStream<MyType> stream = ...;
KeyedStream<MyType> keyedStream = stream.keyBy("id");
keyedStream.map(new RichFlatMapFunction<MyType, List<MyType>>() {
private ListState<MyType> state;
public void open(Configuration cfg) {
state = getRuntimeContext().getListState(
new ListStateDescriptor<>("myState", MyType.class));
}
public void flatMap(MyType value, Collector<MyType> out) {
if (value.isDivider()) {
for (MyType t : state.get()) {
out.collect(t);
}
} else {
state.add(value);
}
}
});
getListState
in interface RuntimeContext
getListState
in class AbstractRuntimeUDFContext
T
- The type of value stored in the state.stateProperties
- The descriptor defining the properties of the stats.public <T> ReducingState<T> getReducingState(ReducingStateDescriptor<T> stateProperties)
RuntimeContext
RuntimeContext.getState(ValueStateDescriptor)
, but is optimized for state that
aggregates values.
This state is only accessible if the function is executed on a KeyedStream.
DataStream<MyType> stream = ...;
KeyedStream<MyType> keyedStream = stream.keyBy("id");
keyedStream.map(new RichMapFunction<MyType, List<MyType>>() {
private ReducingState<Long> sum;
public void open(Configuration cfg) {
state = getRuntimeContext().getReducingState(
new ReducingStateDescriptor<>("sum", MyType.class, 0L, (a, b) -> a + b));
}
public Tuple2<MyType, Long> map(MyType value) {
sum.add(value.count());
return new Tuple2<>(value, sum.get());
}
});
getReducingState
in interface RuntimeContext
getReducingState
in class AbstractRuntimeUDFContext
T
- The type of value stored in the state.stateProperties
- The descriptor defining the properties of the stats.@Deprecated public <S> OperatorState<S> getKeyValueState(String name, Class<S> stateType, S defaultState)
RuntimeContext
ValueState.value()
, the key/value state will
return the value bound to the key of the element currently processed by the function.
Each operator may maintain multiple key/value states, addressed with different names.
Because the scope of each value is the key of the currently processed element, and the elements are distributed by the Flink runtime, the system can transparently scale out and redistribute the state and KeyedStream.
The following code example shows how to implement a continuous counter that counts how many times elements of a certain key occur, and emits an updated count for that element on each occurrence.
DataStream<MyType> stream = ...;
KeyedStream<MyType> keyedStream = stream.keyBy("id");
keyedStream.map(new RichMapFunction<MyType, Tuple2<MyType, Long>>() {
private State<Long> state;
public void open(Configuration cfg) {
state = getRuntimeContext().getKeyValueState(Long.class, 0L);
}
public Tuple2<MyType, Long> map(MyType value) {
long count = state.value();
state.update(value + 1);
return new Tuple2<>(value, count);
}
});
This method attempts to deduce the type information from the given type class. If the
full type cannot be determined from the class (for example because of generic parameters),
the TypeInformation object must be manually passed via
RuntimeContext.getKeyValueState(String, TypeInformation, Object)
.
getKeyValueState
in interface RuntimeContext
getKeyValueState
in class AbstractRuntimeUDFContext
S
- The type of the state.name
- The name of the key/value state.stateType
- The class of the type that is stored in the state. Used to generate
serializers for managed memory and checkpointing.defaultState
- The default state value, returned when the state is accessed and
no value has yet been set for the key. May be null.@Deprecated public <S> OperatorState<S> getKeyValueState(String name, TypeInformation<S> stateType, S defaultState)
RuntimeContext
ValueState.value()
, the key/value state will
return the value bound to the key of the element currently processed by the function.
Each operator may maintain multiple key/value states, addressed with different names.
Because the scope of each value is the key of the currently processed element, and the elements are distributed by the Flink runtime, the system can transparently scale out and redistribute the state and KeyedStream.
The following code example shows how to implement a continuous counter that counts how many times elements of a certain key occur, and emits an updated count for that element on each occurrence.
DataStream<MyType> stream = ...;
KeyedStream<MyType> keyedStream = stream.keyBy("id");
keyedStream.map(new RichMapFunction<MyType, Tuple2<MyType, Long>>() {
private State<Long> state;
public void open(Configuration cfg) {
state = getRuntimeContext().getKeyValueState(Long.class, 0L);
}
public Tuple2<MyType, Long> map(MyType value) {
long count = state.value();
state.update(value + 1);
return new Tuple2<>(value, count);
}
});
getKeyValueState
in interface RuntimeContext
getKeyValueState
in class AbstractRuntimeUDFContext
S
- The type of the state.name
- The name of the key/value state.stateType
- The type information for the type that is stored in the state.
Used to create serializers for managed memory and checkpoints.defaultState
- The default state value, returned when the state is accessed and
no value has yet been set for the key. May be null.public boolean isCheckpointingEnabled()
public CheckpointingMode getCheckpointMode()
public long getBufferTimeout()
Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.