@Internal public final class SavepointRuntimeContext extends Object implements RuntimeContext
RuntimeContext which delegates to the underlying batch RuntimeContext
along with a specified KeyedStateStore.
This RuntimeContext has the ability to force eager state registration by throwing an
exception if state is registered outside of open.
| Constructor and Description |
|---|
SavepointRuntimeContext(RuntimeContext ctx,
KeyedStateStore keyedStateStore) |
| Modifier and Type | Method and Description |
|---|---|
<V,A extends Serializable> |
addAccumulator(String name,
Accumulator<V,A> accumulator)
Add this accumulator.
|
<T> TypeSerializer<T> |
createSerializer(TypeInformation<T> typeInformation)
Create a serializer for a given type.
|
void |
disableStateRegistration() |
<V,A extends Serializable> |
getAccumulator(String name)
Get an existing accumulator object.
|
<IN,ACC,OUT> |
getAggregatingState(AggregatingStateDescriptor<IN,ACC,OUT> stateProperties)
Gets a handle to the system's key/value aggregating state.
|
<RT> List<RT> |
getBroadcastVariable(String name)
Returns the result bound to the broadcast variable identified by the given
name. |
<T,C> C |
getBroadcastVariableWithInitializer(String name,
BroadcastVariableInitializer<T,C> initializer)
Returns the result bound to the broadcast variable identified by the given
name. |
DistributedCache |
getDistributedCache()
Returns the
DistributedCache to get the local temporary file copies of files
otherwise not locally accessible. |
DoubleCounter |
getDoubleCounter(String name)
Convenience function to create a counter object for doubles.
|
ExecutionConfig |
getExecutionConfig()
Deprecated.
|
Set<ExternalResourceInfo> |
getExternalResourceInfos(String resourceName)
Get the specific external resource information by the resourceName.
|
Map<String,String> |
getGlobalJobParameters()
Get global job parameters.
|
Histogram |
getHistogram(String name)
Convenience function to create a counter object for histograms.
|
IntCounter |
getIntCounter(String name)
Convenience function to create a counter object for integers.
|
JobInfo |
getJobInfo()
Get the meta information of current job.
|
<T> ListState<T> |
getListState(ListStateDescriptor<T> stateProperties)
Gets a handle to the system's key/value list state.
|
LongCounter |
getLongCounter(String name)
Convenience function to create a counter object for longs.
|
<UK,UV> MapState<UK,UV> |
getMapState(MapStateDescriptor<UK,UV> stateProperties)
Gets a handle to the system's key/value map state.
|
OperatorMetricGroup |
getMetricGroup()
Returns the metric group for this parallel subtask.
|
<T> ReducingState<T> |
getReducingState(ReducingStateDescriptor<T> stateProperties)
Gets a handle to the system's key/value reducing state.
|
<T> ValueState<T> |
getState(ValueStateDescriptor<T> stateProperties)
Gets a handle to the system's key/value state.
|
List<StateDescriptor<?,?>> |
getStateDescriptors() |
TaskInfo |
getTaskInfo()
Get the meta information of current task.
|
ClassLoader |
getUserCodeClassLoader()
Gets the ClassLoader to load classes that are not in system's classpath, but are part of the
jar file of a user job.
|
boolean |
hasBroadcastVariable(String name)
Tests for the existence of the broadcast variable identified by the given
name. |
boolean |
isObjectReuseEnabled()
Check if object reuse is enabled.
|
void |
registerUserCodeClassLoaderReleaseHookIfAbsent(String releaseHookName,
Runnable releaseHook)
Registers a custom hook for the user code class loader release.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitgetAttemptNumber, getIndexOfThisSubtask, getJobId, getMaxNumberOfParallelSubtasks, getNumberOfParallelSubtasks, getTaskName, getTaskNameWithSubtaskspublic SavepointRuntimeContext(RuntimeContext ctx, KeyedStateStore keyedStateStore)
public JobInfo getJobInfo()
RuntimeContextgetJobInfo in interface RuntimeContextpublic TaskInfo getTaskInfo()
RuntimeContextgetTaskInfo in interface RuntimeContextpublic OperatorMetricGroup getMetricGroup()
RuntimeContextgetMetricGroup in interface RuntimeContext@Deprecated public ExecutionConfig getExecutionConfig()
RuntimeContextExecutionConfig for the currently executing
job.getExecutionConfig in interface RuntimeContextpublic <T> TypeSerializer<T> createSerializer(TypeInformation<T> typeInformation)
RuntimeContextcreateSerializer in interface RuntimeContexttypeInformation - the type information of the object to be serializedpublic Map<String,String> getGlobalJobParameters()
RuntimeContextgetGlobalJobParameters in interface RuntimeContextpublic boolean isObjectReuseEnabled()
RuntimeContextisObjectReuseEnabled in interface RuntimeContextpublic ClassLoader getUserCodeClassLoader()
RuntimeContextgetUserCodeClassLoader in interface RuntimeContextpublic void registerUserCodeClassLoaderReleaseHookIfAbsent(String releaseHookName, Runnable releaseHook)
RuntimeContextThe release hook is executed just before the user code class loader is being released. Registration only happens if no hook has been registered under this name already.
registerUserCodeClassLoaderReleaseHookIfAbsent in interface RuntimeContextreleaseHookName - name of the release hook.releaseHook - release hook which is executed just before the user code class loader is
being releasedpublic <V,A extends Serializable> void addAccumulator(String name, Accumulator<V,A> accumulator)
RuntimeContextaddAccumulator in interface RuntimeContextpublic <V,A extends Serializable> Accumulator<V,A> getAccumulator(String name)
RuntimeContextThrows an exception if the accumulator does not exist or if the accumulator exists, but with different type.
getAccumulator in interface RuntimeContextpublic IntCounter getIntCounter(String name)
RuntimeContextgetIntCounter in interface RuntimeContextpublic LongCounter getLongCounter(String name)
RuntimeContextgetLongCounter in interface RuntimeContextpublic DoubleCounter getDoubleCounter(String name)
RuntimeContextgetDoubleCounter in interface RuntimeContextpublic Histogram getHistogram(String name)
RuntimeContextgetHistogram in interface RuntimeContextpublic Set<ExternalResourceInfo> getExternalResourceInfos(String resourceName)
RuntimeContextgetExternalResourceInfos in interface RuntimeContextresourceName - of the required external resourcepublic boolean hasBroadcastVariable(String name)
RuntimeContextname.hasBroadcastVariable in interface RuntimeContextname - The name under which the broadcast variable is registered;public <RT> List<RT> getBroadcastVariable(String name)
RuntimeContextname.
IMPORTANT: The broadcast variable data structure is shared between the parallel tasks on one machine. Any access that modifies its internal state needs to be manually synchronized by the caller.
getBroadcastVariable in interface RuntimeContextname - The name under which the broadcast variable is registered;public <T,C> C getBroadcastVariableWithInitializer(String name, BroadcastVariableInitializer<T,C> initializer)
RuntimeContextname. The
broadcast variable is returned as a shared data structure that is initialized with the given
BroadcastVariableInitializer.
IMPORTANT: The broadcast variable data structure is shared between the parallel tasks on one machine. Any access that modifies its internal state needs to be manually synchronized by the caller.
getBroadcastVariableWithInitializer in interface RuntimeContextname - The name under which the broadcast variable is registered;initializer - The initializer that creates the shared data structure of the broadcast
variable from the sequence of elements.public DistributedCache getDistributedCache()
RuntimeContextDistributedCache to get the local temporary file copies of files
otherwise not locally accessible.getDistributedCache in interface RuntimeContextpublic <T> ValueState<T> getState(ValueStateDescriptor<T> stateProperties)
RuntimeContextBecause the scope of each value is the key of the currently processed element, and the elements are distributed by the Flink runtime, the system can transparently scale out and redistribute the state and KeyedStream.
The following code example shows how to implement a continuous counter that counts how many times elements of a certain key occur, and emits an updated count for that element on each occurrence.
DataStream<MyType> stream = ...;
KeyedStream<MyType> keyedStream = stream.keyBy("id");
keyedStream.map(new RichMapFunction<MyType, Tuple2<MyType, Long>>() {
private ValueState<Long> state;
public void open(Configuration cfg) {
state = getRuntimeContext().getState(
new ValueStateDescriptor<Long>("count", LongSerializer.INSTANCE, 0L));
}
public Tuple2<MyType, Long> map(MyType value) {
long count = state.value() + 1;
state.update(count);
return new Tuple2<>(value, count);
}
});
getState in interface RuntimeContextT - The type of value stored in the state.stateProperties - The descriptor defining the properties of the stats.public <T> ListState<T> getListState(ListStateDescriptor<T> stateProperties)
RuntimeContextRuntimeContext.getState(ValueStateDescriptor), but is optimized for state that holds
lists. One can add elements to the list, or retrieve the list as a whole.
This state is only accessible if the function is executed on a KeyedStream.
DataStream<MyType> stream = ...;
KeyedStream<MyType> keyedStream = stream.keyBy("id");
keyedStream.map(new RichFlatMapFunction<MyType, List<MyType>>() {
private ListState<MyType> state;
public void open(Configuration cfg) {
state = getRuntimeContext().getListState(
new ListStateDescriptor<>("myState", MyType.class));
}
public void flatMap(MyType value, Collector<MyType> out) {
if (value.isDivider()) {
for (MyType t : state.get()) {
out.collect(t);
}
} else {
state.add(value);
}
}
});
getListState in interface RuntimeContextT - The type of value stored in the state.stateProperties - The descriptor defining the properties of the stats.public <T> ReducingState<T> getReducingState(ReducingStateDescriptor<T> stateProperties)
RuntimeContextRuntimeContext.getState(ValueStateDescriptor), but is optimized for state that
aggregates values.
This state is only accessible if the function is executed on a KeyedStream.
DataStream<MyType> stream = ...;
KeyedStream<MyType> keyedStream = stream.keyBy("id");
keyedStream.map(new RichMapFunction<MyType, List<MyType>>() {
private ReducingState<Long> state;
public void open(Configuration cfg) {
state = getRuntimeContext().getReducingState(
new ReducingStateDescriptor<>("sum", (a, b) -> a + b, Long.class));
}
public Tuple2<MyType, Long> map(MyType value) {
state.add(value.count());
return new Tuple2<>(value, state.get());
}
});
getReducingState in interface RuntimeContextT - The type of value stored in the state.stateProperties - The descriptor defining the properties of the stats.public <IN,ACC,OUT> AggregatingState<IN,OUT> getAggregatingState(AggregatingStateDescriptor<IN,ACC,OUT> stateProperties)
RuntimeContextRuntimeContext.getState(ValueStateDescriptor), but is optimized for state that
aggregates values with different types.
This state is only accessible if the function is executed on a KeyedStream.
DataStream<MyType> stream = ...;
KeyedStream<MyType> keyedStream = stream.keyBy("id");
AggregateFunction<...> aggregateFunction = ...
keyedStream.map(new RichMapFunction<MyType, List<MyType>>() {
private AggregatingState<MyType, Long> state;
public void open(Configuration cfg) {
state = getRuntimeContext().getAggregatingState(
new AggregatingStateDescriptor<>("sum", aggregateFunction, Long.class));
}
public Tuple2<MyType, Long> map(MyType value) {
state.add(value);
return new Tuple2<>(value, state.get());
}
});
getAggregatingState in interface RuntimeContextIN - The type of the values that are added to the state.ACC - The type of the accumulator (intermediate aggregation state).OUT - The type of the values that are returned from the state.stateProperties - The descriptor defining the properties of the stats.public <UK,UV> MapState<UK,UV> getMapState(MapStateDescriptor<UK,UV> stateProperties)
RuntimeContextRuntimeContext.getState(ValueStateDescriptor), but is optimized for state that is
composed of user-defined key-value pairs
This state is only accessible if the function is executed on a KeyedStream.
DataStream<MyType> stream = ...;
KeyedStream<MyType> keyedStream = stream.keyBy("id");
keyedStream.map(new RichMapFunction<MyType, List<MyType>>() {
private MapState<MyType, Long> state;
public void open(Configuration cfg) {
state = getRuntimeContext().getMapState(
new MapStateDescriptor<>("sum", MyType.class, Long.class));
}
public Tuple2<MyType, Long> map(MyType value) {
return new Tuple2<>(value, state.get(value));
}
});
getMapState in interface RuntimeContextUK - The type of the user keys stored in the state.UV - The type of the user values stored in the state.stateProperties - The descriptor defining the properties of the stats.public List<StateDescriptor<?,?>> getStateDescriptors()
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.