public class BroadcastUtils extends Object
Constructor and Description |
---|
BroadcastUtils() |
Modifier and Type | Method and Description |
---|---|
static <OUT> org.apache.flink.streaming.api.datastream.DataStream<OUT> |
withBroadcastStream(List<org.apache.flink.streaming.api.datastream.DataStream<?>> inputList,
Map<String,org.apache.flink.streaming.api.datastream.DataStream<?>> bcStreams,
java.util.function.Function<List<org.apache.flink.streaming.api.datastream.DataStream<?>>,org.apache.flink.streaming.api.datastream.DataStream<OUT>> userDefinedFunction)
Supports withBroadcastStream in DataStream API.
|
public static <OUT> org.apache.flink.streaming.api.datastream.DataStream<OUT> withBroadcastStream(List<org.apache.flink.streaming.api.datastream.DataStream<?>> inputList, Map<String,org.apache.flink.streaming.api.datastream.DataStream<?>> bcStreams, java.util.function.Function<List<org.apache.flink.streaming.api.datastream.DataStream<?>>,org.apache.flink.streaming.api.datastream.DataStream<OUT>> userDefinedFunction)
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator<OUT, ? extends
org.apache.flink.api.common.functions.RichFunction>
. Users can access the broadcast
variables by RichFunction.getRuntimeContext().getBroadcastVariable(...)
or RichFunction.getRuntimeContext().hasBroadcastVariable(...)
or RichFunction.getRuntimeContext().getBroadcastVariableWithInitializer(...)
.
In detail, the broadcast input data streams will be consumed first and further consumed by non-broadcast inputs. For now the non-broadcast input are cached by default to avoid the possible deadlocks.
inputList
- non-broadcast input list.bcStreams
- map of the broadcast data streams, where the key is the name and the value
is the corresponding data stream.userDefinedFunction
- the user defined logic in which users can access the broadcast
data streams and produce the output data stream. Note that though users can add more than
one operator in this logic, but only the operator that generates the result stream can
contain a rich function and access the broadcast variables. Other operators will
encounter NPE when accessing the broadcast variables.Copyright © 2019–2023 The Apache Software Foundation. All rights reserved.