T
- The type of the elements in the Keyed Stream.KEY
- The type of the key in the Keyed Stream.@Public public class KeyedStream<T,KEY> extends DataStream<T>
KeyedStream
represents a DataStream
on which operator state is
partitioned by key using a provided KeySelector
. Typical operations supported by a
DataStream
are also possible on a KeyedStream
, with the exception of
partitioning methods such as shuffle, forward and keyBy.
Reduce-style operations, such as reduce(org.apache.flink.api.common.functions.ReduceFunction<T>)
, sum(int)
and fold(R, org.apache.flink.api.common.functions.FoldFunction<T, R>)
work on elements
that have the same key.
environment, transformation
Constructor and Description |
---|
KeyedStream(DataStream<T> dataStream,
KeySelector<T,KEY> keySelector)
Creates a new
KeyedStream using the given KeySelector
to partition operator state by key. |
KeyedStream(DataStream<T> dataStream,
KeySelector<T,KEY> keySelector,
TypeInformation<KEY> keyType)
Creates a new
KeyedStream using the given KeySelector
to partition operator state by key. |
Modifier and Type | Method and Description |
---|---|
DataStreamSink<T> |
addSink(SinkFunction<T> sinkFunction)
Adds the given sink to this DataStream.
|
protected SingleOutputStreamOperator<T> |
aggregate(AggregationFunction<T> aggregate) |
WindowedStream<T,KEY,GlobalWindow> |
countWindow(long size)
Windows this
KeyedStream into tumbling count windows. |
WindowedStream<T,KEY,GlobalWindow> |
countWindow(long size,
long slide)
Windows this
KeyedStream into sliding count windows. |
<R> SingleOutputStreamOperator<R> |
fold(R initialValue,
FoldFunction<T,R> folder)
Applies a fold transformation on the grouped data stream grouped on by
the given key position.
|
KeySelector<T,KEY> |
getKeySelector()
Gets the key selector that can get the key by which the stream if partitioned from the elements.
|
TypeInformation<KEY> |
getKeyType()
Gets the type of the key by which the stream is partitioned.
|
SingleOutputStreamOperator<T> |
max(int positionToMax)
Applies an aggregation that gives the current maximum of the data stream
at the given position by the given key.
|
SingleOutputStreamOperator<T> |
max(String field)
Applies an aggregation that that gives the current maximum of the pojo
data stream at the given field expression by the given key.
|
SingleOutputStreamOperator<T> |
maxBy(int positionToMaxBy)
Applies an aggregation that that gives the current element with the
maximum value at the given position by the given key.
|
SingleOutputStreamOperator<T> |
maxBy(int positionToMaxBy,
boolean first)
Applies an aggregation that that gives the current element with the
maximum value at the given position by the given key.
|
SingleOutputStreamOperator<T> |
maxBy(String positionToMaxBy)
Applies an aggregation that that gives the current element with the
maximum value at the given position by the given key.
|
SingleOutputStreamOperator<T> |
maxBy(String field,
boolean first)
Applies an aggregation that that gives the current maximum element of the
pojo data stream by the given field expression by the given key.
|
SingleOutputStreamOperator<T> |
min(int positionToMin)
Applies an aggregation that that gives the current minimum of the data
stream at the given position by the given key.
|
SingleOutputStreamOperator<T> |
min(String field)
Applies an aggregation that that gives the current minimum of the pojo
data stream at the given field expression by the given key.
|
SingleOutputStreamOperator<T> |
minBy(int positionToMinBy)
Applies an aggregation that that gives the current element with the
minimum value at the given position by the given key.
|
SingleOutputStreamOperator<T> |
minBy(int positionToMinBy,
boolean first)
Applies an aggregation that that gives the current element with the
minimum value at the given position by the given key.
|
SingleOutputStreamOperator<T> |
minBy(String positionToMinBy)
Applies an aggregation that that gives the current element with the
minimum value at the given position by the given key.
|
SingleOutputStreamOperator<T> |
minBy(String field,
boolean first)
Applies an aggregation that that gives the current minimum element of the
pojo data stream by the given field expression by the given key.
|
SingleOutputStreamOperator<T> |
reduce(ReduceFunction<T> reducer)
Applies a reduce transformation on the grouped data stream grouped on by
the given key position.
|
protected DataStream<T> |
setConnectionType(StreamPartitioner<T> partitioner)
Internal function for setting the partitioner for the DataStream
|
SingleOutputStreamOperator<T> |
sum(int positionToSum)
Applies an aggregation that gives a rolling sum of the data stream at the
given position grouped by the given key.
|
SingleOutputStreamOperator<T> |
sum(String field)
Applies an aggregation that that gives the current sum of the pojo data
stream at the given field expressionby the given key.
|
WindowedStream<T,KEY,TimeWindow> |
timeWindow(Time size)
Windows this
KeyedStream into tumbling time windows. |
WindowedStream<T,KEY,TimeWindow> |
timeWindow(Time size,
Time slide)
Windows this
KeyedStream into sliding time windows. |
<R> SingleOutputStreamOperator<R> |
transform(String operatorName,
TypeInformation<R> outTypeInfo,
OneInputStreamOperator<T,R> operator)
Method for passing user defined operators along with the type
information that will transform the DataStream.
|
<W extends Window> |
window(WindowAssigner<? super T,W> assigner)
Windows this data stream to a
WindowedStream , which evaluates windows
over a key grouped stream. |
assignTimestamps, assignTimestampsAndWatermarks, assignTimestampsAndWatermarks, broadcast, clean, coGroup, connect, countWindowAll, countWindowAll, filter, flatMap, forward, getExecutionConfig, getExecutionEnvironment, getId, getParallelism, getTransformation, getType, global, iterate, iterate, join, keyBy, keyBy, keyBy, map, partitionCustom, partitionCustom, partitionCustom, print, printToErr, project, rebalance, rescale, shuffle, split, timeWindowAll, timeWindowAll, union, windowAll, writeAsCsv, writeAsCsv, writeAsCsv, writeAsText, writeAsText, writeToSocket, writeUsingOutputFormat
public KeyedStream(DataStream<T> dataStream, KeySelector<T,KEY> keySelector)
KeyedStream
using the given KeySelector
to partition operator state by key.dataStream
- Base stream of datakeySelector
- Function for determining state partitionspublic KeyedStream(DataStream<T> dataStream, KeySelector<T,KEY> keySelector, TypeInformation<KEY> keyType)
KeyedStream
using the given KeySelector
to partition operator state by key.dataStream
- Base stream of datakeySelector
- Function for determining state partitions@Internal public KeySelector<T,KEY> getKeySelector()
@Internal public TypeInformation<KEY> getKeyType()
protected DataStream<T> setConnectionType(StreamPartitioner<T> partitioner)
DataStream
setConnectionType
in class DataStream<T>
partitioner
- Partitioner to set.@PublicEvolving public <R> SingleOutputStreamOperator<R> transform(String operatorName, TypeInformation<R> outTypeInfo, OneInputStreamOperator<T,R> operator)
DataStream
transform
in class DataStream<T>
R
- type of the return streamoperatorName
- name of the operator, for logging purposesoutTypeInfo
- the output type of the operatoroperator
- the object containing the transformation logicpublic DataStreamSink<T> addSink(SinkFunction<T> sinkFunction)
DataStream
StreamExecutionEnvironment.execute()
method is called.addSink
in class DataStream<T>
sinkFunction
- The object containing the sink's invoke function.public WindowedStream<T,KEY,TimeWindow> timeWindow(Time size)
KeyedStream
into tumbling time windows.
This is a shortcut for either .window(TumblingEventTimeWindows.of(size))
or
.window(TumblingProcessingTimeWindows.of(size))
depending on the time characteristic
set using
StreamExecutionEnvironment.setStreamTimeCharacteristic(org.apache.flink.streaming.api.TimeCharacteristic)
size
- The size of the window.public WindowedStream<T,KEY,TimeWindow> timeWindow(Time size, Time slide)
KeyedStream
into sliding time windows.
This is a shortcut for either .window(SlidingEventTimeWindows.of(size, slide))
or
.window(SlidingProcessingTimeWindows.of(size, slide))
depending on the time characteristic
set using
StreamExecutionEnvironment.setStreamTimeCharacteristic(org.apache.flink.streaming.api.TimeCharacteristic)
size
- The size of the window.public WindowedStream<T,KEY,GlobalWindow> countWindow(long size)
KeyedStream
into tumbling count windows.size
- The size of the windows in number of elements.public WindowedStream<T,KEY,GlobalWindow> countWindow(long size, long slide)
KeyedStream
into sliding count windows.size
- The size of the windows in number of elements.slide
- The slide interval in number of elements.@PublicEvolving public <W extends Window> WindowedStream<T,KEY,W> window(WindowAssigner<? super T,W> assigner)
WindowedStream
, which evaluates windows
over a key grouped stream. Elements are put into windows by a WindowAssigner
. The
grouping of elements is done both by key and by window.
A Trigger
can be defined to specify
when windows are evaluated. However, WindowAssigners
have a default Trigger
that is used if a Trigger
is not specified.
assigner
- The WindowAssigner
that assigns elements to windows.public SingleOutputStreamOperator<T> reduce(ReduceFunction<T> reducer)
ReduceFunction
will receive input
values based on the key value. Only input values with the same key will
go to the same reducer.reducer
- The ReduceFunction
that will be called for every
element of the input values with the same key.public <R> SingleOutputStreamOperator<R> fold(R initialValue, FoldFunction<T,R> folder)
FoldFunction
will receive input
values based on the key value. Only input values with the same key will
go to the same folder.folder
- The FoldFunction
that will be called for every element
of the input values with the same key.initialValue
- The initialValue passed to the folders for each key.public SingleOutputStreamOperator<T> sum(int positionToSum)
positionToSum
- The position in the data point to sumpublic SingleOutputStreamOperator<T> sum(String field)
DataStream
S underlying type. A dot can be used to drill down into
objects, as in "field1.getInnerField2()"
.field
- The field expression based on which the aggregation will be
applied.public SingleOutputStreamOperator<T> min(int positionToMin)
positionToMin
- The position in the data point to minimizepublic SingleOutputStreamOperator<T> min(String field)
DataStream
S underlying type. A dot can be used to drill down into
objects, as in "field1.getInnerField2()"
.field
- The field expression based on which the aggregation will be
applied.public SingleOutputStreamOperator<T> max(int positionToMax)
positionToMax
- The position in the data point to maximizepublic SingleOutputStreamOperator<T> max(String field)
DataStream
S underlying type. A dot can be used to drill down into
objects, as in "field1.getInnerField2()"
.field
- The field expression based on which the aggregation will be
applied.public SingleOutputStreamOperator<T> minBy(String field, boolean first)
DataStream
S underlying type. A dot can be used to drill down into
objects, as in "field1.getInnerField2()"
.field
- The field expression based on which the aggregation will be
applied.first
- If True then in case of field equality the first object will
be returnedpublic SingleOutputStreamOperator<T> maxBy(String field, boolean first)
DataStream
S underlying type. A dot can be used to drill down into
objects, as in "field1.getInnerField2()"
.field
- The field expression based on which the aggregation will be
applied.first
- If True then in case of field equality the first object will
be returnedpublic SingleOutputStreamOperator<T> minBy(int positionToMinBy)
positionToMinBy
- The position in the data point to minimizepublic SingleOutputStreamOperator<T> minBy(String positionToMinBy)
positionToMinBy
- The position in the data point to minimizepublic SingleOutputStreamOperator<T> minBy(int positionToMinBy, boolean first)
positionToMinBy
- The position in the data point to minimizefirst
- If true, then the operator return the first element with the
minimal value, otherwise returns the lastpublic SingleOutputStreamOperator<T> maxBy(int positionToMaxBy)
positionToMaxBy
- The position in the data point to maximizepublic SingleOutputStreamOperator<T> maxBy(String positionToMaxBy)
positionToMaxBy
- The position in the data point to maximizepublic SingleOutputStreamOperator<T> maxBy(int positionToMaxBy, boolean first)
positionToMaxBy
- The position in the data point to maximize.first
- If true, then the operator return the first element with the
maximum value, otherwise returns the lastprotected SingleOutputStreamOperator<T> aggregate(AggregationFunction<T> aggregate)
Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.