T
- The type of elements in the stream.K
- The type of the key by which elements are grouped.W
- The type of Window
that the WindowAssigner
assigns the elements to.@Public public class WindowedStream<T,K,W extends Window> extends Object
WindowedStream
represents a data stream where elements are grouped by key, and for each
key, the stream of elements is split into windows based on a WindowAssigner
. Window emission is triggered
based on a Trigger
.
The windows are conceptually evaluated for each key individually, meaning windows can trigger at different points for each key.
If an Evictor
is specified it will be used to evict elements from the window after
evaluation was triggered by the Trigger
but before the actual evaluation of the window.
When using an evictor window performance will degrade significantly, since incremental
aggregation of window results cannot be used.
Note that the WindowedStream
is purely an API construct, during runtime the WindowedStream
will be collapsed together with the KeyedStream
and the operation over
the window into one single operation.
Constructor and Description |
---|
WindowedStream(KeyedStream<T,K> input,
WindowAssigner<? super T,W> windowAssigner) |
Modifier and Type | Method and Description |
---|---|
<ACC,R> SingleOutputStreamOperator<R> |
aggregate(AggregateFunction<T,ACC,R> function)
Applies the given aggregation function to each window.
|
<ACC,R> SingleOutputStreamOperator<R> |
aggregate(AggregateFunction<T,ACC,R> function,
TypeInformation<ACC> accumulatorType,
TypeInformation<R> resultType)
Applies the given aggregation function to each window.
|
<ACC,V,R> SingleOutputStreamOperator<R> |
aggregate(AggregateFunction<T,ACC,V> aggFunction,
ProcessWindowFunction<V,R,K,W> windowFunction)
Applies the given window function to each window.
|
<ACC,V,R> SingleOutputStreamOperator<R> |
aggregate(AggregateFunction<T,ACC,V> aggregateFunction,
ProcessWindowFunction<V,R,K,W> windowFunction,
TypeInformation<ACC> accumulatorType,
TypeInformation<V> aggregateResultType,
TypeInformation<R> resultType)
Applies the given window function to each window.
|
<ACC,V,R> SingleOutputStreamOperator<R> |
aggregate(AggregateFunction<T,ACC,V> aggFunction,
WindowFunction<V,R,K,W> windowFunction)
Applies the given window function to each window.
|
<ACC,V,R> SingleOutputStreamOperator<R> |
aggregate(AggregateFunction<T,ACC,V> aggregateFunction,
WindowFunction<V,R,K,W> windowFunction,
TypeInformation<ACC> accumulatorType,
TypeInformation<R> resultType)
Applies the given window function to each window.
|
WindowedStream<T,K,W> |
allowedLateness(Duration lateness)
Sets the time by which elements are allowed to be late.
|
WindowedStream<T,K,W> |
allowedLateness(Time lateness)
Deprecated.
|
<R> SingleOutputStreamOperator<R> |
apply(ReduceFunction<T> reduceFunction,
WindowFunction<T,R,K,W> function)
Deprecated.
Use
reduce(ReduceFunction, WindowFunction) instead. |
<R> SingleOutputStreamOperator<R> |
apply(ReduceFunction<T> reduceFunction,
WindowFunction<T,R,K,W> function,
TypeInformation<R> resultType)
Deprecated.
|
<R> SingleOutputStreamOperator<R> |
apply(WindowFunction<T,R,K,W> function)
Applies the given window function to each window.
|
<R> SingleOutputStreamOperator<R> |
apply(WindowFunction<T,R,K,W> function,
TypeInformation<R> resultType)
Applies the given window function to each window.
|
WindowedStream<T,K,W> |
evictor(Evictor<? super T,? super W> evictor)
Sets the
Evictor that should be used to evict elements from a window before emission. |
StreamExecutionEnvironment |
getExecutionEnvironment() |
TypeInformation<T> |
getInputType() |
SingleOutputStreamOperator<T> |
max(int positionToMax)
Applies an aggregation that gives the maximum value of every window of the data stream at the
given position.
|
SingleOutputStreamOperator<T> |
max(String field)
Applies an aggregation that gives the maximum value of the pojo data stream at the given
field expression for every window.
|
SingleOutputStreamOperator<T> |
maxBy(int positionToMaxBy)
Applies an aggregation that gives the maximum element of every window of the data stream by
the given position.
|
SingleOutputStreamOperator<T> |
maxBy(int positionToMaxBy,
boolean first)
Applies an aggregation that gives the maximum element of every window of the data stream by
the given position.
|
SingleOutputStreamOperator<T> |
maxBy(String field)
Applies an aggregation that gives the maximum element of every window of the data stream by
the given field.
|
SingleOutputStreamOperator<T> |
maxBy(String field,
boolean first)
Applies an aggregation that gives the maximum element of the pojo data stream by the given
field expression for every window.
|
SingleOutputStreamOperator<T> |
min(int positionToMin)
Applies an aggregation that gives the minimum value of every window of the data stream at the
given position.
|
SingleOutputStreamOperator<T> |
min(String field)
Applies an aggregation that gives the minimum value of the pojo data stream at the given
field expression for every window.
|
SingleOutputStreamOperator<T> |
minBy(int positionToMinBy)
Applies an aggregation that gives the minimum element of every window of the data stream by
the given position.
|
SingleOutputStreamOperator<T> |
minBy(int positionToMinBy,
boolean first)
Applies an aggregation that gives the minimum element of every window of the data stream by
the given position.
|
SingleOutputStreamOperator<T> |
minBy(String field)
Applies an aggregation that gives the minimum element of every window of the data stream by
the given field.
|
SingleOutputStreamOperator<T> |
minBy(String field,
boolean first)
Applies an aggregation that gives the minimum element of the pojo data stream by the given
field expression for every window.
|
<R> SingleOutputStreamOperator<R> |
process(ProcessWindowFunction<T,R,K,W> function)
Applies the given window function to each window.
|
<R> SingleOutputStreamOperator<R> |
process(ProcessWindowFunction<T,R,K,W> function,
TypeInformation<R> resultType)
Applies the given window function to each window.
|
SingleOutputStreamOperator<T> |
reduce(ReduceFunction<T> function)
Applies a reduce function to the window.
|
<R> SingleOutputStreamOperator<R> |
reduce(ReduceFunction<T> reduceFunction,
ProcessWindowFunction<T,R,K,W> function)
Applies the given window function to each window.
|
<R> SingleOutputStreamOperator<R> |
reduce(ReduceFunction<T> reduceFunction,
ProcessWindowFunction<T,R,K,W> function,
TypeInformation<R> resultType)
Applies the given window function to each window.
|
<R> SingleOutputStreamOperator<R> |
reduce(ReduceFunction<T> reduceFunction,
WindowFunction<T,R,K,W> function)
Applies the given window function to each window.
|
<R> SingleOutputStreamOperator<R> |
reduce(ReduceFunction<T> reduceFunction,
WindowFunction<T,R,K,W> function,
TypeInformation<R> resultType)
Applies the given window function to each window.
|
WindowedStream<T,K,W> |
sideOutputLateData(OutputTag<T> outputTag)
Send late arriving data to the side output identified by the given
OutputTag . |
SingleOutputStreamOperator<T> |
sum(int positionToSum)
Applies an aggregation that sums every window of the data stream at the given position.
|
SingleOutputStreamOperator<T> |
sum(String field)
Applies an aggregation that sums every window of the pojo data stream at the given field for
every window.
|
WindowedStream<T,K,W> |
trigger(Trigger<? super T,? super W> trigger)
Sets the
Trigger that should be used to trigger window emission. |
@PublicEvolving public WindowedStream(KeyedStream<T,K> input, WindowAssigner<? super T,W> windowAssigner)
@PublicEvolving public WindowedStream<T,K,W> trigger(Trigger<? super T,? super W> trigger)
Trigger
that should be used to trigger window emission.@Deprecated @PublicEvolving public WindowedStream<T,K,W> allowedLateness(Time lateness)
allowedLateness(Duration)
0L
.
Setting an allowed lateness is only valid for event-time windows.
@PublicEvolving public WindowedStream<T,K,W> allowedLateness(Duration lateness)
0L
.
Setting an allowed lateness is only valid for event-time windows.
@PublicEvolving public WindowedStream<T,K,W> sideOutputLateData(OutputTag<T> outputTag)
OutputTag
. Data is
considered late after the watermark has passed the end of the window plus the allowed
lateness set using allowedLateness(Duration)
.
You can get the stream of late data using SingleOutputStreamOperator.getSideOutput(OutputTag)
on the SingleOutputStreamOperator
resulting from the windowed operation with the same OutputTag
.
@PublicEvolving public WindowedStream<T,K,W> evictor(Evictor<? super T,? super W> evictor)
Evictor
that should be used to evict elements from a window before emission.
Note: When using an evictor window performance will degrade significantly, since incremental aggregation of window results cannot be used.
public SingleOutputStreamOperator<T> reduce(ReduceFunction<T> function)
This window will try and incrementally aggregate data as much as the window policies permit. For example, tumbling time windows can aggregate the data, meaning that only one element per key is stored. Sliding time windows will aggregate on the granularity of the slide interval, so a few elements are stored per key (one per slide interval). Custom windows may not be able to incrementally aggregate, or may need to store extra values in an aggregation tree.
function
- The reduce function.public <R> SingleOutputStreamOperator<R> reduce(ReduceFunction<T> reduceFunction, WindowFunction<T,R,K,W> function)
Arriving data is incrementally aggregated using the given reducer.
reduceFunction
- The reduce function that is used for incremental aggregation.function
- The window function.public <R> SingleOutputStreamOperator<R> reduce(ReduceFunction<T> reduceFunction, WindowFunction<T,R,K,W> function, TypeInformation<R> resultType)
Arriving data is incrementally aggregated using the given reducer.
reduceFunction
- The reduce function that is used for incremental aggregation.function
- The window function.resultType
- Type information for the result type of the window function.@PublicEvolving public <R> SingleOutputStreamOperator<R> reduce(ReduceFunction<T> reduceFunction, ProcessWindowFunction<T,R,K,W> function)
Arriving data is incrementally aggregated using the given reducer.
reduceFunction
- The reduce function that is used for incremental aggregation.function
- The window function.@Internal public <R> SingleOutputStreamOperator<R> reduce(ReduceFunction<T> reduceFunction, ProcessWindowFunction<T,R,K,W> function, TypeInformation<R> resultType)
Arriving data is incrementally aggregated using the given reducer.
reduceFunction
- The reduce function that is used for incremental aggregation.function
- The window function.resultType
- Type information for the result type of the window function@PublicEvolving public <ACC,R> SingleOutputStreamOperator<R> aggregate(AggregateFunction<T,ACC,R> function)
ACC
- The type of the AggregateFunction's accumulatorR
- The type of the elements in the resulting stream, equal to the AggregateFunction's
result typefunction
- The aggregation function.@PublicEvolving public <ACC,R> SingleOutputStreamOperator<R> aggregate(AggregateFunction<T,ACC,R> function, TypeInformation<ACC> accumulatorType, TypeInformation<R> resultType)
ACC
- The type of the AggregateFunction's accumulatorR
- The type of the elements in the resulting stream, equal to the AggregateFunction's
result typefunction
- The aggregation function.@PublicEvolving public <ACC,V,R> SingleOutputStreamOperator<R> aggregate(AggregateFunction<T,ACC,V> aggFunction, WindowFunction<V,R,K,W> windowFunction)
Arriving data is incrementally aggregated using the given aggregate function. This means that the window function typically has only a single value to process when called.
ACC
- The type of the AggregateFunction's accumulatorV
- The type of AggregateFunction's result, and the WindowFunction's inputR
- The type of the elements in the resulting stream, equal to the WindowFunction's
result typeaggFunction
- The aggregate function that is used for incremental aggregation.windowFunction
- The window function.@PublicEvolving public <ACC,V,R> SingleOutputStreamOperator<R> aggregate(AggregateFunction<T,ACC,V> aggregateFunction, WindowFunction<V,R,K,W> windowFunction, TypeInformation<ACC> accumulatorType, TypeInformation<R> resultType)
Arriving data is incrementally aggregated using the given aggregate function. This means that the window function typically has only a single value to process when called.
ACC
- The type of the AggregateFunction's accumulatorV
- The type of AggregateFunction's result, and the WindowFunction's inputR
- The type of the elements in the resulting stream, equal to the WindowFunction's
result typeaggregateFunction
- The aggregation function that is used for incremental aggregation.windowFunction
- The window function.accumulatorType
- Type information for the internal accumulator type of the aggregation
functionresultType
- Type information for the result type of the window function@PublicEvolving public <ACC,V,R> SingleOutputStreamOperator<R> aggregate(AggregateFunction<T,ACC,V> aggFunction, ProcessWindowFunction<V,R,K,W> windowFunction)
Arriving data is incrementally aggregated using the given aggregate function. This means that the window function typically has only a single value to process when called.
ACC
- The type of the AggregateFunction's accumulatorV
- The type of AggregateFunction's result, and the WindowFunction's inputR
- The type of the elements in the resulting stream, equal to the WindowFunction's
result typeaggFunction
- The aggregate function that is used for incremental aggregation.windowFunction
- The window function.@PublicEvolving public <ACC,V,R> SingleOutputStreamOperator<R> aggregate(AggregateFunction<T,ACC,V> aggregateFunction, ProcessWindowFunction<V,R,K,W> windowFunction, TypeInformation<ACC> accumulatorType, TypeInformation<V> aggregateResultType, TypeInformation<R> resultType)
Arriving data is incrementally aggregated using the given aggregate function. This means that the window function typically has only a single value to process when called.
ACC
- The type of the AggregateFunction's accumulatorV
- The type of AggregateFunction's result, and the WindowFunction's inputR
- The type of the elements in the resulting stream, equal to the WindowFunction's
result typeaggregateFunction
- The aggregation function that is used for incremental aggregation.windowFunction
- The window function.accumulatorType
- Type information for the internal accumulator type of the aggregation
functionresultType
- Type information for the result type of the window functionpublic <R> SingleOutputStreamOperator<R> apply(WindowFunction<T,R,K,W> function)
Note that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of incremental aggregation.
function
- The window function.public <R> SingleOutputStreamOperator<R> apply(WindowFunction<T,R,K,W> function, TypeInformation<R> resultType)
Note that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of incremental aggregation.
function
- The window function.resultType
- Type information for the result type of the window function@PublicEvolving public <R> SingleOutputStreamOperator<R> process(ProcessWindowFunction<T,R,K,W> function)
Note that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of incremental aggregation.
function
- The window function.@Internal public <R> SingleOutputStreamOperator<R> process(ProcessWindowFunction<T,R,K,W> function, TypeInformation<R> resultType)
Note that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of incremental aggregation.
function
- The window function.resultType
- Type information for the result type of the window function@Deprecated public <R> SingleOutputStreamOperator<R> apply(ReduceFunction<T> reduceFunction, WindowFunction<T,R,K,W> function)
reduce(ReduceFunction, WindowFunction)
instead.Arriving data is incrementally aggregated using the given reducer.
reduceFunction
- The reduce function that is used for incremental aggregation.function
- The window function.@Deprecated public <R> SingleOutputStreamOperator<R> apply(ReduceFunction<T> reduceFunction, WindowFunction<T,R,K,W> function, TypeInformation<R> resultType)
reduce(ReduceFunction, WindowFunction, TypeInformation)
instead.Arriving data is incrementally aggregated using the given reducer.
reduceFunction
- The reduce function that is used for incremental aggregation.function
- The window function.resultType
- Type information for the result type of the window functionpublic SingleOutputStreamOperator<T> sum(int positionToSum)
positionToSum
- The position in the tuple/array to sumpublic SingleOutputStreamOperator<T> sum(String field)
A field expression is either the name of a public field or a getter method with
parentheses of the stream's underlying type. A dot can be used to drill down into objects, as
in "field1.getInnerField2()"
.
field
- The field to sumpublic SingleOutputStreamOperator<T> min(int positionToMin)
positionToMin
- The position to minimizepublic SingleOutputStreamOperator<T> min(String field)
A field * expression is either the name of a public field or a getter method with
parentheses of the DataStream
S underlying type. A dot can be used to drill down into
objects, as in "field1.getInnerField2()"
.
field
- The field expression based on which the aggregation will be applied.public SingleOutputStreamOperator<T> minBy(int positionToMinBy)
positionToMinBy
- The position to minimize bypublic SingleOutputStreamOperator<T> minBy(String field)
field
- The field to minimize bypublic SingleOutputStreamOperator<T> minBy(int positionToMinBy, boolean first)
positionToMinBy
- The position to minimizefirst
- If true, then the operator return the first element with the minimum value,
otherwise returns the lastpublic SingleOutputStreamOperator<T> minBy(String field, boolean first)
DataStreams
underlying type. A dot
can be used to drill down into objects, as in "field1.getInnerField2()"
.field
- The field expression based on which the aggregation will be applied.first
- If True then in case of field equality the first object will be returnedpublic SingleOutputStreamOperator<T> max(int positionToMax)
positionToMax
- The position to maximizepublic SingleOutputStreamOperator<T> max(String field)
DataStreams
underlying type. A dot
can be used to drill down into objects, as in "field1.getInnerField2()"
.field
- The field expression based on which the aggregation will be applied.public SingleOutputStreamOperator<T> maxBy(int positionToMaxBy)
positionToMaxBy
- The position to maximize bypublic SingleOutputStreamOperator<T> maxBy(String field)
field
- The field to maximize bypublic SingleOutputStreamOperator<T> maxBy(int positionToMaxBy, boolean first)
positionToMaxBy
- The position to maximize byfirst
- If true, then the operator return the first element with the maximum value,
otherwise returns the lastpublic SingleOutputStreamOperator<T> maxBy(String field, boolean first)
DataStream
S underlying type. A dot can be
used to drill down into objects, as in "field1.getInnerField2()"
.field
- The field expression based on which the aggregation will be applied.first
- If True then in case of field equality the first object will be returnedpublic StreamExecutionEnvironment getExecutionEnvironment()
public TypeInformation<T> getInputType()
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.