org.apache.flink.streaming.api.datastream

Class WindowedStream<T,K,W extends Window>

java.lang.Object

org.apache.flink.streaming.api.datastream.WindowedStream<T,K,W>

Type Parameters:

T - The type of elements in the stream.

K - The type of the key by which elements are grouped.

W - The type of Window that the WindowAssigner assigns the elements to.

@Public
public class WindowedStream<T,K,W extends Window>
extends Object

A WindowedStream represents a data stream where elements are grouped by key, and for each key, the stream of elements is split into windows based on a WindowAssigner. Window emission is triggered based on a Trigger.

The windows are conceptually evaluated for each key individually, meaning windows can trigger at different points for each key.

If an Evictor is specified it will be used to evict elements from the window after evaluation was triggered by the Trigger but before the actual evaluation of the window. When using an evictor window performance will degrade significantly, since incremental aggregation of window results cannot be used.

Note that the WindowedStream is purely and API construct, during runtime the WindowedStream will be collapsed together with the KeyedStream and the operation over the window into one single operation.

Constructor Summary

Constructors

Constructor and Description
WindowedStream(KeyedStream<T,K> input, WindowAssigner<? super T,W> windowAssigner)

Method Summary

Modifier and Type	Method and Description
<R> SingleOutputStreamOperator<R>	apply(ReduceFunction<T> reduceFunction, WindowFunction<T,R,K,W> function) Applies the given window function to each window.
<R> SingleOutputStreamOperator<R>	apply(ReduceFunction<T> reduceFunction, WindowFunction<T,R,K,W> function, TypeInformation<R> resultType) Applies the given window function to each window.
<R> SingleOutputStreamOperator<R>	apply(R initialValue, FoldFunction<T,R> foldFunction, WindowFunction<R,R,K,W> function) Applies the given window function to each window.
<R> SingleOutputStreamOperator<R>	apply(R initialValue, FoldFunction<T,R> foldFunction, WindowFunction<R,R,K,W> function, TypeInformation<R> resultType) Applies the given window function to each window.
<R> SingleOutputStreamOperator<R>	apply(WindowFunction<T,R,K,W> function) Applies the given window function to each window.
<R> SingleOutputStreamOperator<R>	apply(WindowFunction<T,R,K,W> function, TypeInformation<R> resultType) Applies the given window function to each window.
WindowedStream<T,K,W>	evictor(Evictor<? super T,? super W> evictor) Sets the Evictor that should be used to evict elements from a window before emission.
<R> SingleOutputStreamOperator<R>	fold(R initialValue, FoldFunction<T,R> function) Applies the given fold function to each window.
<R> SingleOutputStreamOperator<R>	fold(R initialValue, FoldFunction<T,R> function, TypeInformation<R> resultType) Applies the given fold function to each window.
StreamExecutionEnvironment	getExecutionEnvironment()
TypeInformation<T>	getInputType()
SingleOutputStreamOperator<T>	max(int positionToMax) Applies an aggregation that gives the maximum value of every window of the data stream at the given position.
SingleOutputStreamOperator<T>	max(String field) Applies an aggregation that that gives the maximum value of the pojo data stream at the given field expression for every window.
SingleOutputStreamOperator<T>	maxBy(int positionToMaxBy) Applies an aggregation that gives the maximum element of every window of the data stream by the given position.
SingleOutputStreamOperator<T>	maxBy(int positionToMaxBy, boolean first) Applies an aggregation that gives the maximum element of every window of the data stream by the given position.
SingleOutputStreamOperator<T>	maxBy(String positionToMaxBy) Applies an aggregation that gives the maximum element of every window of the data stream by the given position.
SingleOutputStreamOperator<T>	maxBy(String field, boolean first) Applies an aggregation that that gives the maximum element of the pojo data stream by the given field expression for every window.
SingleOutputStreamOperator<T>	min(int positionToMin) Applies an aggregation that that gives the minimum value of every window of the data stream at the given position.
SingleOutputStreamOperator<T>	min(String field) Applies an aggregation that that gives the minimum value of the pojo data stream at the given field expression for every window.
SingleOutputStreamOperator<T>	minBy(int positionToMinBy) Applies an aggregation that gives the minimum element of every window of the data stream by the given position.
SingleOutputStreamOperator<T>	minBy(int positionToMinBy, boolean first) Applies an aggregation that gives the minimum element of every window of the data stream by the given position.
SingleOutputStreamOperator<T>	minBy(String positionToMinBy) Applies an aggregation that gives the minimum element of every window of the data stream by the given position.
SingleOutputStreamOperator<T>	minBy(String field, boolean first) Applies an aggregation that that gives the minimum element of the pojo data stream by the given field expression for every window.
SingleOutputStreamOperator<T>	reduce(ReduceFunction<T> function) Applies a reduce function to the window.
SingleOutputStreamOperator<T>	sum(int positionToSum) Applies an aggregation that sums every window of the data stream at the given position.
SingleOutputStreamOperator<T>	sum(String field) Applies an aggregation that sums every window of the pojo data stream at the given field for every window.
WindowedStream<T,K,W>	trigger(Trigger<? super T,? super W> trigger) Sets the Trigger that should be used to trigger window emission.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WindowedStream

@PublicEvolving
public WindowedStream(KeyedStream<T,K> input,
                                      WindowAssigner<? super T,W> windowAssigner)

Method Detail

trigger

@PublicEvolving
public WindowedStream<T,K,W> trigger(Trigger<? super T,? super W> trigger)

Sets the Trigger that should be used to trigger window emission.

evictor

@PublicEvolving
public WindowedStream<T,K,W> evictor(Evictor<? super T,? super W> evictor)

Sets the Evictor that should be used to evict elements from a window before emission.

Note: When using an evictor window performance will degrade significantly, since incremental aggregation of window results cannot be used.

reduce

public SingleOutputStreamOperator<T> reduce(ReduceFunction<T> function)

Applies a reduce function to the window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

This window will try and incrementally aggregate data as much as the window policies permit. For example, tumbling time windows can aggregate the data, meaning that only one element per key is stored. Sliding time windows will aggregate on the granularity of the slide interval, so a few elements are stored per key (one per slide interval). Custom windows may not be able to incrementally aggregate, or may need to store extra values in an aggregation tree.

Parameters:

function - The reduce function.

Returns:

The data stream that is the result of applying the reduce function to the window.

fold

public <R> SingleOutputStreamOperator<R> fold(R initialValue,
                                              FoldFunction<T,R> function)

Applies the given fold function to each window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

Parameters:

function - The fold function.

Returns:

The data stream that is the result of applying the fold function to the window.

fold

public <R> SingleOutputStreamOperator<R> fold(R initialValue,
                                              FoldFunction<T,R> function,
                                              TypeInformation<R> resultType)

Applies the given fold function to each window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

Parameters:

function - The fold function.

Returns:

The data stream that is the result of applying the fold function to the window.

apply

public <R> SingleOutputStreamOperator<R> apply(WindowFunction<T,R,K,W> function)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Not that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of incremental aggregation.

Parameters:

function - The window function.

Returns:

The data stream that is the result of applying the window function to the window.

apply

public <R> SingleOutputStreamOperator<R> apply(WindowFunction<T,R,K,W> function,
                                               TypeInformation<R> resultType)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Not that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of incremental aggregation.

Parameters:

function - The window function.

resultType - Type information for the result type of the window function

Returns:

The data stream that is the result of applying the window function to the window.

apply

public <R> SingleOutputStreamOperator<R> apply(ReduceFunction<T> reduceFunction,
                                               WindowFunction<T,R,K,W> function)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is incrementally aggregated using the given reducer.

Parameters:

reduceFunction - The reduce function that is used for incremental aggregation.

function - The window function.

Returns:

The data stream that is the result of applying the window function to the window.

apply

public <R> SingleOutputStreamOperator<R> apply(ReduceFunction<T> reduceFunction,
                                               WindowFunction<T,R,K,W> function,
                                               TypeInformation<R> resultType)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is incrementally aggregated using the given reducer.

Parameters:

reduceFunction - The reduce function that is used for incremental aggregation.

function - The window function.

resultType - Type information for the result type of the window function

Returns:

The data stream that is the result of applying the window function to the window.

apply

public <R> SingleOutputStreamOperator<R> apply(R initialValue,
                                               FoldFunction<T,R> foldFunction,
                                               WindowFunction<R,R,K,W> function)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is incrementally aggregated using the given fold function.

Parameters:

initialValue - The initial value of the fold.

foldFunction - The fold function that is used for incremental aggregation.

function - The window function.

Returns:

The data stream that is the result of applying the window function to the window.

apply

public <R> SingleOutputStreamOperator<R> apply(R initialValue,
                                               FoldFunction<T,R> foldFunction,
                                               WindowFunction<R,R,K,W> function,
                                               TypeInformation<R> resultType)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is incrementally aggregated using the given fold function.

Parameters:

initialValue - The initial value of the fold.

foldFunction - The fold function that is used for incremental aggregation.

function - The window function.

resultType - Type information for the result type of the window function

Returns:

The data stream that is the result of applying the window function to the window.

sum

public SingleOutputStreamOperator<T> sum(int positionToSum)

Applies an aggregation that sums every window of the data stream at the given position.

Parameters:

positionToSum - The position in the tuple/array to sum

Returns:

The transformed DataStream.

sum

public SingleOutputStreamOperator<T> sum(String field)

Applies an aggregation that sums every window of the pojo data stream at the given field for every window.

A field expression is either the name of a public field or a getter method with parentheses of the stream's underlying type. A dot can be used to drill down into objects, as in "field1.getInnerField2()" .

Parameters:

field - The field to sum

Returns:

The transformed DataStream.

min

public SingleOutputStreamOperator<T> min(int positionToMin)

Applies an aggregation that that gives the minimum value of every window of the data stream at the given position.

Parameters:

positionToMin - The position to minimize

Returns:

The transformed DataStream.

min

public SingleOutputStreamOperator<T> min(String field)

Applies an aggregation that that gives the minimum value of the pojo data stream at the given field expression for every window.

A field expression is either the name of a public field or a getter method with parentheses of the DataStreamS underlying type. A dot can be used to drill down into objects, as in "field1.getInnerField2()" .

Parameters:

field - The field expression based on which the aggregation will be applied.

Returns:

The transformed DataStream.

minBy

public SingleOutputStreamOperator<T> minBy(int positionToMinBy)

Applies an aggregation that gives the minimum element of every window of the data stream by the given position. If more elements have the same minimum value the operator returns the first element by default.

Parameters:

positionToMinBy - The position to minimize by

Returns:

The transformed DataStream.

minBy

public SingleOutputStreamOperator<T> minBy(String positionToMinBy)

Applies an aggregation that gives the minimum element of every window of the data stream by the given position. If more elements have the same minimum value the operator returns the first element by default.

Parameters:

positionToMinBy - The position to minimize by

Returns:

The transformed DataStream.

minBy

public SingleOutputStreamOperator<T> minBy(int positionToMinBy,
                                           boolean first)

Applies an aggregation that gives the minimum element of every window of the data stream by the given position. If more elements have the same minimum value the operator returns either the first or last one depending on the parameter setting.

Parameters:

positionToMinBy - The position to minimize

first - If true, then the operator return the first element with the minimum value, otherwise returns the last

Returns:

The transformed DataStream.

minBy

public SingleOutputStreamOperator<T> minBy(String field,
                                           boolean first)

Applies an aggregation that that gives the minimum element of the pojo data stream by the given field expression for every window. A field expression is either the name of a public field or a getter method with parentheses of the DataStreams underlying type. A dot can be used to drill down into objects, as in "field1.getInnerField2()" .

Parameters:

field - The field expression based on which the aggregation will be applied.

first - If True then in case of field equality the first object will be returned

Returns:

The transformed DataStream.

max

public SingleOutputStreamOperator<T> max(int positionToMax)

Applies an aggregation that gives the maximum value of every window of the data stream at the given position.

Parameters:

positionToMax - The position to maximize

Returns:

The transformed DataStream.

max

public SingleOutputStreamOperator<T> max(String field)

Applies an aggregation that that gives the maximum value of the pojo data stream at the given field expression for every window. A field expression is either the name of a public field or a getter method with parentheses of the DataStreams underlying type. A dot can be used to drill down into objects, as in "field1.getInnerField2()" .

Parameters:

field - The field expression based on which the aggregation will be applied.

Returns:

The transformed DataStream.

maxBy

public SingleOutputStreamOperator<T> maxBy(int positionToMaxBy)

Applies an aggregation that gives the maximum element of every window of the data stream by the given position. If more elements have the same maximum value the operator returns the first by default.

Parameters:

positionToMaxBy - The position to maximize by

Returns:

The transformed DataStream.

maxBy

public SingleOutputStreamOperator<T> maxBy(String positionToMaxBy)

Applies an aggregation that gives the maximum element of every window of the data stream by the given position. If more elements have the same maximum value the operator returns the first by default.

Parameters:

positionToMaxBy - The position to maximize by

Returns:

The transformed DataStream.

maxBy

public SingleOutputStreamOperator<T> maxBy(int positionToMaxBy,
                                           boolean first)

Applies an aggregation that gives the maximum element of every window of the data stream by the given position. If more elements have the same maximum value the operator returns either the first or last one depending on the parameter setting.

Parameters:

positionToMaxBy - The position to maximize by

first - If true, then the operator return the first element with the maximum value, otherwise returns the last

Returns:

The transformed DataStream.

maxBy

public SingleOutputStreamOperator<T> maxBy(String field,
                                           boolean first)

Applies an aggregation that that gives the maximum element of the pojo data stream by the given field expression for every window. A field expression is either the name of a public field or a getter method with parentheses of the DataStreamS underlying type. A dot can be used to drill down into objects, as in "field1.getInnerField2()" .

Parameters:

field - The field expression based on which the aggregation will be applied.

first - If True then in case of field equality the first object will be returned

Returns:

The transformed DataStream.

getExecutionEnvironment

public StreamExecutionEnvironment getExecutionEnvironment()

getInputType

public TypeInformation<T> getInputType()