org.apache.flink.streaming.api.scala

Class WindowedStream<T,K,W extends Window>

java.lang.Object

org.apache.flink.streaming.api.scala.WindowedStream<T,K,W>

public class WindowedStream<T,K,W extends Window>
extends Object

A WindowedStream represents a data stream where elements are grouped by key, and for each key, the stream of elements is split into windows based on a WindowAssigner. Window emission is triggered based on a Trigger.

The windows are conceptually evaluated for each key individually, meaning windows can trigger at different points for each key.

If an Evictor is specified it will be used to evict elements from the window after evaluation was triggered by the Trigger but before the actual evaluation of the window. When using an evictor window performance will degrade significantly, since pre-aggregation of window results cannot be used.

Note that the WindowedStream is purely and API construct, during runtime the WindowedStream will be collapsed together with the KeyedStream and the operation over the window into one single operation.

Constructor Summary

Constructors

Constructor and Description
WindowedStream(WindowedStream<T,K,W> javaStream)

Method Summary

Modifier and Type	Method and Description
DataStream<T>	aggregate(AggregationFunction.AggregationType aggregationType, int position)
<R> DataStream<R>	apply(scala.Function2<T,T,T> preAggregator, scala.Function4<K,W,scala.collection.Iterable<T>,Collector<R>,scala.runtime.BoxedUnit> windowFunction, TypeInformation<R> evidence$6) Applies the given window function to each window.
<R> DataStream<R>	apply(scala.Function4<K,W,scala.collection.Iterable<T>,Collector<R>,scala.runtime.BoxedUnit> function, TypeInformation<R> evidence$4) Applies the given window function to each window.
<R> DataStream<R>	apply(ReduceFunction<T> preAggregator, WindowFunction<T,R,K,W> function, TypeInformation<R> evidence$5) Applies the given window function to each window.
<R> DataStream<R>	apply(R initialValue, FoldFunction<T,R> foldFunction, WindowFunction<R,R,K,W> function, TypeInformation<R> evidence$7) Applies the given window function to each window.
<R> DataStream<R>	apply(R initialValue, scala.Function2<R,T,R> foldFunction, scala.Function4<K,W,scala.collection.Iterable<R>,Collector<R>,scala.runtime.BoxedUnit> windowFunction, TypeInformation<R> evidence$8) Applies the given window function to each window.
<R> DataStream<R>	apply(WindowFunction<T,R,K,W> function, TypeInformation<R> evidence$3) Applies the given window function to each window.
<F> F	clean(F f) Returns a "closure-cleaned" version of the given function.
WindowedStream<T,K,W>	evictor(Evictor<? super T,? super W> evictor) Sets the Evictor that should be used to evict elements from a window before emission.
<R> DataStream<R>	fold(R initialValue, FoldFunction<T,R> function, TypeInformation<R> evidence$1) Applies the given fold function to each window.
<R> DataStream<R>	fold(R initialValue, scala.Function2<R,T,R> function, TypeInformation<R> evidence$2) Applies the given fold function to each window.
DataStream<T>	max(int position) Applies an aggregation that that gives the maximum of the elements in the window at the given position.
DataStream<T>	max(String field) Applies an aggregation that that gives the maximum of the elements in the window at the given field.
DataStream<T>	maxBy(int position) Applies an aggregation that that gives the maximum element of the window by the given position.
DataStream<T>	maxBy(String field) Applies an aggregation that that gives the maximum element of the window by the given field.
DataStream<T>	min(int position) Applies an aggregation that that gives the minimum of the elements in the window at the given position.
DataStream<T>	min(String field) Applies an aggregation that that gives the minimum of the elements in the window at the given field.
DataStream<T>	minBy(int position) Applies an aggregation that that gives the minimum element of the window by the given position.
DataStream<T>	minBy(String field) Applies an aggregation that that gives the minimum element of the window by the given field.
DataStream<T>	reduce(scala.Function2<T,T,T> function) Applies a reduce function to the window.
DataStream<T>	reduce(ReduceFunction<T> function) Applies a reduce function to the window.
DataStream<T>	sum(int position) Applies an aggregation that sums the elements in the window at the given position.
DataStream<T>	sum(String field) Applies an aggregation that sums the elements in the window at the given field.
WindowedStream<T,K,W>	trigger(Trigger<? super T,? super W> trigger) Sets the Trigger that should be used to trigger window emission.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WindowedStream

public WindowedStream(WindowedStream<T,K,W> javaStream)

Method Detail

trigger

public WindowedStream<T,K,W> trigger(Trigger<? super T,? super W> trigger)

Sets the Trigger that should be used to trigger window emission.

evictor

public WindowedStream<T,K,W> evictor(Evictor<? super T,? super W> evictor)

Sets the Evictor that should be used to evict elements from a window before emission.

Note: When using an evictor window performance will degrade significantly, since pre-aggregation of window results cannot be used.

reduce

public DataStream<T> reduce(ReduceFunction<T> function)

Applies a reduce function to the window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

This window will try and pre-aggregate data as much as the window policies permit. For example, tumbling time windows can perfectly pre-aggregate the data, meaning that only one element per key is stored. Sliding time windows will pre-aggregate on the granularity of the slide interval, so a few elements are stored per key (one per slide interval). Custom windows may not be able to pre-aggregate, or may need to store extra values in an aggregation tree.

Parameters:

function - The reduce function.

Returns:

The data stream that is the result of applying the reduce function to the window.

reduce

public DataStream<T> reduce(scala.Function2<T,T,T> function)

Applies a reduce function to the window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

This window will try and pre-aggregate data as much as the window policies permit. For example, tumbling time windows can perfectly pre-aggregate the data, meaning that only one element per key is stored. Sliding time windows will pre-aggregate on the granularity of the slide interval, so a few elements are stored per key (one per slide interval). Custom windows may not be able to pre-aggregate, or may need to store extra values in an aggregation tree.

Parameters:

function - The reduce function.

Returns:

The data stream that is the result of applying the reduce function to the window.

fold

public <R> DataStream<R> fold(R initialValue,
                              FoldFunction<T,R> function,
                              TypeInformation<R> evidence$1)

Applies the given fold function to each window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

Parameters:

function - The fold function.

Returns:

The data stream that is the result of applying the fold function to the window.

fold

public <R> DataStream<R> fold(R initialValue,
                              scala.Function2<R,T,R> function,
                              TypeInformation<R> evidence$2)

Applies the given fold function to each window. The window function is called for each evaluation of the window for each key individually. The output of the reduce function is interpreted as a regular non-windowed stream.

Parameters:

function - The fold function.

Returns:

The data stream that is the result of applying the fold function to the window.

apply

public <R> DataStream<R> apply(WindowFunction<T,R,K,W> function,
                               TypeInformation<R> evidence$3)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Not that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of pre-aggregation.

Parameters:

function - The window function.

Returns:

The data stream that is the result of applying the window function to the window.

apply

public <R> DataStream<R> apply(scala.Function4<K,W,scala.collection.Iterable<T>,Collector<R>,scala.runtime.BoxedUnit> function,
                               TypeInformation<R> evidence$4)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Not that this function requires that all data in the windows is buffered until the window is evaluated, as the function provides no means of pre-aggregation.

Parameters:

function - The window function.

Returns:

The data stream that is the result of applying the window function to the window.

apply

public <R> DataStream<R> apply(ReduceFunction<T> preAggregator,
                               WindowFunction<T,R,K,W> function,
                               TypeInformation<R> evidence$5)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is pre-aggregated using the given pre-aggregation reducer.

Parameters:

preAggregator - The reduce function that is used for pre-aggregation

function - The window function.

Returns:

The data stream that is the result of applying the window function to the window.

apply

public <R> DataStream<R> apply(scala.Function2<T,T,T> preAggregator,
                               scala.Function4<K,W,scala.collection.Iterable<T>,Collector<R>,scala.runtime.BoxedUnit> windowFunction,
                               TypeInformation<R> evidence$6)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is pre-aggregated using the given pre-aggregation reducer.

Parameters:

preAggregator - The reduce function that is used for pre-aggregation

windowFunction - The window function.

Returns:

The data stream that is the result of applying the window function to the window.

apply

public <R> DataStream<R> apply(R initialValue,
                               FoldFunction<T,R> foldFunction,
                               WindowFunction<R,R,K,W> function,
                               TypeInformation<R> evidence$7)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is incrementally aggregated using the given fold function.

Parameters:

initialValue - The initial value of the fold

foldFunction - The fold function that is used for incremental aggregation

function - The window function.

Returns:

The data stream that is the result of applying the window function to the window.

apply

public <R> DataStream<R> apply(R initialValue,
                               scala.Function2<R,T,R> foldFunction,
                               scala.Function4<K,W,scala.collection.Iterable<R>,Collector<R>,scala.runtime.BoxedUnit> windowFunction,
                               TypeInformation<R> evidence$8)

Applies the given window function to each window. The window function is called for each evaluation of the window for each key individually. The output of the window function is interpreted as a regular non-windowed stream.

Arriving data is incrementally aggregated using the given fold function.

Parameters:

foldFunction - The fold function that is used for incremental aggregation

windowFunction - The window function.

Returns:

The data stream that is the result of applying the window function to the window.

max

public DataStream<T> max(int position)

Applies an aggregation that that gives the maximum of the elements in the window at the given position.

max

public DataStream<T> max(String field)

Applies an aggregation that that gives the maximum of the elements in the window at the given field.

min

public DataStream<T> min(int position)

Applies an aggregation that that gives the minimum of the elements in the window at the given position.

min

public DataStream<T> min(String field)

Applies an aggregation that that gives the minimum of the elements in the window at the given field.

sum

public DataStream<T> sum(int position)

Applies an aggregation that sums the elements in the window at the given position.

sum

public DataStream<T> sum(String field)

Applies an aggregation that sums the elements in the window at the given field.

maxBy

public DataStream<T> maxBy(int position)

Applies an aggregation that that gives the maximum element of the window by the given position. When equality, returns the first.

maxBy

public DataStream<T> maxBy(String field)

Applies an aggregation that that gives the maximum element of the window by the given field. When equality, returns the first.

minBy

public DataStream<T> minBy(int position)

Applies an aggregation that that gives the minimum element of the window by the given position. When equality, returns the first.

minBy

public DataStream<T> minBy(String field)

Applies an aggregation that that gives the minimum element of the window by the given field. When equality, returns the first.

aggregate

public DataStream<T> aggregate(AggregationFunction.AggregationType aggregationType,
                               int position)

clean

public <F> F clean(F f)

Returns a "closure-cleaned" version of the given function. Cleans only if closure cleaning is not disabled in the ExecutionConfig.