org.apache.flink.streaming.api.scala

Class KeyedStream<T,K>

java.lang.Object

org.apache.flink.streaming.api.scala.DataStream<T>

org.apache.flink.streaming.api.scala.KeyedStream<T,K>

public class KeyedStream<T,K>
extends DataStream<T>

Constructor Summary

Constructors

Constructor and Description
KeyedStream(KeyedStream<T,K> javaStream)

Method Summary

Modifier and Type	Method and Description
WindowedStream<T,K,GlobalWindow>	countWindow(long size) Windows this KeyedStream into tumbling count windows.
WindowedStream<T,K,GlobalWindow>	countWindow(long size, long slide) Windows this KeyedStream into sliding count windows.
<S> DataStream<T>	filterWithState(scala.Function2<T,scala.Option<S>,scala.Tuple2<Object,scala.Option<S>>> fun, TypeInformation<S> evidence$3) Creates a new DataStream that contains only the elements satisfying the given stateful filter predicate.
<R,S> DataStream<R>	flatMapWithState(scala.Function2<T,scala.Option<S>,scala.Tuple2<scala.collection.TraversableOnce<R>,scala.Option<S>>> fun, TypeInformation<R> evidence$6, TypeInformation<S> evidence$7) Creates a new DataStream by applying the given stateful function to every element and flattening the results.
<R> DataStream<R>	fold(R initialValue, FoldFunction<T,R> folder, TypeInformation<R> evidence$1) Creates a new DataStream by folding the elements of this DataStream using an associative fold function and an initial value.
<R> DataStream<R>	fold(R initialValue, scala.Function2<R,T,R> fun, TypeInformation<R> evidence$2) Creates a new DataStream by folding the elements of this DataStream using an associative fold function and an initial value.
TypeInformation<K>	getKeyType() Gets the type of the key by which this stream is keyed.
<R,S> DataStream<R>	mapWithState(scala.Function2<T,scala.Option<S>,scala.Tuple2<R,scala.Option<S>>> fun, TypeInformation<R> evidence$4, TypeInformation<S> evidence$5) Creates a new DataStream by applying the given stateful function to every element of this DataStream.
DataStream<T>	max(int position) Applies an aggregation that that gives the current maximum of the data stream at the given position by the given key.
DataStream<T>	max(String field) Applies an aggregation that that gives the current maximum of the data stream at the given field by the given key.
DataStream<T>	maxBy(int position) Applies an aggregation that that gives the current maximum element of the data stream by the given position by the given key.
DataStream<T>	maxBy(String field) Applies an aggregation that that gives the current maximum element of the data stream by the given field by the given key.
DataStream<T>	min(int position) Applies an aggregation that that gives the current minimum of the data stream at the given position by the given key.
DataStream<T>	min(String field) Applies an aggregation that that gives the current minimum of the data stream at the given field by the given key.
DataStream<T>	minBy(int position) Applies an aggregation that that gives the current minimum element of the data stream by the given position by the given key.
DataStream<T>	minBy(String field) Applies an aggregation that that gives the current minimum element of the data stream by the given field by the given key.
DataStream<T>	reduce(scala.Function2<T,T,T> fun) Creates a new DataStream by reducing the elements of this DataStream using an associative reduce function.
DataStream<T>	reduce(ReduceFunction<T> reducer) Creates a new DataStream by reducing the elements of this DataStream using an associative reduce function.
DataStream<T>	sum(int position) Applies an aggregation that sums the data stream at the given position by the given key.
DataStream<T>	sum(String field) Applies an aggregation that sums the data stream at the given field by the given key.
WindowedStream<T,K,TimeWindow>	timeWindow(Time size) Windows this KeyedStream into tumbling time windows.
WindowedStream<T,K,TimeWindow>	timeWindow(Time size, Time slide) Windows this KeyedStream into sliding time windows.
<W extends Window> WindowedStream<T,K,W>	window(WindowAssigner<? super T,W> assigner) Windows this data stream to a WindowedStream, which evaluates windows over a key grouped stream.

Methods inherited from class org.apache.flink.streaming.api.scala.DataStream

addSink, addSink, assignAscendingTimestamps, assignTimestamps, assignTimestampsAndWatermarks, assignTimestampsAndWatermarks, broadcast, clean, coGroup, connect, countWindowAll, countWindowAll, dataType, disableChaining, executionConfig, executionEnvironment, filter, filter, flatMap, flatMap, flatMap, forward, getExecutionConfig, getExecutionEnvironment, getId, getName, getParallelism, getType, global, iterate, iterate, javaStream, join, keyBy, keyBy, keyBy, map, map, name, name, parallelism, partitionCustom, partitionCustom, partitionCustom, print, printToErr, rebalance, rescale, setBufferTimeout, setParallelism, shuffle, slotSharingGroup, split, split, startNewChain, timeWindowAll, timeWindowAll, transform, uid, union, windowAll, writeAsCsv, writeAsCsv, writeAsCsv, writeAsText, writeAsText, writeToSocket, writeUsingOutputFormat

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KeyedStream

public KeyedStream(KeyedStream<T,K> javaStream)

Method Detail

getKeyType

public TypeInformation<K> getKeyType()

Gets the type of the key by which this stream is keyed.

timeWindow

public WindowedStream<T,K,TimeWindow> timeWindow(Time size)

Windows this KeyedStream into tumbling time windows.

This is a shortcut for either .window(TumblingEventTimeWindows.of(size)) or .window(TumblingProcessingTimeWindows.of(size)) depending on the time characteristic set using StreamExecutionEnvironment.setStreamTimeCharacteristic()

Parameters:

size - The size of the window.

countWindow

public WindowedStream<T,K,GlobalWindow> countWindow(long size,
                                                    long slide)

Windows this KeyedStream into sliding count windows.

Parameters:

size - The size of the windows in number of elements.

slide - The slide interval in number of elements.

countWindow

public WindowedStream<T,K,GlobalWindow> countWindow(long size)

Windows this KeyedStream into tumbling count windows.

Parameters:

size - The size of the windows in number of elements.

timeWindow

public WindowedStream<T,K,TimeWindow> timeWindow(Time size,
                                                 Time slide)

Windows this KeyedStream into sliding time windows.

This is a shortcut for either .window(SlidingEventTimeWindows.of(size)) or .window(SlidingProcessingTimeWindows.of(size)) depending on the time characteristic set using StreamExecutionEnvironment.setStreamTimeCharacteristic()

Parameters:

size - The size of the window.

window

public <W extends Window> WindowedStream<T,K,W> window(WindowAssigner<? super T,W> assigner)

Windows this data stream to a WindowedStream, which evaluates windows over a key grouped stream. Elements are put into windows by a WindowAssigner. The grouping of elements is done both by key and by window.

A Trigger can be defined to specify when windows are evaluated. However, WindowAssigner have a default Trigger that is used if a Trigger is not specified.

Parameters:

assigner - The WindowAssigner that assigns elements to windows.

Returns:

The trigger windows data stream.

reduce

public DataStream<T> reduce(ReduceFunction<T> reducer)

Creates a new DataStream by reducing the elements of this DataStream using an associative reduce function. An independent aggregate is kept per key.

reduce

public DataStream<T> reduce(scala.Function2<T,T,T> fun)

Creates a new DataStream by reducing the elements of this DataStream using an associative reduce function. An independent aggregate is kept per key.

fold

public <R> DataStream<R> fold(R initialValue,
                              FoldFunction<T,R> folder,
                              TypeInformation<R> evidence$1)

Creates a new DataStream by folding the elements of this DataStream using an associative fold function and an initial value. An independent aggregate is kept per key.

fold

public <R> DataStream<R> fold(R initialValue,
                              scala.Function2<R,T,R> fun,
                              TypeInformation<R> evidence$2)

Creates a new DataStream by folding the elements of this DataStream using an associative fold function and an initial value. An independent aggregate is kept per key.

max

public DataStream<T> max(int position)

Applies an aggregation that that gives the current maximum of the data stream at the given position by the given key. An independent aggregate is kept per key.

max

public DataStream<T> max(String field)

Applies an aggregation that that gives the current maximum of the data stream at the given field by the given key. An independent aggregate is kept per key.

min

public DataStream<T> min(int position)

Applies an aggregation that that gives the current minimum of the data stream at the given position by the given key. An independent aggregate is kept per key.

min

public DataStream<T> min(String field)

Applies an aggregation that that gives the current minimum of the data stream at the given field by the given key. An independent aggregate is kept per key.

sum

public DataStream<T> sum(int position)

Applies an aggregation that sums the data stream at the given position by the given key. An independent aggregate is kept per key.

sum

public DataStream<T> sum(String field)

Applies an aggregation that sums the data stream at the given field by the given key. An independent aggregate is kept per key.

minBy

public DataStream<T> minBy(int position)

Applies an aggregation that that gives the current minimum element of the data stream by the given position by the given key. An independent aggregate is kept per key. When equality, the first element is returned with the minimal value.

minBy

public DataStream<T> minBy(String field)

Applies an aggregation that that gives the current minimum element of the data stream by the given field by the given key. An independent aggregate is kept per key. When equality, the first element is returned with the minimal value.

maxBy

public DataStream<T> maxBy(int position)

Applies an aggregation that that gives the current maximum element of the data stream by the given position by the given key. An independent aggregate is kept per key. When equality, the first element is returned with the maximal value.

maxBy

public DataStream<T> maxBy(String field)

Applies an aggregation that that gives the current maximum element of the data stream by the given field by the given key. An independent aggregate is kept per key. When equality, the first element is returned with the maximal value.

filterWithState

public <S> DataStream<T> filterWithState(scala.Function2<T,scala.Option<S>,scala.Tuple2<Object,scala.Option<S>>> fun,
                                         TypeInformation<S> evidence$3)

Creates a new DataStream that contains only the elements satisfying the given stateful filter predicate. To use state partitioning, a key must be defined using .keyBy(..), in which case an independent state will be kept per key.

Note that the user state object needs to be serializable.

mapWithState

public <R,S> DataStream<R> mapWithState(scala.Function2<T,scala.Option<S>,scala.Tuple2<R,scala.Option<S>>> fun,
                                        TypeInformation<R> evidence$4,
                                        TypeInformation<S> evidence$5)

Creates a new DataStream by applying the given stateful function to every element of this DataStream. To use state partitioning, a key must be defined using .keyBy(..), in which case an independent state will be kept per key.

Note that the user state object needs to be serializable.

flatMapWithState

public <R,S> DataStream<R> flatMapWithState(scala.Function2<T,scala.Option<S>,scala.Tuple2<scala.collection.TraversableOnce<R>,scala.Option<S>>> fun,
                                            TypeInformation<R> evidence$6,
                                            TypeInformation<S> evidence$7)

Creates a new DataStream by applying the given stateful function to every element and flattening the results. To use state partitioning, a key must be defined using .keyBy(..), in which case an independent state will be kept per key.

Note that the user state object needs to be serializable.