Java 8 introduced several new language features designed for faster and clearer coding. With the most important feature,
the so-called “Lambda Expressions”, it opened the door to functional programming. Lambda expressions allow for implementing and
passing functions in a straightforward way without having to declare additional (anonymous) classes.
Attention Flink supports the usage of lambda expressions for all operators of the Java API, however, whenever a lambda expression uses Java generics you need to declare type information explicitly.
This document shows how to use lambda expressions and describes current
limitations. For a general introduction to the Flink API, please refer to the
DataSteam API overview
Examples and Limitations
The following example illustrates how to implement a simple, inline map() function that squares its input using a lambda expression.
The types of input i and output parameters of the map() function need not to be declared as they are inferred by the Java compiler.
Flink can automatically extract the result type information from the implementation of the method signature OUT map(IN value) because OUT is not generic but Integer.
Unfortunately, functions such as flatMap() with a signature void flatMap(IN value, Collector<OUT> out) are compiled into void flatMap(IN value, Collector out) by the Java compiler. This makes it impossible for Flink to infer the type information for the output type automatically.
Flink will most likely throw an exception similar to the following:
In this case, the type information needs to be specified explicitly, otherwise the output will be treated as type Object which leads to unefficient serialization.
Similar problems occur when using a map() function with a generic return type. A method signature Tuple2<Integer, Integer> map(Integer value) is erasured to Tuple2 map(Integer value) in the example below.
In general, those problems can be solved in multiple ways: