org.apache.flink.streaming.api.scala

Class ConnectedStreams<IN1,IN2>

java.lang.Object

org.apache.flink.streaming.api.scala.ConnectedStreams<IN1,IN2>

public class ConnectedStreams<IN1,IN2>
extends Object

ConnectedStreams represents two connected streams of (possibly) different data types. Connected streams are useful for cases where operations on one stream directly affect the operations on the other stream, usually via shared state between the streams.

An example for the use of connected streams would be to apply rules that change over time onto another stream. One of the connected streams has the rules, the other stream the elements to apply the rules to. The operation on the connected stream maintains the current set of rules in the state. It may receive either a rule update and update the state or a data element and apply the rules in the state to the element.

The connected stream can be conceptually viewed as a union stream of an Either type, that holds either the first stream's type or the second stream's type.

Constructor Summary

Constructors

Constructor and Description
ConnectedStreams(ConnectedStreams<IN1,IN2> javaStream)

Method Summary

Modifier and Type	Method and Description
<F> F	clean(F f) Returns a "closure-cleaned" version of the given function.
<R> DataStream<R>	flatMap(CoFlatMapFunction<IN1,IN2,R> coFlatMapper, TypeInformation<R> evidence$3) Applies a CoFlatMap transformation on these connected streams.
<R> DataStream<R>	flatMap(scala.Function1<IN1,scala.collection.TraversableOnce<R>> fun1, scala.Function1<IN2,scala.collection.TraversableOnce<R>> fun2, TypeInformation<R> evidence$5) Applies a CoFlatMap transformation on the connected streams.
<R> DataStream<R>	flatMap(scala.Function2<IN1,Collector<R>,scala.runtime.BoxedUnit> fun1, scala.Function2<IN2,Collector<R>,scala.runtime.BoxedUnit> fun2, TypeInformation<R> evidence$4) Applies a CoFlatMap transformation on the connected streams.
<K1,K2> ConnectedStreams<IN1,IN2>	keyBy(scala.Function1<IN1,K1> fun1, scala.Function1<IN2,K2> fun2, TypeInformation<K1> evidence$6, TypeInformation<K2> evidence$7) Keys the two connected streams together.
ConnectedStreams<IN1,IN2>	keyBy(int[] keyPositions1, int[] keyPositions2) Keys the two connected streams together.
ConnectedStreams<IN1,IN2>	keyBy(int keyPosition1, int keyPosition2) Keys the two connected streams together.
ConnectedStreams<IN1,IN2>	keyBy(String[] fields1, String[] fields2) Keys the two connected streams together.
ConnectedStreams<IN1,IN2>	keyBy(String field1, String field2) Keys the two connected streams together.
<R> DataStream<R>	map(CoMapFunction<IN1,IN2,R> coMapper, TypeInformation<R> evidence$2) Applies a CoMap transformation on these connected streams.
<R> DataStream<R>	map(scala.Function1<IN1,R> fun1, scala.Function1<IN2,R> fun2, TypeInformation<R> evidence$1) Applies a CoMap transformation on the connected streams.
<R> DataStream<R>	transform(String functionName, TwoInputStreamOperator<IN1,IN2,R> operator, TypeInformation<R> evidence$8)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ConnectedStreams

public ConnectedStreams(ConnectedStreams<IN1,IN2> javaStream)

Method Detail

map

public <R> DataStream<R> map(scala.Function1<IN1,R> fun1,
                             scala.Function1<IN2,R> fun2,
                             TypeInformation<R> evidence$1)

Applies a CoMap transformation on the connected streams.

The transformation consists of two separate functions, where the first one is called for each element of the first connected stream, and the second one is called for each element of the second connected stream.

Parameters:

fun1 - Function called per element of the first input.

fun2 - Function called per element of the second input.

Returns:

The resulting data stream.

map

public <R> DataStream<R> map(CoMapFunction<IN1,IN2,R> coMapper,
                             TypeInformation<R> evidence$2)

Applies a CoMap transformation on these connected streams.

The transformation calls CoMapFunction#map1 for each element in the first stream and CoMapFunction#map2 for each element of the second stream.

On can pass a subclass of RichCoMapFunction to gain access to the RuntimeContext and to additional life cycle methods.

Parameters:

coMapper - The CoMapFunction used to transform the two connected streams

Returns:

The resulting data stream

flatMap

public <R> DataStream<R> flatMap(CoFlatMapFunction<IN1,IN2,R> coFlatMapper,
                                 TypeInformation<R> evidence$3)

Applies a CoFlatMap transformation on these connected streams.

The transformation calls CoFlatMapFunction#flatMap1 for each element in the first stream and CoFlatMapFunction#flatMap2 for each element of the second stream.

On can pass a subclass of RichCoFlatMapFunction to gain access to the RuntimeContext and to additional life cycle methods.

Parameters:

coFlatMapper - The CoFlatMapFunction used to transform the two connected streams

Returns:

The resulting data stream.

flatMap

public <R> DataStream<R> flatMap(scala.Function2<IN1,Collector<R>,scala.runtime.BoxedUnit> fun1,
                                 scala.Function2<IN2,Collector<R>,scala.runtime.BoxedUnit> fun2,
                                 TypeInformation<R> evidence$4)

Applies a CoFlatMap transformation on the connected streams.

The transformation consists of two separate functions, where the first one is called for each element of the first connected stream, and the second one is called for each element of the second connected stream.

Parameters:

fun1 - Function called per element of the first input.

fun2 - Function called per element of the second input.

Returns:

The resulting data stream.

flatMap

public <R> DataStream<R> flatMap(scala.Function1<IN1,scala.collection.TraversableOnce<R>> fun1,
                                 scala.Function1<IN2,scala.collection.TraversableOnce<R>> fun2,
                                 TypeInformation<R> evidence$5)

Applies a CoFlatMap transformation on the connected streams.

The transformation consists of two separate functions, where the first one is called for each element of the first connected stream, and the second one is called for each element of the second connected stream.

Parameters:

fun1 - Function called per element of the first input.

fun2 - Function called per element of the second input.

Returns:

The resulting data stream.

keyBy

public ConnectedStreams<IN1,IN2> keyBy(int keyPosition1,
                                       int keyPosition2)

Keys the two connected streams together. After this operation, all elements with the same key from both streams will be sent to the same parallel instance of the transformation functions.

Parameters:

keyPosition1 - The first stream's key field

keyPosition2 - The second stream's key field

Returns:

The key-grouped connected streams

keyBy

public ConnectedStreams<IN1,IN2> keyBy(int[] keyPositions1,
                                       int[] keyPositions2)

Keys the two connected streams together. After this operation, all elements with the same key from both streams will be sent to the same parallel instance of the transformation functions.

Parameters:

keyPositions1 - The first stream's key fields

keyPositions2 - The second stream's key fields

Returns:

The key-grouped connected streams

keyBy

public ConnectedStreams<IN1,IN2> keyBy(String field1,
                                       String field2)

Keys the two connected streams together. After this operation, all elements with the same key from both streams will be sent to the same parallel instance of the transformation functions.

Parameters:

field1 - The first stream's key expression

field2 - The second stream's key expression

Returns:

The key-grouped connected streams

keyBy

public ConnectedStreams<IN1,IN2> keyBy(String[] fields1,
                                       String[] fields2)

Keys the two connected streams together. After this operation, all elements with the same key from both streams will be sent to the same parallel instance of the transformation functions.

Parameters:

fields1 - The first stream's key expressions

fields2 - The second stream's key expressions

Returns:

The key-grouped connected streams

keyBy

public <K1,K2> ConnectedStreams<IN1,IN2> keyBy(scala.Function1<IN1,K1> fun1,
                                               scala.Function1<IN2,K2> fun2,
                                               TypeInformation<K1> evidence$6,
                                               TypeInformation<K2> evidence$7)

Keys the two connected streams together. After this operation, all elements with the same key from both streams will be sent to the same parallel instance of the transformation functions.

Parameters:

fun1 - The first stream's key function

fun2 - The second stream's key function

Returns:

The key-grouped connected streams

clean

public <F> F clean(F f)

Returns a "closure-cleaned" version of the given function. Cleans only if closure cleaning is not disabled in the ExecutionConfig

transform

public <R> DataStream<R> transform(String functionName,
                                   TwoInputStreamOperator<IN1,IN2,R> operator,
                                   TypeInformation<R> evidence$8)