org.apache.flink.streaming.api.datastream

Class CoGroupedStreams<T1,T2>

java.lang.Object

org.apache.flink.streaming.api.datastream.CoGroupedStreams<T1,T2>

@Public
public class CoGroupedStreams<T1,T2>
extends Object

CoGroupedStreams represents two DataStreams that have been co-grouped. A streaming co-group operation is evaluated over elements in a window.

To finalize co-group operation you also need to specify a KeySelector for both the first and second input and a WindowAssigner.

Note: Right now, the groups are being built in memory so you need to ensure that they don't get too big. Otherwise the JVM might crash.

Example:

 DataStream<Tuple2<String, Integer>> one = ...;
 DataStream<Tuple2<String, Integer>> two = ...;

 DataStream<T> result = one.coGroup(two)
     .where(new MyFirstKeySelector())
     .equalTo(new MyFirstKeySelector())
     .window(TumblingEventTimeWindows.of(Duration.ofSeconds(5)))
     .apply(new MyCoGroupFunction());
 

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CoGroupedStreams.TaggedUnion<T1,T2> Internal class for implementing tagged union co-group.
static class	CoGroupedStreams.UnionSerializer<T1,T2> TypeSerializer for CoGroupedStreams.TaggedUnion.
static class	CoGroupedStreams.UnionSerializerSnapshot<T1,T2> The TypeSerializerSnapshot for the CoGroupedStreams.UnionSerializer.
class	CoGroupedStreams.Where<KEY> CoGrouped streams that have the key for one side defined.
static class	CoGroupedStreams.WithWindow<T1,T2,KEY,W extends Window> A co-group operation that has KeySelectors defined for both inputs as well as a WindowAssigner.

Constructor Summary

Constructors

Constructor and Description
CoGroupedStreams(DataStream<T1> input1, DataStream<T2> input2) Creates new CoGrouped data streams, which are the first step towards building a streaming co-group.

Method Summary

Modifier and Type	Method and Description
<KEY> CoGroupedStreams.Where<KEY>	where(KeySelector<T1,KEY> keySelector) Specifies a KeySelector for elements from the first input.
<KEY> CoGroupedStreams.Where<KEY>	where(KeySelector<T1,KEY> keySelector, TypeInformation<KEY> keyType) Specifies a KeySelector for elements from the first input with explicit type information.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CoGroupedStreams

public CoGroupedStreams(DataStream<T1> input1,
                        DataStream<T2> input2)

Creates new CoGrouped data streams, which are the first step towards building a streaming co-group.

Parameters:

input1 - The first data stream.

input2 - The second data stream.

Method Detail

where

public <KEY> CoGroupedStreams.Where<KEY> where(KeySelector<T1,KEY> keySelector)

Specifies a KeySelector for elements from the first input.

Parameters:

keySelector - The KeySelector to be used for extracting the first input's key for partitioning.

where

public <KEY> CoGroupedStreams.Where<KEY> where(KeySelector<T1,KEY> keySelector,
                                               TypeInformation<KEY> keyType)

Specifies a KeySelector for elements from the first input with explicit type information.

Parameters:

keySelector - The KeySelector to be used for extracting the first input's key for partitioning.

keyType - The type information describing the key type.