org.apache.flink.api.common.functions

Interface CoGroupFunction<IN1,IN2,O>

Type Parameters:

IN1 - The data type of the first input data set.

IN2 - The data type of the second input data set.

O - The data type of the returned elements.

All Superinterfaces:

Function, Serializable

All Known Implementing Classes:

RichCoGroupFunction, WebLogAnalysis.AntiJoinVisits

Functional Interface:

This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.

@Public
 @FunctionalInterface
public interface CoGroupFunction<IN1,IN2,O>
extends Function, Serializable

The interface for CoGroup functions. CoGroup functions combine two data sets by first grouping each data set after a key and then "joining" the groups by calling this function with the two sets for each key. If a key is present in only one of the two inputs, it may be that one of the groups is empty.

The basic syntax for using CoGroup on two data sets is as follows:

 DataSet<X> set1 = ...;
 DataSet<Y> set2 = ...;

 set1.coGroup(set2).where(<key-definition>).equalTo(<key-definition>).with(new MyCoGroupFunction());
 

set1 is here considered the first input, set2 the second input.

Some keys may only be contained in one of the two original data sets. In that case, the CoGroup function is invoked with in empty input for the side of the data set that did not contain elements with that specific key.

Method Summary

Modifier and Type	Method and Description
void	coGroup(Iterable<IN1> first, Iterable<IN2> second, Collector<O> out) This method must be implemented to provide a user implementation of a coGroup.

Method Detail

coGroup

void coGroup(Iterable<IN1> first,
             Iterable<IN2> second,
             Collector<O> out)
      throws Exception

This method must be implemented to provide a user implementation of a coGroup. It is called for each pair of element groups where the elements share the same key.

Parameters:

first - The records from the first input.

second - The records from the second.

out - A collector to return elements.

Throws:

Exception - The function may throw Exceptions, which will cause the program to cancel, and may trigger the recovery logic.