Interface CoGroupFunction<IN1,​IN2,​O>

  • Type Parameters:
    IN1 - The data type of the first input data stream.
    IN2 - The data type of the second input data stream.
    O - The data type of the returned elements.
    All Superinterfaces:
    Function, Serializable
    All Known Implementing Classes:
    RichCoGroupFunction
    Functional Interface:
    This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.

    @Public
    @FunctionalInterface
    public interface CoGroupFunction<IN1,​IN2,​O>
    extends Function, Serializable
    The interface for CoGroup functions. CoGroup functions combine two DataStreams by first grouping each data stream after a key and then "joining" the groups by calling this function with the two streams for each key. If a key is present in only one of the two inputs, it may be that one of the groups is empty.

    The basic syntax for using CoGroup on two data streams is as follows:

    
     DataStream<X> stream1 = ...;
     DataStream<Y> stream2 = ...;
    
     stream1.coGroup(stream2)
            .where(<key-definition>)
            .equalTo(<key-definition>)
            .window(<windowAssigner>)
            .apply(new MyCoGroupFunction());
     

    stream1 is here considered the first input, stream2 the second input.

    Some keys may only be contained in one of the two original data streams. In that case, the CoGroup function is invoked with in empty input for the side of the data stream that did not contain elements with that specific key.

    • Method Detail

      • coGroup

        void coGroup​(Iterable<IN1> first,
                     Iterable<IN2> second,
                     Collector<O> out)
              throws Exception
        This method must be implemented to provide a user implementation of a coGroup. It is called for each pair of element groups where the elements share the same key.
        Parameters:
        first - The records from the first input.
        second - The records from the second.
        out - A collector to return elements.
        Throws:
        Exception - The function may throw Exceptions, which will cause the program to cancel, and may trigger the recovery logic.