org.apache.flink.api.common.functions

Interface MapPartitionFunction<T,O>

Type Parameters:

T - Type of the input elements.

O - Type of the returned elements.

All Superinterfaces:

Function, Serializable

All Known Implementing Classes:

AssignRangeIndex, BoundedOneInputStreamTaskRunner, RangeBoundaryBuilder, RichMapPartitionFunction, SampleInPartition, SampleWithFraction

Functional Interface:

This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.

@Public
 @FunctionalInterface
public interface MapPartitionFunction<T,O>
extends Function, Serializable

Interface for "mapPartition" functions. A "mapPartition" function is called a single time per data partition receives an Iterable with data elements of that partition. It may return an arbitrary number of data elements.

This function is intended to provide enhanced flexibility in the processing of elements in a partition. For most of the simple use cases, consider using the MapFunction or FlatMapFunction.

The basic syntax for a MapPartitionFunction is as follows:

 DataSet<X> input = ...;

 DataSet<Y> result = input.mapPartition(new MyMapPartitionFunction());
 

Method Summary

Modifier and Type	Method and Description
void	mapPartition(Iterable<T> values, Collector<O> out) A user-implemented function that modifies or transforms an incoming object.

Method Detail

mapPartition

void mapPartition(Iterable<T> values,
                  Collector<O> out)
           throws Exception

A user-implemented function that modifies or transforms an incoming object.

Parameters:

values - All records for the mapper

out - The collector to hand results to.

Throws:

Exception - This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.