Interface MapPartitionFunction<T,O>
-
- Type Parameters:
T
- Type of the input elements.O
- Type of the returned elements.
- All Superinterfaces:
Function
,Serializable
- All Known Implementing Classes:
AssignRangeIndex
,RangeBoundaryBuilder
,RichMapPartitionFunction
- Functional Interface:
- This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.
@Public @FunctionalInterface public interface MapPartitionFunction<T,O> extends Function, Serializable
Interface for "mapPartition" functions. A "mapPartition" function is called a single time per data partition receives an Iterable with data elements of that partition. It may return an arbitrary number of data elements.This function is intended to provide enhanced flexibility in the processing of elements in a partition. For most of the simple use cases, consider using the
MapFunction
orFlatMapFunction
.The basic syntax for a MapPartitionFunction is as follows:
DataSet<X> input = ...; DataSet<Y> result = input.mapPartition(new MyMapPartitionFunction());
-
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description void
mapPartition(Iterable<T> values, Collector<O> out)
A user-implemented function that modifies or transforms an incoming object.
-
-
-
Method Detail
-
mapPartition
void mapPartition(Iterable<T> values, Collector<O> out) throws Exception
A user-implemented function that modifies or transforms an incoming object.- Parameters:
values
- All records for the mapperout
- The collector to hand results to.- Throws:
Exception
- This method may throw exceptions. Throwing an exception will cause the operation to fail and may trigger recovery.
-
-