public class CrossDataSet<L,R> extends DataSet<scala.Tuple2<L,R>>
DataSet
that results from a cross
operation. The result of a default cross is a
tuple containing the two values from the two sides of the cartesian product. The result of the
cross can be changed by specifying a custom cross function using the apply
method or by
providing a RichCrossFunction
.
Example:
val left = ...
val right = ...
val crossResult = left.cross(right) {
(left, right) => new MyCrossResult(left, right)
}
Constructor and Description |
---|
CrossDataSet(CrossOperator<L,R,scala.Tuple2<L,R>> defaultCross,
DataSet<L> leftInput,
DataSet<R> rightInput) |
Modifier and Type | Method and Description |
---|---|
<O> DataSet<O> |
apply(CrossFunction<L,R,O> crosser,
TypeInformation<O> evidence$3,
scala.reflect.ClassTag<O> evidence$4)
Creates a new
DataSet by passing each pair of values to the given function. |
<O> DataSet<O> |
apply(scala.Function2<L,R,O> fun,
TypeInformation<O> evidence$1,
scala.reflect.ClassTag<O> evidence$2)
Creates a new
DataSet where the result for each pair of elements is the result
of the given function. |
static <L,R> CrossDataSet<L,R> |
createCrossOperator(DataSet<L> leftInput,
DataSet<R> rightInput,
CrossOperatorBase.CrossHint crossHint)
Creates a default cross operation with Tuple2 as result.
|
aggregate, aggregate, clean, coGroup, collect, combineGroup, combineGroup, count, cross, crossWithHuge, crossWithTiny, distinct, distinct, distinct, distinct, filter, filter, first, flatMap, flatMap, flatMap, fullOuterJoin, fullOuterJoin, getExecutionEnvironment, getParallelism, getType, groupBy, groupBy, groupBy, iterate, iterateDelta, iterateDelta, iterateDelta, iterateDelta, iterateWithTermination, javaSet, join, join, joinWithHuge, joinWithTiny, leftOuterJoin, leftOuterJoin, map, map, mapPartition, mapPartition, mapPartition, max, max, maxBy, min, min, minBy, name, output, partitionByHash, partitionByHash, partitionByHash, partitionByRange, partitionByRange, partitionByRange, partitionCustom, partitionCustom, partitionCustom, print, print, printOnTaskManager, printToErr, printToErr, rebalance, reduce, reduce, reduceGroup, reduceGroup, reduceGroup, registerAggregator, rightOuterJoin, rightOuterJoin, setParallelism, sortPartition, sortPartition, sortPartition, sum, sum, union, withBroadcastSet, withForwardedFields, withForwardedFieldsFirst, withForwardedFieldsSecond, withParameters, write, writeAsCsv, writeAsText
public static <L,R> CrossDataSet<L,R> createCrossOperator(DataSet<L> leftInput, DataSet<R> rightInput, CrossOperatorBase.CrossHint crossHint)
public <O> DataSet<O> apply(scala.Function2<L,R,O> fun, TypeInformation<O> evidence$1, scala.reflect.ClassTag<O> evidence$2)
DataSet
where the result for each pair of elements is the result
of the given function.public <O> DataSet<O> apply(CrossFunction<L,R,O> crosser, TypeInformation<O> evidence$3, scala.reflect.ClassTag<O> evidence$4)
DataSet
by passing each pair of values to the given function.
The function can output zero or more elements using the Collector
which will form the
result.
A RichCrossFunction
can be used to access the
broadcast variables and the RuntimeContext
.
Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.