package extensions
acceptPartialFunctions extends the original DataSet with methods with unique names that delegate
to core higher-order functions (e.g. map
) so that we can work around the fact that overloaded
methods taking functions as parameters can't accept partial functions as well. This enables the
possibility to directly apply pattern matching to decompose inputs such as tuples, case classes
and collections.
The following is a small example that showcases how this extensions would work on a Flink data set:
object Main { import org.apache.flink.api.scala.extensions._ case class Point(x: Double, y: Double) def main(args: Array[String]): Unit = { val env = ExecutionEnvironment.getExecutionEnvironment val ds = env.fromElements(Point(1, 2), Point(3, 4), Point(5, 6)) ds.filterWith { case Point(x, _) => x > 1 }.reduceWith { case (Point(x1, y1), (Point(x2, y2))) => Point(x1 + y1, x2 + y2) }.mapWith { case Point(x, y) => (x, y) }.flatMapWith { case (x, y) => Seq('x' -> x, 'y' -> y) }.groupingBy { case (id, value) => id } } }
The extension consists of several implicit conversions over all the data set representations that
could gain from this feature. To use this set of extensions methods the user has to explicitly
opt-in by importing org.apache.flink.api.scala.extensions.acceptPartialFunctions
.
For more information and usage examples please consult the Apache Flink official documentation.
- Deprecated
All Flink Scala APIs are deprecated and will be removed in a future Flink major version. You can still build your application in Scala, but you should move to the Java version of either the DataStream and/or Table API.
- See also
- Alphabetic
- By Inheritance
- extensions
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
-
implicit
def
acceptPartialFunctions[L, R, O](ds: UnfinishedKeyPairOperation[L, R, O]): OnUnfinishedKeyPairOperation[L, R, O]
- Annotations
- @PublicEvolving()
-
implicit
def
acceptPartialFunctions[L, R, O](ds: HalfUnfinishedKeyPairOperation[L, R, O]): OnHalfUnfinishedKeyPairOperation[L, R, O]
- Annotations
- @PublicEvolving()
-
implicit
def
acceptPartialFunctions[L, R](ds: CoGroupDataSet[L, R]): OnCoGroupDataSet[L, R]
- Annotations
- @PublicEvolving()
-
implicit
def
acceptPartialFunctions[T](ds: GroupedDataSet[T]): OnGroupedDataSet[T]
- Annotations
- @PublicEvolving()
-
implicit
def
acceptPartialFunctions[L, R](ds: CrossDataSet[L, R]): OnCrossDataSet[L, R]
- Annotations
- @PublicEvolving()
-
implicit
def
acceptPartialFunctions[L, R](ds: JoinFunctionAssigner[L, R]): OnJoinFunctionAssigner[L, R]
- Annotations
- @PublicEvolving()
-
implicit
def
acceptPartialFunctions[T](ds: DataSet[T]): OnDataSet[T]
- Annotations
- @PublicEvolving()