Modifier and Type | Method and Description |
---|---|
DataSet<Row> |
HBaseTableSource.getDataSet(ExecutionEnvironment execEnv) |
Modifier and Type | Method and Description |
---|---|
<X> DataSet<X> |
DataSet.runOperation(CustomUnaryOperation<T,X> operation)
Runs a
CustomUnaryOperation on the data set. |
Modifier and Type | Method and Description |
---|---|
protected static void |
DataSet.checkSameExecutionContext(DataSet<?> set1,
DataSet<?> set2) |
protected static void |
DataSet.checkSameExecutionContext(DataSet<?> set1,
DataSet<?> set2) |
<R> CoGroupOperator.CoGroupOperatorSets<T,R> |
DataSet.coGroup(DataSet<R> other)
Initiates a CoGroup transformation.
A CoGroup transformation combines the elements of two DataSets into one DataSet. |
<R> CrossOperator.DefaultCross<T,R> |
DataSet.cross(DataSet<R> other)
Initiates a Cross transformation.
A Cross transformation combines the elements of two DataSets into one DataSet. |
<R> CrossOperator.DefaultCross<T,R> |
DataSet.crossWithHuge(DataSet<R> other)
Initiates a Cross transformation.
A Cross transformation combines the elements of two DataSets into one DataSet. |
<R> CrossOperator.DefaultCross<T,R> |
DataSet.crossWithTiny(DataSet<R> other)
Initiates a Cross transformation.
A Cross transformation combines the elements of two DataSets into one DataSet. |
<R> JoinOperatorSetsBase<T,R> |
DataSet.fullOuterJoin(DataSet<R> other)
Initiates a Full Outer Join transformation.
An Outer Join transformation joins two elements of two DataSets on key equality and provides multiple ways to combine
joining elements into one DataSet.Elements of both DataSets that do not have a matching element on the opposing side are joined with null and emitted to the
resulting DataSet. |
<R> JoinOperatorSetsBase<T,R> |
DataSet.fullOuterJoin(DataSet<R> other,
JoinOperatorBase.JoinHint strategy)
Initiates a Full Outer Join transformation.
An Outer Join transformation joins two elements of two DataSets on key equality and provides multiple ways to combine
joining elements into one DataSet.Elements of both DataSets that do not have a matching element on the opposing side are joined with null and emitted to the
resulting DataSet. |
<R> DeltaIteration<T,R> |
DataSet.iterateDelta(DataSet<R> workset,
int maxIterations,
int... keyPositions)
Initiates a delta iteration.
|
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.join(DataSet<R> other)
Initiates a Join transformation.
|
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.join(DataSet<R> other,
JoinOperatorBase.JoinHint strategy)
Initiates a Join transformation.
|
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.joinWithHuge(DataSet<R> other)
Initiates a Join transformation.
A Join transformation joins the elements of two DataSets on key equality and provides multiple ways to combine
joining elements into one DataSet.This method also gives the hint to the optimizer that the second DataSet to join is much larger than the first one. This method returns a JoinOperator.JoinOperatorSets on which one of the where methods
can be called to define the join key of the first joining (i.e., this) DataSet. |
<R> JoinOperator.JoinOperatorSets<T,R> |
DataSet.joinWithTiny(DataSet<R> other)
Initiates a Join transformation.
|
<R> JoinOperatorSetsBase<T,R> |
DataSet.leftOuterJoin(DataSet<R> other)
Initiates a Left Outer Join transformation.
An Outer Join transformation joins two elements of two DataSets on key equality and provides multiple ways to combine
joining elements into one DataSet.Elements of the left DataSet (i.e. |
<R> JoinOperatorSetsBase<T,R> |
DataSet.leftOuterJoin(DataSet<R> other,
JoinOperatorBase.JoinHint strategy)
Initiates a Left Outer Join transformation.
An Outer Join transformation joins two elements of two DataSets on key equality and provides multiple ways to combine
joining elements into one DataSet.Elements of the left DataSet (i.e. |
<R> JoinOperatorSetsBase<T,R> |
DataSet.rightOuterJoin(DataSet<R> other)
Initiates a Right Outer Join transformation.
An Outer Join transformation joins two elements of two DataSets on key equality and provides multiple ways to combine
joining elements into one DataSet.Elements of the right DataSet (i.e. |
<R> JoinOperatorSetsBase<T,R> |
DataSet.rightOuterJoin(DataSet<R> other,
JoinOperatorBase.JoinHint strategy)
Initiates a Right Outer Join transformation.
An Outer Join transformation joins two elements of two DataSets on key equality and provides multiple ways to combine
joining elements into one DataSet.Elements of the right DataSet (i.e. |
UnionOperator<T> |
DataSet.union(DataSet<T> other)
Creates a union of this DataSet with an other DataSet.
|
Modifier and Type | Class and Description |
---|---|
class |
AggregateOperator<IN>
This operator represents the application of a "aggregate" operation on a data set, and the
result data set produced by the function.
|
class |
BulkIterationResultSet<T> |
class |
CoGroupOperator<I1,I2,OUT>
A
DataSet that is the result of a CoGroup transformation. |
class |
CoGroupRawOperator<I1,I2,OUT>
A
DataSet that is the result of a CoGroup transformation. |
class |
CrossOperator<I1,I2,OUT>
A
DataSet that is the result of a Cross transformation. |
static class |
CrossOperator.DefaultCross<I1,I2>
|
static class |
CrossOperator.ProjectCross<I1,I2,OUT extends Tuple>
|
class |
DataSource<OUT>
An operation that creates a new data set (data source).
|
static class |
DeltaIteration.SolutionSetPlaceHolder<ST>
A
DataSet that acts as a placeholder for the solution set during the iteration. |
static class |
DeltaIteration.WorksetPlaceHolder<WT>
A
DataSet that acts as a placeholder for the workset during the iteration. |
class |
DeltaIterationResultSet<ST,WT> |
class |
DistinctOperator<T>
This operator represents the application of a "distinct" function on a data set, and the
result data set produced by the function.
|
class |
FilterOperator<T>
This operator represents the application of a "filter" function on a data set, and the
result data set produced by the function.
|
class |
FlatMapOperator<IN,OUT>
This operator represents the application of a "flatMap" function on a data set, and the
result data set produced by the function.
|
class |
GroupCombineOperator<IN,OUT>
This operator behaves like the GroupReduceOperator with Combine but only runs the Combine part which reduces all data
locally in their partitions.
|
class |
GroupReduceOperator<IN,OUT>
This operator represents the application of a "reduceGroup" function on a data set, and the
result data set produced by the function.
|
class |
IterativeDataSet<T>
The IterativeDataSet represents the start of an iteration.
|
class |
JoinOperator<I1,I2,OUT>
A
DataSet that is the result of a Join transformation. |
static class |
JoinOperator.DefaultJoin<I1,I2>
|
static class |
JoinOperator.EquiJoin<I1,I2,OUT>
A Join transformation that applies a
JoinFunction on each pair of joining elements.It also represents the DataSet that is the result of a Join transformation. |
static class |
JoinOperator.ProjectJoin<I1,I2,OUT extends Tuple>
|
class |
MapOperator<IN,OUT>
This operator represents the application of a "map" function on a data set, and the
result data set produced by the function.
|
class |
MapPartitionOperator<IN,OUT>
This operator represents the application of a "mapPartition" function on a data set, and the
result data set produced by the function.
|
class |
NoOpOperator<IN>
This operator will be ignored during translation.
|
class |
Operator<OUT,O extends Operator<OUT,O>>
Base class of all operators in the Java API.
|
class |
PartitionOperator<T>
This operator represents a partitioning.
|
class |
ProjectOperator<IN,OUT extends Tuple>
This operator represents the application of a projection operation on a data set, and the
result data set produced by the function.
|
class |
ReduceOperator<IN>
This operator represents the application of a "reduce" function on a data set, and the
result data set produced by the function.
|
class |
SingleInputOperator<IN,OUT,O extends SingleInputOperator<IN,OUT,O>>
Base class for operations that operates on a single input data set.
|
class |
SingleInputUdfOperator<IN,OUT,O extends SingleInputUdfOperator<IN,OUT,O>>
The SingleInputUdfOperator is the base class of all unary operators that execute
user-defined functions (UDFs).
|
class |
SortPartitionOperator<T>
This operator represents a DataSet with locally sorted partitions.
|
class |
TwoInputOperator<IN1,IN2,OUT,O extends TwoInputOperator<IN1,IN2,OUT,O>>
Base class for operations that operates on two input data sets.
|
class |
TwoInputUdfOperator<IN1,IN2,OUT,O extends TwoInputUdfOperator<IN1,IN2,OUT,O>>
The TwoInputUdfOperator is the base class of all binary operators that execute
user-defined functions (UDFs).
|
class |
UnionOperator<T>
Java API operator for union of two data sets
|
Modifier and Type | Field and Description |
---|---|
protected DataSet<T> |
Grouping.inputDataSet |
Modifier and Type | Method and Description |
---|---|
DataSet<ST> |
DeltaIteration.closeWith(DataSet<ST> solutionSetDelta,
DataSet<WT> newWorkset)
Closes the delta iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult)
Closes the iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult,
DataSet<?> terminationCriterion)
Closes the iteration and specifies a termination criterion.
|
DataSet<OUT> |
CustomUnaryOperation.createResult() |
DataSet<T> |
DataSink.getDataSet() |
DataSet<ST> |
DeltaIteration.getInitialSolutionSet()
Gets the initial solution set.
|
DataSet<WT> |
DeltaIteration.getInitialWorkset()
Gets the initial workset.
|
DataSet<IN> |
SingleInputOperator.getInput()
Gets the data set that this operation uses as its input.
|
DataSet<IN> |
NoOpOperator.getInput() |
DataSet<IN1> |
TwoInputOperator.getInput1()
Gets the data set that this operation uses as its first input.
|
DataSet<IN2> |
TwoInputOperator.getInput2()
Gets the data set that this operation uses as its second input.
|
DataSet<T> |
Grouping.getInputDataSet()
Returns the input DataSet of a grouping operation, that is the one before the grouping.
|
DataSet<T> |
BulkIterationResultSet.getNextPartialSolution() |
DataSet<ST> |
DeltaIterationResultSet.getNextSolutionSet() |
DataSet<WT> |
DeltaIterationResultSet.getNextWorkset() |
DataSet<?> |
BulkIterationResultSet.getTerminationCriterion() |
Modifier and Type | Method and Description |
---|---|
Map<String,DataSet<?>> |
UdfOperator.getBroadcastSets()
Gets the broadcast sets (name and data set) that have been added to context of the UDF.
|
Map<String,DataSet<?>> |
TwoInputUdfOperator.getBroadcastSets() |
Map<String,DataSet<?>> |
SingleInputUdfOperator.getBroadcastSets() |
Modifier and Type | Method and Description |
---|---|
DataSet<ST> |
DeltaIteration.closeWith(DataSet<ST> solutionSetDelta,
DataSet<WT> newWorkset)
Closes the delta iteration.
|
DataSet<ST> |
DeltaIteration.closeWith(DataSet<ST> solutionSetDelta,
DataSet<WT> newWorkset)
Closes the delta iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult)
Closes the iteration.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult,
DataSet<?> terminationCriterion)
Closes the iteration and specifies a termination criterion.
|
DataSet<T> |
IterativeDataSet.closeWith(DataSet<T> iterationResult,
DataSet<?> terminationCriterion)
Closes the iteration and specifies a termination criterion.
|
void |
NoOpOperator.setInput(DataSet<IN> input) |
void |
CustomUnaryOperation.setInput(DataSet<IN> inputData) |
O |
UdfOperator.withBroadcastSet(DataSet<?> data,
String name)
Adds a certain data set as a broadcast set to this operator.
|
O |
TwoInputUdfOperator.withBroadcastSet(DataSet<?> data,
String name) |
O |
SingleInputUdfOperator.withBroadcastSet(DataSet<?> data,
String name) |
Constructor and Description |
---|
AggregateOperator(DataSet<IN> input,
Aggregations function,
int field,
String aggregateLocationName)
Non grouped aggregation
|
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderFirst,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderSecond,
Partitioner<?> customPartitioner,
String defaultName) |
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderFirst,
List<org.apache.commons.lang3.tuple.Pair<Integer,Order>> groupSortKeyOrderSecond,
Partitioner<?> customPartitioner,
String defaultName) |
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
Partitioner<?> customPartitioner,
String defaultName) |
CoGroupOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
Partitioner<?> customPartitioner,
String defaultName) |
CoGroupOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
CoGroupOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
CoGroupRawOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
String defaultName) |
CoGroupRawOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
CoGroupFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
String defaultName) |
CrossOperator(DataSet<I1> input1,
DataSet<I2> input2,
CrossFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
CrossOperatorBase.CrossHint hint,
String defaultName) |
CrossOperator(DataSet<I1> input1,
DataSet<I2> input2,
CrossFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
CrossOperatorBase.CrossHint hint,
String defaultName) |
CrossProjection(DataSet<I1> ds1,
DataSet<I2> ds2,
int[] firstFieldIndexes,
int[] secondFieldIndexes,
CrossOperatorBase.CrossHint hint) |
CrossProjection(DataSet<I1> ds1,
DataSet<I2> ds2,
int[] firstFieldIndexes,
int[] secondFieldIndexes,
CrossOperatorBase.CrossHint hint) |
DataSink(DataSet<T> data,
OutputFormat<T> format,
TypeInformation<T> type) |
DefaultCross(DataSet<I1> input1,
DataSet<I2> input2,
CrossOperatorBase.CrossHint hint,
String defaultName) |
DefaultCross(DataSet<I1> input1,
DataSet<I2> input2,
CrossOperatorBase.CrossHint hint,
String defaultName) |
DefaultJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
DefaultJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
DeltaIteration(ExecutionEnvironment context,
TypeInformation<ST> type,
DataSet<ST> solutionSet,
DataSet<WT> workset,
Keys<ST> keys,
int maxIterations) |
DeltaIteration(ExecutionEnvironment context,
TypeInformation<ST> type,
DataSet<ST> solutionSet,
DataSet<WT> workset,
Keys<ST> keys,
int maxIterations) |
DistinctOperator(DataSet<T> input,
Keys<T> keys,
String distinctLocationName) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> generatedFunction,
JoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> generatedFunction,
JoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> generatedFunction,
JoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> generatedFunction,
JoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
EquiJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
FlatJoinFunction<I1,I2,OUT> function,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
String joinLocationName,
JoinType type) |
FilterOperator(DataSet<T> input,
FilterFunction<T> function,
String defaultName) |
FlatMapOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
FlatMapFunction<IN,OUT> function,
String defaultName) |
GroupCombineOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
GroupCombineFunction<IN,OUT> function,
String defaultName)
Constructor for a non-grouped reduce (all reduce).
|
Grouping(DataSet<T> set,
Keys<T> keys) |
GroupReduceOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
GroupReduceFunction<IN,OUT> function,
String defaultName)
Constructor for a non-grouped reduce (all reduce).
|
IterativeDataSet(ExecutionEnvironment context,
TypeInformation<T> type,
DataSet<T> input,
int maxIterations) |
JoinOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
JoinType type) |
JoinOperator(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
TypeInformation<OUT> returnType,
JoinOperatorBase.JoinHint hint,
JoinType type) |
JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2) |
JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint) |
JoinOperatorSets(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint) |
JoinProjection(DataSet<I1> ds1,
DataSet<I2> ds2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] firstFieldIndexes,
int[] secondFieldIndexes) |
JoinProjection(DataSet<I1> ds1,
DataSet<I2> ds2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] firstFieldIndexes,
int[] secondFieldIndexes) |
MapOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
MapFunction<IN,OUT> function,
String defaultName) |
MapPartitionOperator(DataSet<IN> input,
TypeInformation<OUT> resultType,
MapPartitionFunction<IN,OUT> function,
String defaultName) |
NoOpOperator(DataSet<IN> input,
TypeInformation<IN> resultType) |
PartitionOperator(DataSet<T> input,
Keys<T> pKeys,
Partitioner<?> customPartitioner,
String partitionLocationName) |
PartitionOperator(DataSet<T> input,
Keys<T> pKeys,
Partitioner<P> customPartitioner,
TypeInformation<P> partitionerTypeInfo,
String partitionLocationName) |
PartitionOperator(DataSet<T> input,
PartitionOperatorBase.PartitionMethod pMethod,
Keys<T> pKeys,
DataDistribution distribution,
String partitionLocationName) |
PartitionOperator(DataSet<T> input,
PartitionOperatorBase.PartitionMethod pMethod,
Keys<T> pKeys,
String partitionLocationName) |
PartitionOperator(DataSet<T> input,
PartitionOperatorBase.PartitionMethod pMethod,
String partitionLocationName) |
ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
CrossOperator.CrossProjection<I1,I2> crossProjection,
CrossOperatorBase.CrossHint hint) |
ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
CrossOperator.CrossProjection<I1,I2> crossProjection,
CrossOperatorBase.CrossHint hint) |
ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
CrossOperatorBase.CrossHint hint) |
ProjectCross(DataSet<I1> input1,
DataSet<I2> input2,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
CrossOperatorBase.CrossHint hint) |
Projection(DataSet<T> ds,
int[] fieldIndexes) |
ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType) |
ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType) |
ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
JoinOperator.JoinProjection<I1,I2> joinProj) |
ProjectJoin(DataSet<I1> input1,
DataSet<I2> input2,
Keys<I1> keys1,
Keys<I2> keys2,
JoinOperatorBase.JoinHint hint,
int[] fields,
boolean[] isFromFirst,
TupleTypeInfo<OUT> returnType,
JoinOperator.JoinProjection<I1,I2> joinProj) |
ProjectOperator(DataSet<IN> input,
int[] fields,
TupleTypeInfo<OUT> returnType) |
ReduceOperator(DataSet<IN> input,
ReduceFunction<IN> function,
String defaultName)
This is the case for a reduce-all case (in contrast to the reduce-per-group case).
|
SingleInputOperator(DataSet<IN> input,
TypeInformation<OUT> resultType) |
SingleInputUdfOperator(DataSet<IN> input,
TypeInformation<OUT> resultType)
Creates a new operators with the given data set as input.
|
SortedGrouping(DataSet<T> set,
Keys<T> keys,
int field,
Order order) |
SortedGrouping(DataSet<T> set,
Keys<T> keys,
Keys.SelectorFunctionKeys<T,K> keySelector,
Order order) |
SortedGrouping(DataSet<T> set,
Keys<T> keys,
String field,
Order order) |
SortPartitionOperator(DataSet<T> dataSet,
int sortField,
Order sortOrder,
String sortLocationName) |
SortPartitionOperator(DataSet<T> dataSet,
Keys.SelectorFunctionKeys<T,K> sortKey,
Order sortOrder,
String sortLocationName) |
SortPartitionOperator(DataSet<T> dataSet,
String sortField,
Order sortOrder,
String sortLocationName) |
TwoInputOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType) |
TwoInputOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType) |
TwoInputUdfOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType)
Creates a new operators with the two given data sets as inputs.
|
TwoInputUdfOperator(DataSet<IN1> input1,
DataSet<IN2> input2,
TypeInformation<OUT> resultType)
Creates a new operators with the two given data sets as inputs.
|
UnionOperator(DataSet<T> input1,
DataSet<T> input2,
String unionLocationName)
Create an operator that produces the union of the two given data sets.
|
UnionOperator(DataSet<T> input1,
DataSet<T> input2,
String unionLocationName)
Create an operator that produces the union of the two given data sets.
|
UnsortedGrouping(DataSet<T> set,
Keys<T> keys) |
Modifier and Type | Field and Description |
---|---|
protected DataSet<I1> |
JoinOperatorSetsBase.input1 |
protected DataSet<I2> |
JoinOperatorSetsBase.input2 |
Constructor and Description |
---|
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2) |
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2) |
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint) |
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint) |
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint,
JoinType type) |
JoinOperatorSetsBase(DataSet<I1> input1,
DataSet<I2> input2,
JoinOperatorBase.JoinHint hint,
JoinType type) |
Modifier and Type | Method and Description |
---|---|
static <T> DataSet<Tuple2<Integer,Long>> |
DataSetUtils.countElementsPerPartition(DataSet<T> input)
Method that goes over all the elements in each partition in order to retrieve
the total number of elements.
|
static <T> DataSet<T> |
DataSetUtils.sampleWithSize(DataSet<T> input,
boolean withReplacement,
int numSamples)
Generate a sample of DataSet which contains fixed size elements.
|
static <T> DataSet<T> |
DataSetUtils.sampleWithSize(DataSet<T> input,
boolean withReplacement,
int numSamples,
long seed)
Generate a sample of DataSet which contains fixed size elements.
|
static <T> DataSet<Tuple2<Long,T>> |
DataSetUtils.zipWithIndex(DataSet<T> input)
Method that assigns a unique
Long value to all elements in the input data set. |
static <T> DataSet<Tuple2<Long,T>> |
DataSetUtils.zipWithUniqueId(DataSet<T> input)
Method that assigns a unique
Long value to all elements in the input data set in the following way:
a map function is applied to the input data set
each map task holds a counter c which is increased for each record
c is shifted by n bits where n = log2(number of parallel tasks)
to create a unique ID among all tasks, the task id is added to the counter
for each record, the resulting counter is collected
|
Modifier and Type | Method and Description |
---|---|
static <T> Utils.ChecksumHashCode |
DataSetUtils.checksumHashCode(DataSet<T> input)
Deprecated.
replaced with
org.apache.flink.graph.asm.dataset.ChecksumHashCode in Gelly |
static <T> DataSet<Tuple2<Integer,Long>> |
DataSetUtils.countElementsPerPartition(DataSet<T> input)
Method that goes over all the elements in each partition in order to retrieve
the total number of elements.
|
static <T> PartitionOperator<T> |
DataSetUtils.partitionByRange(DataSet<T> input,
DataDistribution distribution,
int... fields)
Range-partitions a DataSet on the specified tuple field positions.
|
static <T,K extends Comparable<K>> |
DataSetUtils.partitionByRange(DataSet<T> input,
DataDistribution distribution,
KeySelector<T,K> keyExtractor)
Range-partitions a DataSet using the specified key selector function.
|
static <T> PartitionOperator<T> |
DataSetUtils.partitionByRange(DataSet<T> input,
DataDistribution distribution,
String... fields)
Range-partitions a DataSet on the specified fields.
|
static <T> MapPartitionOperator<T,T> |
DataSetUtils.sample(DataSet<T> input,
boolean withReplacement,
double fraction)
Generate a sample of DataSet by the probability fraction of each element.
|
static <T> MapPartitionOperator<T,T> |
DataSetUtils.sample(DataSet<T> input,
boolean withReplacement,
double fraction,
long seed)
Generate a sample of DataSet by the probability fraction of each element.
|
static <T> DataSet<T> |
DataSetUtils.sampleWithSize(DataSet<T> input,
boolean withReplacement,
int numSamples)
Generate a sample of DataSet which contains fixed size elements.
|
static <T> DataSet<T> |
DataSetUtils.sampleWithSize(DataSet<T> input,
boolean withReplacement,
int numSamples,
long seed)
Generate a sample of DataSet which contains fixed size elements.
|
static <R extends Tuple,T extends Tuple> |
DataSetUtils.summarize(DataSet<T> input)
Summarize a DataSet of Tuples by collecting single pass statistics for all columns
Example usage:
|
static <T> DataSet<Tuple2<Long,T>> |
DataSetUtils.zipWithIndex(DataSet<T> input)
Method that assigns a unique
Long value to all elements in the input data set. |
static <T> DataSet<Tuple2<Long,T>> |
DataSetUtils.zipWithUniqueId(DataSet<T> input)
Method that assigns a unique
Long value to all elements in the input data set in the following way:
a map function is applied to the input data set
each map task holds a counter c which is increased for each record
c is shifted by n bits where n = log2(number of parallel tasks)
to create a unique ID among all tasks, the task id is added to the counter
for each record, the resulting counter is collected
|
Modifier and Type | Method and Description |
---|---|
static DataSet<T> |
CrossDataSet.javaSet() |
DataSet<T> |
DataSet.javaSet()
Returns the underlying Java DataSet.
|
Constructor and Description |
---|
DataSet(DataSet<T> set,
scala.reflect.ClassTag<T> evidence$1) |
Modifier and Type | Class and Description |
---|---|
class |
ScalaAggregateOperator<IN>
This operator represents the application of a "aggregate" operation on a data set, and the
result data set produced by the function.
|
Constructor and Description |
---|
ScalaAggregateOperator(DataSet<IN> input,
Aggregations function,
int field)
Non grouped aggregation
|
Modifier and Type | Method and Description |
---|---|
static DataSet<KMeans.Centroid> |
KMeansData.getDefaultCentroidDataSet(ExecutionEnvironment env) |
static DataSet<KMeans.Point> |
KMeansData.getDefaultPointDataSet(ExecutionEnvironment env) |
Modifier and Type | Method and Description |
---|---|
static DataSet<Tuple2<Long,Long>> |
PageRankData.getDefaultEdgeDataSet(ExecutionEnvironment env) |
static DataSet<EnumTrianglesDataTypes.Edge> |
EnumTrianglesData.getDefaultEdgeDataSet(ExecutionEnvironment env) |
static DataSet<Tuple2<Long,Long>> |
ConnectedComponentsData.getDefaultEdgeDataSet(ExecutionEnvironment env) |
static DataSet<Long> |
PageRankData.getDefaultPagesDataSet(ExecutionEnvironment env) |
static DataSet<Long> |
ConnectedComponentsData.getDefaultVertexDataSet(ExecutionEnvironment env) |
Modifier and Type | Method and Description |
---|---|
static DataSet<LinearRegression.Data> |
LinearRegressionData.getDefaultDataDataSet(ExecutionEnvironment env) |
static DataSet<LinearRegression.Params> |
LinearRegressionData.getDefaultParamsDataSet(ExecutionEnvironment env) |
Modifier and Type | Method and Description |
---|---|
static DataSet<Tuple2<String,String>> |
WebLogData.getDocumentDataSet(ExecutionEnvironment env) |
static DataSet<Tuple3<Integer,String,Integer>> |
WebLogData.getRankDataSet(ExecutionEnvironment env) |
static DataSet<Tuple2<String,String>> |
WebLogData.getVisitDataSet(ExecutionEnvironment env) |
Modifier and Type | Method and Description |
---|---|
static DataSet<String> |
WordCountData.getDefaultTextLineDataSet(ExecutionEnvironment env) |
Modifier and Type | Method and Description |
---|---|
DataSet<Tuple2<K,LongValue>> |
Graph.getDegrees()
Return the degree of all vertices in the graph
|
DataSet<Tuple2<K,K>> |
Graph.getEdgeIds() |
DataSet<Edge<K,EV>> |
Graph.getEdges() |
DataSet<Tuple3<K,K,EV>> |
Graph.getEdgesAsTuple3() |
DataSet<Triplet<K,VV,EV>> |
Graph.getTriplets()
This method allows access to the graph's edge values along with its source and target vertex values.
|
DataSet<K> |
Graph.getVertexIds() |
DataSet<Vertex<K,VV>> |
Graph.getVertices() |
DataSet<Tuple2<K,VV>> |
Graph.getVerticesAsTuple2() |
<T> DataSet<T> |
Graph.groupReduceOnEdges(EdgesFunction<K,EV,T> edgesFunction,
EdgeDirection direction)
Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex.
|
<T> DataSet<T> |
Graph.groupReduceOnEdges(EdgesFunction<K,EV,T> edgesFunction,
EdgeDirection direction,
TypeInformation<T> typeInfo)
Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex.
|
<T> DataSet<T> |
Graph.groupReduceOnEdges(EdgesFunctionWithVertexValue<K,VV,EV,T> edgesFunction,
EdgeDirection direction)
Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex.
|
<T> DataSet<T> |
Graph.groupReduceOnEdges(EdgesFunctionWithVertexValue<K,VV,EV,T> edgesFunction,
EdgeDirection direction,
TypeInformation<T> typeInfo)
Groups by vertex and computes a GroupReduce transformation over the edge values of each vertex.
|
<T> DataSet<T> |
Graph.groupReduceOnNeighbors(NeighborsFunction<K,VV,EV,T> neighborsFunction,
EdgeDirection direction)
Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices)
of each vertex.
|
<T> DataSet<T> |
Graph.groupReduceOnNeighbors(NeighborsFunction<K,VV,EV,T> neighborsFunction,
EdgeDirection direction,
TypeInformation<T> typeInfo)
Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices)
of each vertex.
|
<T> DataSet<T> |
Graph.groupReduceOnNeighbors(NeighborsFunctionWithVertexValue<K,VV,EV,T> neighborsFunction,
EdgeDirection direction)
Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices)
of each vertex.
|
<T> DataSet<T> |
Graph.groupReduceOnNeighbors(NeighborsFunctionWithVertexValue<K,VV,EV,T> neighborsFunction,
EdgeDirection direction,
TypeInformation<T> typeInfo)
Groups by vertex and computes a GroupReduce transformation over the neighbors (both edges and vertices)
of each vertex.
|
DataSet<Tuple2<K,LongValue>> |
Graph.inDegrees()
Return the in-degree of all vertices in the graph
|
DataSet<Tuple2<K,LongValue>> |
Graph.outDegrees()
Return the out-degree of all vertices in the graph
|
DataSet<Tuple2<K,EV>> |
Graph.reduceOnEdges(ReduceEdgesFunction<EV> reduceEdgesFunction,
EdgeDirection direction)
Compute a reduce transformation over the edge values of each vertex.
|
DataSet<Tuple2<K,VV>> |
Graph.reduceOnNeighbors(ReduceNeighborsFunction<VV> reduceNeighborsFunction,
EdgeDirection direction)
Compute a reduce transformation over the neighbors' vertex values of each vertex.
|
Modifier and Type | Method and Description |
---|---|
static <K,EV> Graph<K,NullValue,EV> |
Graph.fromDataSet(DataSet<Edge<K,EV>> edges,
ExecutionEnvironment context)
Creates a graph from a DataSet of edges.
|
static <K,VV,EV> Graph<K,VV,EV> |
Graph.fromDataSet(DataSet<Edge<K,EV>> edges,
MapFunction<K,VV> vertexValueInitializer,
ExecutionEnvironment context)
Creates a graph from a DataSet of edges.
|
static <K,VV,EV> Graph<K,VV,EV> |
Graph.fromDataSet(DataSet<Vertex<K,VV>> vertices,
DataSet<Edge<K,EV>> edges,
ExecutionEnvironment context)
Creates a graph from a DataSet of vertices and a DataSet of edges.
|
static <K,VV,EV> Graph<K,VV,EV> |
Graph.fromDataSet(DataSet<Vertex<K,VV>> vertices,
DataSet<Edge<K,EV>> edges,
ExecutionEnvironment context)
Creates a graph from a DataSet of vertices and a DataSet of edges.
|
static <K> Graph<K,NullValue,NullValue> |
Graph.fromTuple2DataSet(DataSet<Tuple2<K,K>> edges,
ExecutionEnvironment context)
Creates a graph from a DataSet of Tuple2 objects for edges.
|
static <K,VV> Graph<K,VV,NullValue> |
Graph.fromTuple2DataSet(DataSet<Tuple2<K,K>> edges,
MapFunction<K,VV> vertexValueInitializer,
ExecutionEnvironment context)
Creates a graph from a DataSet of Tuple2 objects for edges.
|
static <K,VV,EV> Graph<K,VV,EV> |
Graph.fromTupleDataSet(DataSet<Tuple2<K,VV>> vertices,
DataSet<Tuple3<K,K,EV>> edges,
ExecutionEnvironment context)
Creates a graph from a DataSet of Tuple2 objects for vertices and
Tuple3 objects for edges.
|
static <K,VV,EV> Graph<K,VV,EV> |
Graph.fromTupleDataSet(DataSet<Tuple2<K,VV>> vertices,
DataSet<Tuple3<K,K,EV>> edges,
ExecutionEnvironment context)
Creates a graph from a DataSet of Tuple2 objects for vertices and
Tuple3 objects for edges.
|
static <K,EV> Graph<K,NullValue,EV> |
Graph.fromTupleDataSet(DataSet<Tuple3<K,K,EV>> edges,
ExecutionEnvironment context)
Creates a graph from a DataSet of Tuple3 objects for edges.
|
static <K,VV,EV> Graph<K,VV,EV> |
Graph.fromTupleDataSet(DataSet<Tuple3<K,K,EV>> edges,
MapFunction<K,VV> vertexValueInitializer,
ExecutionEnvironment context)
Creates a graph from a DataSet of Tuple3 objects for edges.
|
<T> Graph<K,VV,EV> |
Graph.joinWithEdges(DataSet<Tuple3<K,K,T>> inputDataSet,
EdgeJoinFunction<EV,T> edgeJoinFunction)
Joins the edge DataSet with an input DataSet on the composite key of both
source and target IDs and applies a user-defined transformation on the values
of the matched records.
|
<T> Graph<K,VV,EV> |
Graph.joinWithEdgesOnSource(DataSet<Tuple2<K,T>> inputDataSet,
EdgeJoinFunction<EV,T> edgeJoinFunction)
Joins the edge DataSet with an input Tuple2 DataSet and applies a user-defined transformation
on the values of the matched records.
|
<T> Graph<K,VV,EV> |
Graph.joinWithEdgesOnTarget(DataSet<Tuple2<K,T>> inputDataSet,
EdgeJoinFunction<EV,T> edgeJoinFunction)
Joins the edge DataSet with an input Tuple2 DataSet and applies a user-defined transformation
on the values of the matched records.
|
<T> Graph<K,VV,EV> |
Graph.joinWithVertices(DataSet<Tuple2<K,T>> inputDataSet,
VertexJoinFunction<VV,T> vertexJoinFunction)
Joins the vertex DataSet of this graph with an input Tuple2 DataSet and applies
a user-defined transformation on the values of the matched records.
|
Constructor and Description |
---|
Graph(DataSet<Vertex<K,VV>> vertices,
DataSet<Edge<K,EV>> edges,
ExecutionEnvironment context)
Creates a graph from two DataSets: vertices and edges
|
Graph(DataSet<Vertex<K,VV>> vertices,
DataSet<Edge<K,EV>> edges,
ExecutionEnvironment context)
Creates a graph from two DataSets: vertices and edges
|
Modifier and Type | Method and Description |
---|---|
DataSetAnalytic<T,R> |
DataSetAnalytic.run(DataSet<T> input)
All
DataSetAnalytic processing must be terminated by an
OutputFormat and obtained via accumulators rather than
returned by a DataSet . |
Count<T> |
Count.run(DataSet<T> input) |
Collect<T> |
Collect.run(DataSet<T> input) |
ChecksumHashCode<T> |
ChecksumHashCode.run(DataSet<T> input) |
AbstractDataSetAnalytic<T,R> |
AbstractDataSetAnalytic.run(DataSet<T> input) |
Modifier and Type | Method and Description |
---|---|
DataSet<Vertex<K,LongValue>> |
VertexOutDegree.runInternal(Graph<K,VV,EV> input) |
DataSet<Vertex<K,LongValue>> |
VertexInDegree.runInternal(Graph<K,VV,EV> input) |
DataSet<Vertex<K,VertexDegrees.Degrees>> |
VertexDegrees.runInternal(Graph<K,VV,EV> input) |
DataSet<Edge<K,Tuple2<EV,VertexDegrees.Degrees>>> |
EdgeTargetDegrees.runInternal(Graph<K,VV,EV> input) |
DataSet<Edge<K,Tuple2<EV,VertexDegrees.Degrees>>> |
EdgeSourceDegrees.runInternal(Graph<K,VV,EV> input) |
DataSet<Edge<K,Tuple3<EV,VertexDegrees.Degrees,VertexDegrees.Degrees>>> |
EdgeDegreesPair.runInternal(Graph<K,VV,EV> input) |
Modifier and Type | Method and Description |
---|---|
DataSet<Vertex<K,LongValue>> |
VertexDegree.runInternal(Graph<K,VV,EV> input) |
DataSet<Edge<K,Tuple2<EV,LongValue>>> |
EdgeTargetDegree.runInternal(Graph<K,VV,EV> input) |
DataSet<Edge<K,Tuple2<EV,LongValue>>> |
EdgeSourceDegree.runInternal(Graph<K,VV,EV> input) |
DataSet<Edge<K,Tuple3<EV,LongValue,LongValue>>> |
EdgeDegreePair.runInternal(Graph<K,VV,EV> input) |
Modifier and Type | Method and Description |
---|---|
static <OLD,NEW,EV> |
Translate.translateEdgeIds(DataSet<Edge<OLD,EV>> edges,
TranslateFunction<OLD,NEW> translator)
Translate
Edge IDs using the given TranslateFunction . |
static <OLD,NEW,EV> |
Translate.translateEdgeIds(DataSet<Edge<OLD,EV>> edges,
TranslateFunction<OLD,NEW> translator,
int parallelism)
Translate
Edge IDs using the given TranslateFunction . |
static <K,OLD,NEW> |
Translate.translateEdgeValues(DataSet<Edge<K,OLD>> edges,
TranslateFunction<OLD,NEW> translator)
Translate
Edge values using the given TranslateFunction . |
static <K,OLD,NEW> |
Translate.translateEdgeValues(DataSet<Edge<K,OLD>> edges,
TranslateFunction<OLD,NEW> translator,
int parallelism)
Translate
Edge values using the given TranslateFunction . |
static <OLD,NEW,VV> |
Translate.translateVertexIds(DataSet<Vertex<OLD,VV>> vertices,
TranslateFunction<OLD,NEW> translator)
Translate
Vertex IDs using the given TranslateFunction . |
static <OLD,NEW,VV> |
Translate.translateVertexIds(DataSet<Vertex<OLD,VV>> vertices,
TranslateFunction<OLD,NEW> translator,
int parallelism)
Translate
Vertex IDs using the given TranslateFunction . |
static <K,OLD,NEW> |
Translate.translateVertexValues(DataSet<Vertex<K,OLD>> vertices,
TranslateFunction<OLD,NEW> translator)
Translate
Vertex values using the given TranslateFunction . |
static <K,OLD,NEW> |
Translate.translateVertexValues(DataSet<Vertex<K,OLD>> vertices,
TranslateFunction<OLD,NEW> translator,
int parallelism)
Translate
Vertex values using the given TranslateFunction . |
Modifier and Type | Method and Description |
---|---|
static <OLD,NEW,EV> |
Translate.translateEdgeIds(DataSet<Edge<OLD,EV>> edges,
TranslateFunction<OLD,NEW> translator)
Translate
Edge IDs using the given TranslateFunction . |
static <OLD,NEW,EV> |
Translate.translateEdgeIds(DataSet<Edge<OLD,EV>> edges,
TranslateFunction<OLD,NEW> translator,
int parallelism)
Translate
Edge IDs using the given TranslateFunction . |
static <K,OLD,NEW> |
Translate.translateEdgeValues(DataSet<Edge<K,OLD>> edges,
TranslateFunction<OLD,NEW> translator)
Translate
Edge values using the given TranslateFunction . |
static <K,OLD,NEW> |
Translate.translateEdgeValues(DataSet<Edge<K,OLD>> edges,
TranslateFunction<OLD,NEW> translator,
int parallelism)
Translate
Edge values using the given TranslateFunction . |
static <OLD,NEW,VV> |
Translate.translateVertexIds(DataSet<Vertex<OLD,VV>> vertices,
TranslateFunction<OLD,NEW> translator)
Translate
Vertex IDs using the given TranslateFunction . |
static <OLD,NEW,VV> |
Translate.translateVertexIds(DataSet<Vertex<OLD,VV>> vertices,
TranslateFunction<OLD,NEW> translator,
int parallelism)
Translate
Vertex IDs using the given TranslateFunction . |
static <K,OLD,NEW> |
Translate.translateVertexValues(DataSet<Vertex<K,OLD>> vertices,
TranslateFunction<OLD,NEW> translator)
Translate
Vertex values using the given TranslateFunction . |
static <K,OLD,NEW> |
Translate.translateVertexValues(DataSet<Vertex<K,OLD>> vertices,
TranslateFunction<OLD,NEW> translator,
int parallelism)
Translate
Vertex values using the given TranslateFunction . |
Modifier and Type | Method and Description |
---|---|
DataSet<Vertex<KB,VVB>> |
BipartiteGraph.getBottomVertices()
Get dataset with bottom vertices.
|
DataSet<BipartiteEdge<KT,KB,EV>> |
BipartiteGraph.getEdges()
Get dataset with graph edges.
|
DataSet<Vertex<KT,VVT>> |
BipartiteGraph.getTopVertices()
Get dataset with top vertices.
|
Modifier and Type | Method and Description |
---|---|
static <KT,KB,VVT,VVB,EV> |
BipartiteGraph.fromDataSet(DataSet<Vertex<KT,VVT>> topVertices,
DataSet<Vertex<KB,VVB>> bottomVertices,
DataSet<BipartiteEdge<KT,KB,EV>> edges,
ExecutionEnvironment context)
Create bipartite graph from datasets.
|
static <KT,KB,VVT,VVB,EV> |
BipartiteGraph.fromDataSet(DataSet<Vertex<KT,VVT>> topVertices,
DataSet<Vertex<KB,VVB>> bottomVertices,
DataSet<BipartiteEdge<KT,KB,EV>> edges,
ExecutionEnvironment context)
Create bipartite graph from datasets.
|
static <KT,KB,VVT,VVB,EV> |
BipartiteGraph.fromDataSet(DataSet<Vertex<KT,VVT>> topVertices,
DataSet<Vertex<KB,VVB>> bottomVertices,
DataSet<BipartiteEdge<KT,KB,EV>> edges,
ExecutionEnvironment context)
Create bipartite graph from datasets.
|
Modifier and Type | Method and Description |
---|---|
protected DataSet<R> |
SimpleDriver.getResult() |
protected DataSet<PrintableResult> |
TriangleListing.simplePlan(Graph<K,VV,EV> graph) |
protected abstract DataSet<R> |
SimpleDriver.simplePlan(Graph<K,VV,EV> graph)
Plan the algorithm and return the result
DataSet . |
protected DataSet<PageRank.Result<K>> |
PageRank.simplePlan(Graph<K,VV,EV> graph) |
protected DataSet<JaccardIndex.Result<K>> |
JaccardIndex.simplePlan(Graph<K,VV,EV> graph) |
protected DataSet<HITS.Result<K>> |
HITS.simplePlan(Graph<K,VV,EV> graph) |
protected DataSet<PrintableResult> |
ClusteringCoefficient.simplePlan(Graph<K,VV,EV> graph) |
protected DataSet<AdamicAdar.Result<K>> |
AdamicAdar.simplePlan(Graph<K,VV,EV> graph) |
Modifier and Type | Method and Description |
---|---|
DataSet<Vertex<K,Double>> |
PageRank.run(Graph<K,Double,Double> network) |
DataSet<Vertex<K,Double>> |
GSAPageRank.run(Graph<K,Double,Double> network) |
Modifier and Type | Method and Description |
---|---|
static boolean |
IncrementalSSSP.isInSSSP(Edge<Long,Double> edgeToBeRemoved,
DataSet<Edge<Long,Double>> edgesInSSSP)
Function that verifies whether the edge to be removed is part of the SSSP or not.
|
Modifier and Type | Method and Description |
---|---|
static DataSet<Vertex<LongValue,NullValue>> |
GraphGeneratorUtils.vertexSequence(ExecutionEnvironment env,
int parallelism,
long vertexCount)
Generates
Vertices with sequential, numerical labels. |
static <K,EV> DataSet<Vertex<K,NullValue>> |
GraphGeneratorUtils.vertexSet(DataSet<Edge<K,EV>> edges,
int parallelism)
|
Modifier and Type | Method and Description |
---|---|
static <K,EV> DataSet<Vertex<K,NullValue>> |
GraphGeneratorUtils.vertexSet(DataSet<Edge<K,EV>> edges,
int parallelism)
|
Modifier and Type | Method and Description |
---|---|
DataSet<Vertex<K,VV>> |
GatherSumApplyIteration.createResult()
Computes the results of the gather-sum-apply iteration
|
Modifier and Type | Method and Description |
---|---|
List<Tuple2<String,DataSet<?>>> |
GSAConfiguration.getApplyBcastVars()
Get the broadcast variables of the ApplyFunction.
|
List<Tuple2<String,DataSet<?>>> |
GSAConfiguration.getGatherBcastVars()
Get the broadcast variables of the GatherFunction.
|
List<Tuple2<String,DataSet<?>>> |
GSAConfiguration.getSumBcastVars()
Get the broadcast variables of the SumFunction.
|
Modifier and Type | Method and Description |
---|---|
void |
GSAConfiguration.addBroadcastSetForApplyFunction(String name,
DataSet<?> data)
Adds a data set as a broadcast set to the apply function.
|
void |
GSAConfiguration.addBroadcastSetForGatherFunction(String name,
DataSet<?> data)
Adds a data set as a broadcast set to the gather function.
|
void |
GSAConfiguration.addBroadcastSetForSumFunction(String name,
DataSet<?> data)
Adds a data set as a broadcast set to the sum function.
|
void |
GatherSumApplyIteration.setInput(DataSet<Vertex<K,VV>> dataSet)
Sets the input data set for this operator.
|
static <K,VV,EV,M> |
GatherSumApplyIteration.withEdges(DataSet<Edge<K,EV>> edges,
GatherFunction<VV,EV,M> gather,
SumFunction<VV,EV,M> sum,
ApplyFunction<K,VV,M> apply,
int maximumNumberOfIterations)
Creates a new gather-sum-apply iteration operator for graphs
|
Modifier and Type | Method and Description |
---|---|
DataSet<Vertex<K,Double>> |
SingleSourceShortestPaths.run(Graph<K,VV,Double> input) |
DataSet<Vertex<K,Double>> |
GSASingleSourceShortestPaths.run(Graph<K,VV,Double> input) |
DataSet<Tuple3<K,K,K>> |
TriangleEnumerator.run(Graph<K,VV,EV> input) |
DataSet<Vertex<K,VV>> |
LabelPropagation.run(Graph<K,VV,EV> input) |
DataSet<Vertex<K,VV>> |
GSAConnectedComponents.run(Graph<K,VV,EV> graph) |
DataSet<Vertex<K,VV>> |
ConnectedComponents.run(Graph<K,VV,EV> graph) |
Modifier and Type | Method and Description |
---|---|
DataSet<TriangleListing.Result<K>> |
TriangleListing.runInternal(Graph<K,VV,EV> input) |
DataSet<LocalClusteringCoefficient.Result<K>> |
LocalClusteringCoefficient.runInternal(Graph<K,VV,EV> input) |
Modifier and Type | Method and Description |
---|---|
DataSet<TriangleListing.Result<K>> |
TriangleListing.runInternal(Graph<K,VV,EV> input) |
DataSet<LocalClusteringCoefficient.Result<K>> |
LocalClusteringCoefficient.runInternal(Graph<K,VV,EV> input) |
Modifier and Type | Method and Description |
---|---|
DataSet<PageRank.Result<K>> |
PageRank.runInternal(Graph<K,VV,EV> input) |
DataSet<HITS.Result<K>> |
HITS.runInternal(Graph<K,VV,EV> input) |
Modifier and Type | Method and Description |
---|---|
DataSet<JaccardIndex.Result<K>> |
JaccardIndex.runInternal(Graph<K,VV,EV> input) |
DataSet<AdamicAdar.Result<K>> |
AdamicAdar.runInternal(Graph<K,VV,EV> input) |
Modifier and Type | Method and Description |
---|---|
DataSet<Vertex<K,VV>> |
VertexCentricIteration.createResult()
Creates the operator that represents this vertex-centric graph computation.
|
Modifier and Type | Method and Description |
---|---|
List<Tuple2<String,DataSet<?>>> |
VertexCentricConfiguration.getBcastVars()
Get the broadcast variables of the compute function.
|
Modifier and Type | Method and Description |
---|---|
void |
VertexCentricConfiguration.addBroadcastSet(String name,
DataSet<?> data)
Adds a data set as a broadcast set to the compute function.
|
void |
VertexCentricIteration.setInput(DataSet<Vertex<K,VV>> inputData)
Sets the input data set for this operator.
|
static <K,VV,EV,Message> |
VertexCentricIteration.withEdges(DataSet<Edge<K,EV>> edgesWithValue,
ComputeFunction<K,VV,EV,Message> cf,
int maximumNumberOfIterations)
Creates a new vertex-centric iteration operator.
|
static <K,VV,EV,Message> |
VertexCentricIteration.withEdges(DataSet<Edge<K,EV>> edgesWithValue,
ComputeFunction<K,VV,EV,Message> cf,
MessageCombiner<K,Message> mc,
int maximumNumberOfIterations)
Creates a new vertex-centric iteration operator for graphs where the edges are associated with a value (such as
a weight or distance).
|
Modifier and Type | Method and Description |
---|---|
DataSet<Vertex<K,VV>> |
ScatterGatherIteration.createResult()
Creates the operator that represents this scatter-gather graph computation.
|
Modifier and Type | Method and Description |
---|---|
List<Tuple2<String,DataSet<?>>> |
ScatterGatherConfiguration.getGatherBcastVars()
Get the broadcast variables of the GatherFunction.
|
List<Tuple2<String,DataSet<?>>> |
ScatterGatherConfiguration.getScatterBcastVars()
Get the broadcast variables of the ScatterFunction.
|
Modifier and Type | Method and Description |
---|---|
void |
ScatterGatherConfiguration.addBroadcastSetForGatherFunction(String name,
DataSet<?> data)
Adds a data set as a broadcast set to the gather function.
|
void |
ScatterGatherConfiguration.addBroadcastSetForScatterFunction(String name,
DataSet<?> data)
Adds a data set as a broadcast set to the scatter function.
|
void |
ScatterGatherIteration.setInput(DataSet<Vertex<K,VV>> inputData)
Sets the input data set for this operator.
|
static <K,VV,Message,EV> |
ScatterGatherIteration.withEdges(DataSet<Edge<K,EV>> edgesWithValue,
ScatterFunction<K,VV,Message,EV> sf,
GatherFunction<K,VV,Message> gf,
int maximumNumberOfIterations)
Creates a new scatter-gather iteration operator for graphs where the edges are associated with a value (such as
a weight or distance).
|
Modifier and Type | Method and Description |
---|---|
static <T> DataSet<LongValue> |
GraphUtils.count(DataSet<T> input)
Count the number of elements in a DataSet.
|
Modifier and Type | Method and Description |
---|---|
static <T> DataSet<LongValue> |
GraphUtils.count(DataSet<T> input)
Count the number of elements in a DataSet.
|
Modifier and Type | Method and Description |
---|---|
DataSet<T> |
GraphAlgorithmWrappingDataSet.run(Graph<K,VV,EV> input) |
protected abstract DataSet<T> |
GraphAlgorithmWrappingDataSet.runInternal(Graph<K,VV,EV> input)
The implementation of the algorithm, renamed from
GraphAlgorithm.run(Graph) . |
Modifier and Type | Method and Description |
---|---|
<D extends DataSet<?>> |
SetCache.add(int id,
D set)
Adds the given
DataSet to this cache for the given ID. |
Modifier and Type | Method and Description |
---|---|
<T> DataSet<T> |
SetCache.getDataSet(int id)
Returns the cached
DataSet for the given ID. |
Modifier and Type | Method and Description |
---|---|
protected <A> DataSet<A> |
BatchTableEnvironment.translate(org.apache.calcite.rel.RelNode logicalPlan,
org.apache.calcite.rel.type.RelDataType logicalType,
TypeInformation<A> tpe)
Translates a logical
RelNode into a DataSet . |
protected <A> DataSet<A> |
BatchTableEnvironment.translate(Table table,
TypeInformation<A> tpe)
Translates a
Table into a DataSet . |
Modifier and Type | Method and Description |
---|---|
protected <T> void |
BatchTableEnvironment.registerDataSetInternal(String name,
DataSet<T> dataSet)
Registers a
DataSet as a table under a given name in the TableEnvironment 's catalog. |
protected <T> void |
BatchTableEnvironment.registerDataSetInternal(String name,
DataSet<T> dataSet,
Expression[] fields)
Registers a
DataSet as a table under a given name with field names as specified by
field expressions in the TableEnvironment 's catalog. |
Modifier and Type | Method and Description |
---|---|
<T> DataSet<T> |
BatchTableEnvironment.toDataSet(Table table,
Class<T> clazz)
Converts the given
Table into a DataSet of a specified type. |
<T> DataSet<T> |
BatchTableEnvironment.toDataSet(Table table,
TypeInformation<T> typeInfo)
Converts the given
Table into a DataSet of a specified type. |
Modifier and Type | Method and Description |
---|---|
<T> Table |
BatchTableEnvironment.fromDataSet(DataSet<T> dataSet)
Converts the given
DataSet into a Table . |
<T> Table |
BatchTableEnvironment.fromDataSet(DataSet<T> dataSet,
String fields)
Converts the given
DataSet into a Table with specified field names. |
<T> void |
BatchTableEnvironment.registerDataSet(String name,
DataSet<T> dataSet)
Registers the given
DataSet as table in the
TableEnvironment 's catalog. |
<T> void |
BatchTableEnvironment.registerDataSet(String name,
DataSet<T> dataSet,
String fields)
Registers the given
DataSet as table with specified field names in the
TableEnvironment 's catalog. |
Modifier and Type | Method and Description |
---|---|
DataSet<Row> |
BatchScan.convertToInternalRow(DataSet<Object> input,
FlinkTable<?> flinkTable,
TableConfig config) |
Modifier and Type | Method and Description |
---|---|
DataSet<T> |
DataSetTable.dataSet() |
Constructor and Description |
---|
DataSetTable(DataSet<T> dataSet,
int[] fieldIndexes,
String[] fieldNames,
FlinkStatistic statistic) |
Modifier and Type | Method and Description |
---|---|
void |
CsvTableSink.emitDataSet(DataSet<Row> dataSet) |
void |
BatchTableSink.emitDataSet(DataSet<T> dataSet)
Emits the DataSet.
|
Modifier and Type | Method and Description |
---|---|
DataSet<Row> |
CsvTableSource.getDataSet(ExecutionEnvironment execEnv)
Returns the data of the table as a
DataSet of Row . |
DataSet<T> |
BatchTableSource.getDataSet(ExecutionEnvironment execEnv)
Returns the data of the table as a
DataSet . |
Copyright © 2014–2018 The Apache Software Foundation. All rights reserved.