Package | Description |
---|---|
org.apache.flink.api.common | |
org.apache.flink.api.common.operators | |
org.apache.flink.api.common.operators.base | |
org.apache.flink.api.java.operators | |
org.apache.flink.api.java.operators.translation | |
org.apache.flink.api.scala.operators | |
org.apache.flink.optimizer.dag | |
org.apache.flink.optimizer.plan | |
org.apache.flink.optimizer.traversals |
This package contains the various traversals over the program plan and the optimizer DAG
(directed acyclic graph) that are made in the course of the optimization.
|
org.apache.flink.optimizer.util |
Modifier and Type | Method and Description |
---|---|
void |
Plan.accept(Visitor<Operator<?>> visitor)
Traverses the job depth first from all data sinks on towards the sources.
|
Modifier and Type | Class and Description |
---|---|
class |
AbstractUdfOperator<OUT,FT extends Function>
Abstract superclass for all contracts that represent actual operators.
|
class |
DualInputOperator<IN1,IN2,OUT,FT extends Function>
Abstract operator superclass for all operators that have two inputs, like "Join", "CoGroup", or
"Cross".
|
class |
GenericDataSinkBase<IN>
Operator for nodes that act as data sinks, storing the data they receive.
|
class |
GenericDataSourceBase<OUT,T extends InputFormat<OUT,?>>
Abstract superclass for data sources in a Pact plan.
|
class |
SingleInputOperator<IN,OUT,FT extends Function>
Abstract superclass for all operators that have one input like "map" or "reduce".
|
class |
Union<T>
This operator represents a Union between two inputs.
|
Modifier and Type | Field and Description |
---|---|
protected Operator<IN> |
GenericDataSinkBase.input |
protected Operator<IN> |
SingleInputOperator.input
The input which produces the data consumed by this operator.
|
protected Operator<IN1> |
DualInputOperator.input1
The operator producing the first input.
|
protected Operator<IN2> |
DualInputOperator.input2
The operator producing the second input.
|
Modifier and Type | Field and Description |
---|---|
protected Map<String,Operator<?>> |
AbstractUdfOperator.broadcastInputs
The extra inputs which parameterize the user function.
|
Modifier and Type | Method and Description |
---|---|
static <T> Operator<T> |
Operator.createUnionCascade(List<? extends Operator<T>> operators)
Takes a list of operators and creates a cascade of unions of this inputs, if needed.
|
static <T> Operator<T> |
Operator.createUnionCascade(Operator<T>... operators)
Takes a list of operators and creates a cascade of unions of this inputs, if needed.
|
static <T> Operator<T> |
Operator.createUnionCascade(Operator<T> input1,
Operator<T>... input2)
Takes a single Operator and a list of operators and creates a cascade of unions of this
inputs, if needed.
|
Operator<IN1> |
DualInputOperator.getFirstInput()
Returns the first input, or null, if none is set.
|
Operator<IN> |
GenericDataSinkBase.getInput()
Returns this operator's input operator.
|
Operator<IN> |
SingleInputOperator.getInput()
Returns the input operator or data source, or null, if none is set.
|
Operator<IN2> |
DualInputOperator.getSecondInput()
Returns the second input, or null, if none is set.
|
Modifier and Type | Method and Description |
---|---|
Map<String,Operator<?>> |
AbstractUdfOperator.getBroadcastInputs()
Returns the input, or null, if none is set.
|
Modifier and Type | Method and Description |
---|---|
void |
DualInputOperator.addFirstInput(Operator<IN1>... input)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
void |
GenericDataSinkBase.addInput(Operator<IN>... inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator instead. |
void |
SingleInputOperator.addInput(Operator<IN>... input)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
void |
DualInputOperator.addSecondInput(Operator<IN2>... input)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
static <T> Operator<T> |
Operator.createUnionCascade(Operator<T>... operators)
Takes a list of operators and creates a cascade of unions of this inputs, if needed.
|
static <T> Operator<T> |
Operator.createUnionCascade(Operator<T> input1,
Operator<T>... input2)
Takes a single Operator and a list of operators and creates a cascade of unions of this
inputs, if needed.
|
static <T> Operator<T> |
Operator.createUnionCascade(Operator<T> input1,
Operator<T>... input2)
Takes a single Operator and a list of operators and creates a cascade of unions of this
inputs, if needed.
|
void |
AbstractUdfOperator.setBroadcastVariable(String name,
Operator<?> root)
Binds the result produced by a plan rooted at
root to a variable used by the UDF
wrapped in this operator. |
void |
DualInputOperator.setFirstInput(Operator<IN1>... inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
void |
DualInputOperator.setFirstInput(Operator<IN1> input)
Clears all previous connections and connects the first input to the task wrapped in this
contract
|
void |
SingleInputOperator.setInput(Operator<IN>... input)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
void |
GenericDataSinkBase.setInput(Operator<IN> input)
Sets the given operator as the input to this operator.
|
void |
SingleInputOperator.setInput(Operator<IN> input)
Sets the given operator as the input to this operator.
|
void |
GenericDataSinkBase.setInputs(Operator<IN>... inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator instead. |
void |
DualInputOperator.setSecondInput(Operator<IN2>... inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
void |
DualInputOperator.setSecondInput(Operator<IN2> input)
Clears all previous connections and connects the second input to the task wrapped in this
contract
|
Modifier and Type | Method and Description |
---|---|
void |
GenericDataSourceBase.accept(Visitor<Operator<?>> visitor)
Accepts the visitor and applies it this instance.
|
void |
DualInputOperator.accept(Visitor<Operator<?>> visitor) |
void |
GenericDataSinkBase.accept(Visitor<Operator<?>> visitor)
Accepts the visitor and applies it this instance.
|
void |
SingleInputOperator.accept(Visitor<Operator<?>> visitor)
Accepts the visitor and applies it this instance.
|
void |
DualInputOperator.addFirstInputs(List<Operator<IN1>> inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
void |
SingleInputOperator.addInput(List<Operator<IN>> inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
void |
GenericDataSinkBase.addInputs(List<? extends Operator<IN>> inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator instead. |
void |
DualInputOperator.addSecondInputs(List<Operator<IN2>> inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
static <T> Operator<T> |
Operator.createUnionCascade(List<? extends Operator<T>> operators)
Takes a list of operators and creates a cascade of unions of this inputs, if needed.
|
<T> void |
AbstractUdfOperator.setBroadcastVariables(Map<String,Operator<T>> inputs)
Clears all previous broadcast inputs and binds the given inputs as broadcast variables of
this operator.
|
void |
DualInputOperator.setFirstInputs(List<Operator<IN1>> inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
void |
GenericDataSinkBase.setInputs(List<Operator<IN>> inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator instead. |
void |
SingleInputOperator.setInputs(List<Operator<IN>> inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
void |
DualInputOperator.setSecondInputs(List<Operator<IN2>> inputs)
Deprecated.
This method will be removed in future versions. Use the
Union operator
instead. |
Constructor and Description |
---|
Union(Operator<T> input1,
Operator<T> input2,
String unionLocationName) |
Union(Operator<T> input1,
Operator<T> input2,
String unionLocationName) |
Modifier and Type | Class and Description |
---|---|
class |
BulkIterationBase<T> |
static class |
BulkIterationBase.PartialSolutionPlaceHolder<OT>
Specialized operator to use as a recognizable place-holder for the input to the step function
when composing the nested data flow.
|
class |
CoGroupOperatorBase<IN1,IN2,OUT,FT extends CoGroupFunction<IN1,IN2,OUT>> |
class |
CoGroupRawOperatorBase<IN1,IN2,OUT,FT extends CoGroupFunction<IN1,IN2,OUT>> |
class |
CrossOperatorBase<IN1,IN2,OUT,FT extends CrossFunction<IN1,IN2,OUT>> |
class |
DeltaIterationBase<ST,WT>
A DeltaIteration is similar to a
BulkIterationBase , but maintains state across the
individual iteration steps. |
static class |
DeltaIterationBase.SolutionSetPlaceHolder<ST>
Specialized operator to use as a recognizable place-holder for the solution set input to the
step function.
|
static class |
DeltaIterationBase.WorksetPlaceHolder<WT>
Specialized operator to use as a recognizable place-holder for the working set input to the
step function.
|
class |
FilterOperatorBase<T,FT extends FlatMapFunction<T,T>> |
class |
FlatMapOperatorBase<IN,OUT,FT extends FlatMapFunction<IN,OUT>> |
class |
GroupCombineOperatorBase<IN,OUT,FT extends GroupCombineFunction<IN,OUT>>
Base operator for the combineGroup transformation.
|
class |
GroupReduceOperatorBase<IN,OUT,FT extends GroupReduceFunction<IN,OUT>> |
class |
InnerJoinOperatorBase<IN1,IN2,OUT,FT extends FlatJoinFunction<IN1,IN2,OUT>> |
class |
JoinOperatorBase<IN1,IN2,OUT,FT extends FlatJoinFunction<IN1,IN2,OUT>> |
class |
MapOperatorBase<IN,OUT,FT extends MapFunction<IN,OUT>> |
class |
MapPartitionOperatorBase<IN,OUT,FT extends MapPartitionFunction<IN,OUT>> |
class |
OuterJoinOperatorBase<IN1,IN2,OUT,FT extends FlatJoinFunction<IN1,IN2,OUT>> |
class |
PartitionOperatorBase<IN> |
class |
ReduceOperatorBase<T,FT extends ReduceFunction<T>>
Base data flow operator for Reduce user-defined functions.
|
class |
SortPartitionOperatorBase<IN> |
Modifier and Type | Field and Description |
---|---|
protected Operator<?> |
BulkIterationBase.terminationCriterion |
Modifier and Type | Method and Description |
---|---|
Operator<ST> |
DeltaIterationBase.getInitialSolutionSet()
Returns the initial solution set input, or null, if none is set.
|
Operator<WT> |
DeltaIterationBase.getInitialWorkset()
Returns the initial workset input, or null, if none is set.
|
Operator<T> |
BulkIterationBase.getNextPartialSolution() |
Operator<WT> |
DeltaIterationBase.getNextWorkset()
Gets the contract that has been set as the next workset.
|
Operator<T> |
BulkIterationBase.getPartialSolution() |
Operator<ST> |
DeltaIterationBase.getSolutionSet()
Gets the contract that represents the solution set for the step function.
|
Operator<ST> |
DeltaIterationBase.getSolutionSetDelta()
Gets the contract that has been set as the solution set delta.
|
Operator<?> |
BulkIterationBase.getTerminationCriterion() |
Operator<WT> |
DeltaIterationBase.getWorkset()
Gets the contract that represents the workset for the step function.
|
Modifier and Type | Method and Description |
---|---|
Map<String,Operator<?>> |
BulkIterationBase.getBroadcastInputs()
The BulkIteration meta operator cannot have broadcast inputs.
|
Map<String,Operator<?>> |
DeltaIterationBase.getBroadcastInputs()
DeltaIteration meta operator cannot have broadcast inputs.
|
Modifier and Type | Method and Description |
---|---|
void |
BulkIterationBase.setBroadcastVariable(String name,
Operator<?> root)
The BulkIteration meta operator cannot have broadcast inputs.
|
void |
DeltaIterationBase.setBroadcastVariable(String name,
Operator<?> root)
The DeltaIteration meta operator cannot have broadcast inputs.
|
void |
DeltaIterationBase.setInitialSolutionSet(Operator<ST> input)
Sets the given input as the initial solution set.
|
void |
DeltaIterationBase.setInitialWorkset(Operator<WT> input)
Sets the given input as the initial workset.
|
void |
BulkIterationBase.setNextPartialSolution(Operator<T> result) |
void |
DeltaIterationBase.setNextWorkset(Operator<WT> result)
Sets the contract of the step function that represents the next workset.
|
void |
DeltaIterationBase.setSolutionSetDelta(Operator<ST> delta)
Sets the contract of the step function that represents the solution set delta.
|
<X> void |
BulkIterationBase.setTerminationCriterion(Operator<X> criterion) |
Modifier and Type | Method and Description |
---|---|
void |
BulkIterationBase.PartialSolutionPlaceHolder.accept(Visitor<Operator<?>> visitor) |
void |
DeltaIterationBase.WorksetPlaceHolder.accept(Visitor<Operator<?>> visitor) |
void |
DeltaIterationBase.SolutionSetPlaceHolder.accept(Visitor<Operator<?>> visitor) |
<X> void |
BulkIterationBase.setBroadcastVariables(Map<String,Operator<X>> inputs)
The BulkIteration meta operator cannot have broadcast inputs.
|
<X> void |
DeltaIterationBase.setBroadcastVariables(Map<String,Operator<X>> inputs)
The DeltaIteration meta operator cannot have broadcast inputs.
|
Modifier and Type | Method and Description |
---|---|
static <T,K> Operator<Tuple2<K,T>> |
KeyFunctions.appendKeyExtractor(Operator<T> input,
Keys.SelectorFunctionKeys<T,K> key) |
static <T,K1,K2> Operator<Tuple3<K1,K2,T>> |
KeyFunctions.appendKeyExtractor(Operator<T> input,
Keys.SelectorFunctionKeys<T,K1> key1,
Keys.SelectorFunctionKeys<T,K2> key2) |
protected abstract Operator<OUT> |
SingleInputOperator.translateToDataFlow(Operator<IN> input)
Deprecated.
Translates this operation to a data flow operator of the common data flow API.
|
protected abstract Operator<OUT> |
TwoInputOperator.translateToDataFlow(Operator<IN1> input1,
Operator<IN2> input2)
Deprecated.
Translates this java API operator into a common API operator with two inputs.
|
Modifier and Type | Method and Description |
---|---|
static <T,K> Operator<Tuple2<K,T>> |
KeyFunctions.appendKeyExtractor(Operator<T> input,
Keys.SelectorFunctionKeys<T,K> key) |
static <T,K1,K2> Operator<Tuple3<K1,K2,T>> |
KeyFunctions.appendKeyExtractor(Operator<T> input,
Keys.SelectorFunctionKeys<T,K1> key1,
Keys.SelectorFunctionKeys<T,K2> key2) |
static <T,K> SingleInputOperator<?,T,?> |
KeyFunctions.appendKeyRemover(Operator<Tuple2<K,T>> inputWithKey,
Keys.SelectorFunctionKeys<T,K> key) |
protected JoinOperatorBase<?,?,OUT,?> |
JoinOperator.EquiJoin.translateToDataFlow(Operator<I1> input1,
Operator<I2> input2) |
protected JoinOperatorBase<?,?,OUT,?> |
JoinOperator.EquiJoin.translateToDataFlow(Operator<I1> input1,
Operator<I2> input2) |
protected CoGroupOperatorBase<?,?,OUT,?> |
CoGroupOperator.translateToDataFlow(Operator<I1> input1,
Operator<I2> input2)
Deprecated.
|
protected CoGroupOperatorBase<?,?,OUT,?> |
CoGroupOperator.translateToDataFlow(Operator<I1> input1,
Operator<I2> input2)
Deprecated.
|
protected CoGroupRawOperatorBase<?,?,OUT,?> |
CoGroupRawOperator.translateToDataFlow(Operator<I1> input1,
Operator<I2> input2) |
protected CoGroupRawOperatorBase<?,?,OUT,?> |
CoGroupRawOperator.translateToDataFlow(Operator<I1> input1,
Operator<I2> input2) |
protected CrossOperatorBase<I1,I2,OUT,CrossFunction<I1,I2,OUT>> |
CrossOperator.translateToDataFlow(Operator<I1> input1,
Operator<I2> input2)
Deprecated.
|
protected CrossOperatorBase<I1,I2,OUT,CrossFunction<I1,I2,OUT>> |
CrossOperator.translateToDataFlow(Operator<I1> input1,
Operator<I2> input2)
Deprecated.
|
protected MapOperatorBase<IN,OUT,MapFunction<IN,OUT>> |
ProjectOperator.translateToDataFlow(Operator<IN> input)
Deprecated.
|
protected GroupCombineOperatorBase<?,OUT,?> |
GroupCombineOperator.translateToDataFlow(Operator<IN> input)
Deprecated.
|
protected MapOperatorBase<IN,OUT,MapFunction<IN,OUT>> |
MapOperator.translateToDataFlow(Operator<IN> input)
Deprecated.
|
protected GroupReduceOperatorBase<?,OUT,?> |
GroupReduceOperator.translateToDataFlow(Operator<IN> input)
Deprecated.
|
protected FlatMapOperatorBase<IN,OUT,FlatMapFunction<IN,OUT>> |
FlatMapOperator.translateToDataFlow(Operator<IN> input)
Deprecated.
|
protected GroupReduceOperatorBase<IN,IN,GroupReduceFunction<IN,IN>> |
AggregateOperator.translateToDataFlow(Operator<IN> input)
Deprecated.
|
protected MapPartitionOperatorBase<IN,OUT,MapPartitionFunction<IN,OUT>> |
MapPartitionOperator.translateToDataFlow(Operator<IN> input)
Deprecated.
|
protected abstract Operator<OUT> |
SingleInputOperator.translateToDataFlow(Operator<IN> input)
Deprecated.
Translates this operation to a data flow operator of the common data flow API.
|
protected SingleInputOperator<?,IN,?> |
ReduceOperator.translateToDataFlow(Operator<IN> input)
Deprecated.
|
protected abstract Operator<OUT> |
TwoInputOperator.translateToDataFlow(Operator<IN1> input1,
Operator<IN2> input2)
Deprecated.
Translates this java API operator into a common API operator with two inputs.
|
protected abstract Operator<OUT> |
TwoInputOperator.translateToDataFlow(Operator<IN1> input1,
Operator<IN2> input2)
Deprecated.
Translates this java API operator into a common API operator with two inputs.
|
protected SingleInputOperator<?,T,?> |
SortPartitionOperator.translateToDataFlow(Operator<T> input)
Deprecated.
|
protected SingleInputOperator<?,T,?> |
PartitionOperator.translateToDataFlow(Operator<T> input)
Deprecated.
|
protected SingleInputOperator<?,T,?> |
DistinctOperator.translateToDataFlow(Operator<T> input)
Deprecated.
|
protected FilterOperatorBase<T,FlatMapFunction<T,T>> |
FilterOperator.translateToDataFlow(Operator<T> input)
Deprecated.
|
protected SingleInputOperator<T,T,?> |
IterativeDataSet.translateToDataFlow(Operator<T> input)
Deprecated.
|
protected GenericDataSinkBase<T> |
DataSink.translateToDataFlow(Operator<T> input)
Deprecated.
|
protected Union<T> |
UnionOperator.translateToDataFlow(Operator<T> input1,
Operator<T> input2)
Deprecated.
Returns the BinaryNodeTranslation of the Union.
|
protected Union<T> |
UnionOperator.translateToDataFlow(Operator<T> input1,
Operator<T> input2)
Deprecated.
Returns the BinaryNodeTranslation of the Union.
|
Modifier and Type | Class and Description |
---|---|
class |
PlanBothUnwrappingCoGroupOperator<I1,I2,OUT,K>
A co group operator that applies the operation only on the unwrapped values.
|
class |
PlanFilterOperator<T> |
class |
PlanLeftUnwrappingCoGroupOperator<I1,I2,OUT,K>
A co group operator that applies the operation only on the unwrapped values on the left.
|
class |
PlanProjectOperator<T,R extends Tuple>
A map operator that retains a subset of fields from incoming tuples.
|
class |
PlanRightUnwrappingCoGroupOperator<I1,I2,OUT,K>
A co group operator that applies the operation only on the unwrapped values on the right.
|
class |
PlanUnwrappingGroupCombineOperator<IN,OUT,K>
A group combine operator that takes 2-tuples (key-value pairs), and applies the group combine
operation only on the unwrapped values.
|
class |
PlanUnwrappingReduceGroupOperator<IN,OUT,K>
A reduce operator that takes 2-tuples (key-value pairs), and applies the group reduce operation
only on the unwrapped values.
|
class |
PlanUnwrappingReduceOperator<T,K>
A reduce operator that takes 2-tuples (key-value pairs), and applies the reduce operation only on
the unwrapped values.
|
class |
PlanUnwrappingSortedGroupCombineOperator<IN,OUT,K1,K2>
A reduce operator that takes 3-tuples (groupKey, sortKey, value), and applies the sorted partial
group reduce operation only on the unwrapped values.
|
class |
PlanUnwrappingSortedReduceGroupOperator<IN,OUT,K1,K2>
A reduce operator that takes 3-tuples (groupKey, sortKey, value), and applies the sorted group
reduce operation only on the unwrapped values.
|
Modifier and Type | Method and Description |
---|---|
protected GroupReduceOperatorBase<IN,IN,GroupReduceFunction<IN,IN>> |
ScalaAggregateOperator.translateToDataFlow(Operator<IN> input)
Deprecated.
|
Modifier and Type | Method and Description |
---|---|
Operator<?> |
OptimizerNode.getOperator()
Gets the operator represented by this optimizer node.
|
Modifier and Type | Method and Description |
---|---|
void |
OptimizerNode.setBroadcastInputs(Map<Operator<?>,OptimizerNode> operatorToNode,
ExecutionMode defaultExchangeMode)
This function connects the operators that produce the broadcast inputs to this operator.
|
void |
SingleInputNode.setInput(Map<Operator<?>,OptimizerNode> contractToNode,
ExecutionMode defaultExchangeMode) |
void |
AbstractPartialSolutionNode.setInput(Map<Operator<?>,OptimizerNode> contractToNode,
ExecutionMode dataExchangeMode) |
void |
TwoInputNode.setInput(Map<Operator<?>,OptimizerNode> contractToNode,
ExecutionMode defaultExecutionMode) |
void |
DataSinkNode.setInput(Map<Operator<?>,OptimizerNode> contractToNode,
ExecutionMode defaultExchangeMode) |
void |
DataSourceNode.setInput(Map<Operator<?>,OptimizerNode> contractToNode,
ExecutionMode defaultDataExchangeMode) |
abstract void |
OptimizerNode.setInput(Map<Operator<?>,OptimizerNode> operatorToNode,
ExecutionMode defaultExchangeMode)
This function connects the predecessors to this operator.
|
Constructor and Description |
---|
AbstractPartialSolutionNode(Operator<?> contract) |
OptimizerNode(Operator<?> op)
Creates a new optimizer node that represents the given program operator.
|
Modifier and Type | Method and Description |
---|---|
Operator<?> |
PlanNode.getProgramOperator()
Gets the program operator that this node represents in the plan.
|
Modifier and Type | Method and Description |
---|---|
void |
StepFunctionValidator.postVisit(Operator<?> visitable) |
void |
GraphCreatingVisitor.postVisit(Operator<?> c) |
boolean |
StepFunctionValidator.preVisit(Operator<?> visitable) |
boolean |
GraphCreatingVisitor.preVisit(Operator<?> c) |
Modifier and Type | Class and Description |
---|---|
class |
NoOpBinaryUdfOp<OUT> |
class |
NoOpUnaryUdfOp<OUT> |
Copyright © 2014–2024 The Apache Software Foundation. All rights reserved.