org.apache.flink.optimizer.plan

Class Channel

java.lang.Object

org.apache.flink.optimizer.plan.Channel

All Implemented Interfaces:

Cloneable, EstimateProvider, DumpableConnection<PlanNode>

Direct Known Subclasses:

NamedChannel

public class Channel
extends Object
implements EstimateProvider, Cloneable, DumpableConnection<PlanNode>

A Channel represents the result produced by an operator and the data exchange before the consumption by the target operator.

The channel defines and tracks various properties and characteristics of the data set and data exchange.

Data set characteristics:

• The "global properties" of the data, i.e., how the data is distributed across partitions
• The "required global properties" of the data, i.e., the global properties that, if absent, would cause the program to return a wrong result.
• The "local properties" of the data, i.e., how the data is organized within a partition
• The "required local properties" of the data, i.e., the local properties that, if absent, would cause the program to return a wrong result.

Data exchange parameters:

• The "ship strategy", i.e., whether to forward the data, shuffle it, broadcast it, ...
• The "ship keys", which are the positions of the key fields in the exchanged records.
• The "data exchange mode", which defines whether to pipeline or batch the exchange
• Several more...

Constructor Summary

Constructors

Constructor and Description
Channel(PlanNode sourceNode)
Channel(PlanNode sourceNode, TempMode tempMode)

Method Summary

Modifier and Type	Method and Description
void	adjustGlobalPropertiesForFullParallelismChange()
Channel	clone()
int	getCostWeight()
DataDistribution	getDataDistribution()
DataExchangeMode	getDataExchangeMode() Gets the data exchange mode (batch / pipelined) to use for the data exchange of this channel.
float	getEstimatedAvgWidthPerOutputRecord() Gets the estimated number of bytes per record.
long	getEstimatedNumRecords() Gets the estimated number of records in the output of this node.
long	getEstimatedOutputSize() Gets the estimated output size from this node.
GlobalProperties	getGlobalProperties()
LocalProperties	getLocalProperties()
LocalStrategy	getLocalStrategy()
TypeComparatorFactory<?>	getLocalStrategyComparator() Gets the local strategy comparator from this Channel.
FieldList	getLocalStrategyKeys()
boolean[]	getLocalStrategySortOrder()
int	getMaxDepth()
Partitioner<?>	getPartitioner()
double	getRelativeMemoryLocalStrategy()
double	getRelativeTempMemory() Gets the memory for materializing the channel's result from this Channel.
int	getReplicationFactor() Returns the replication factor of the connection.
RequestedGlobalProperties	getRequiredGlobalProps()
RequestedLocalProperties	getRequiredLocalProps()
TypeSerializerFactory<?>	getSerializer() Gets the serializer from this Channel.
ShipStrategyType	getShipStrategy()
TypeComparatorFactory<?>	getShipStrategyComparator() Gets the ship strategy comparator from this Channel.
FieldList	getShipStrategyKeys()
boolean[]	getShipStrategySortOrder()
PlanNode	getSource() Gets the source of this Channel.
PlanNode	getTarget() Gets the target of this Channel.
TempMode	getTempMode()
boolean	isOnDynamicPath()
void	setDataDistribution(DataDistribution dataDistribution)
void	setDataExchangeMode(DataExchangeMode dataExchangeMode) Sets the data exchange mode (batch / pipelined) to use for the data exchange of this channel.
void	setLocalStrategy(LocalStrategy strategy)
void	setLocalStrategy(LocalStrategy strategy, FieldList keys, boolean[] sortDirection)
void	setLocalStrategyComparator(TypeComparatorFactory<?> localStrategyComparator) Sets the local strategy comparator for this Channel.
void	setRelativeMemoryLocalStrategy(double relativeMemoryLocalStrategy)
void	setRelativeTempMemory(double relativeTempMemory) Sets the memory for materializing the channel's result from this Channel.
void	setReplicationFactor(int factor) Sets the replication factor of the connection.
void	setRequiredGlobalProps(RequestedGlobalProperties requiredGlobalProps)
void	setRequiredLocalProps(RequestedLocalProperties requiredLocalProps)
void	setSerializer(TypeSerializerFactory<?> serializer) Sets the serializer for this Channel.
void	setShipStrategy(ShipStrategyType strategy, DataExchangeMode dataExchangeMode)
void	setShipStrategy(ShipStrategyType strategy, FieldList keys, boolean[] sortDirection, DataExchangeMode dataExchangeMode)
void	setShipStrategy(ShipStrategyType strategy, FieldList keys, boolean[] sortDirection, Partitioner<?> partitioner, DataExchangeMode dataExchangeMode)
void	setShipStrategy(ShipStrategyType strategy, FieldList keys, DataExchangeMode dataExchangeMode)
void	setShipStrategy(ShipStrategyType strategy, FieldList keys, Partitioner<?> partitioner, DataExchangeMode dataExchangeMode)
void	setShipStrategyComparator(TypeComparatorFactory<?> shipStrategyComparator) Sets the ship strategy comparator for this Channel.
void	setSource(PlanNode source)
void	setTarget(PlanNode target) Sets the target of this Channel.
void	setTempMode(TempMode tempMode) Sets the temp mode of the connection.
void	swapUnionNodes(PlanNode newUnionNode) Utility method used while swapping binary union nodes for n-ary union nodes.
String	toString()

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Channel

public Channel(PlanNode sourceNode)

Channel

public Channel(PlanNode sourceNode,
               TempMode tempMode)

Method Detail

getSource

public PlanNode getSource()

Gets the source of this Channel.

Specified by:

getSource in interface DumpableConnection<PlanNode>

Returns:

The source.

setSource

public void setSource(PlanNode source)

setTarget

public void setTarget(PlanNode target)

Sets the target of this Channel.

Parameters:

target - The target.

getTarget

public PlanNode getTarget()

Gets the target of this Channel.

Returns:

The target.

setShipStrategy

public void setShipStrategy(ShipStrategyType strategy,
                            DataExchangeMode dataExchangeMode)

setShipStrategy

public void setShipStrategy(ShipStrategyType strategy,
                            FieldList keys,
                            DataExchangeMode dataExchangeMode)

setShipStrategy

public void setShipStrategy(ShipStrategyType strategy,
                            FieldList keys,
                            boolean[] sortDirection,
                            DataExchangeMode dataExchangeMode)

setShipStrategy

public void setShipStrategy(ShipStrategyType strategy,
                            FieldList keys,
                            Partitioner<?> partitioner,
                            DataExchangeMode dataExchangeMode)

setShipStrategy

public void setShipStrategy(ShipStrategyType strategy,
                            FieldList keys,
                            boolean[] sortDirection,
                            Partitioner<?> partitioner,
                            DataExchangeMode dataExchangeMode)

setDataExchangeMode

public void setDataExchangeMode(DataExchangeMode dataExchangeMode)

Sets the data exchange mode (batch / pipelined) to use for the data exchange of this channel.

getDataExchangeMode

public DataExchangeMode getDataExchangeMode()

Gets the data exchange mode (batch / pipelined) to use for the data exchange of this channel.

Returns:

The data exchange mode of this channel.

getShipStrategy

public ShipStrategyType getShipStrategy()

Specified by:

getShipStrategy in interface DumpableConnection<PlanNode>

getShipStrategyKeys

public FieldList getShipStrategyKeys()

getShipStrategySortOrder

public boolean[] getShipStrategySortOrder()

setLocalStrategy

public void setLocalStrategy(LocalStrategy strategy)

setLocalStrategy

public void setLocalStrategy(LocalStrategy strategy,
                             FieldList keys,
                             boolean[] sortDirection)

getLocalStrategy

public LocalStrategy getLocalStrategy()

getLocalStrategyKeys

public FieldList getLocalStrategyKeys()

getLocalStrategySortOrder

public boolean[] getLocalStrategySortOrder()

setDataDistribution

public void setDataDistribution(DataDistribution dataDistribution)

getDataDistribution

public DataDistribution getDataDistribution()

getPartitioner

public Partitioner<?> getPartitioner()

getTempMode

public TempMode getTempMode()

setTempMode

public void setTempMode(TempMode tempMode)

Sets the temp mode of the connection.

Parameters:

tempMode - The temp mode of the connection.

getRelativeTempMemory

public double getRelativeTempMemory()

Gets the memory for materializing the channel's result from this Channel.

Returns:

The temp memory.

setRelativeTempMemory

public void setRelativeTempMemory(double relativeTempMemory)

Sets the memory for materializing the channel's result from this Channel.

Parameters:

relativeTempMemory - The memory for materialization.

setReplicationFactor

public void setReplicationFactor(int factor)

Sets the replication factor of the connection.

Parameters:

factor - The replication factor of the connection.

getReplicationFactor

public int getReplicationFactor()

Returns the replication factor of the connection.

Returns:

The replication factor of the connection.

getSerializer

public TypeSerializerFactory<?> getSerializer()

Gets the serializer from this Channel.

Returns:

The serializer.

setSerializer

public void setSerializer(TypeSerializerFactory<?> serializer)

Sets the serializer for this Channel.

Parameters:

serializer - The serializer to set.

getShipStrategyComparator

public TypeComparatorFactory<?> getShipStrategyComparator()

Gets the ship strategy comparator from this Channel.

Returns:

The ship strategy comparator.

setShipStrategyComparator

public void setShipStrategyComparator(TypeComparatorFactory<?> shipStrategyComparator)

Sets the ship strategy comparator for this Channel.

Parameters:

shipStrategyComparator - The ship strategy comparator to set.

getLocalStrategyComparator

public TypeComparatorFactory<?> getLocalStrategyComparator()

Gets the local strategy comparator from this Channel.

Returns:

The local strategy comparator.

setLocalStrategyComparator

public void setLocalStrategyComparator(TypeComparatorFactory<?> localStrategyComparator)

Sets the local strategy comparator for this Channel.

Parameters:

localStrategyComparator - The local strategy comparator to set.

getRelativeMemoryLocalStrategy

public double getRelativeMemoryLocalStrategy()

setRelativeMemoryLocalStrategy

public void setRelativeMemoryLocalStrategy(double relativeMemoryLocalStrategy)

isOnDynamicPath

public boolean isOnDynamicPath()

getCostWeight

public int getCostWeight()

getEstimatedOutputSize

public long getEstimatedOutputSize()

Description copied from interface: EstimateProvider

Gets the estimated output size from this node.

Specified by:

getEstimatedOutputSize in interface EstimateProvider

Returns:

The estimated output size.

getEstimatedNumRecords

public long getEstimatedNumRecords()

Description copied from interface: EstimateProvider

Gets the estimated number of records in the output of this node.

Specified by:

getEstimatedNumRecords in interface EstimateProvider

Returns:

The estimated number of records.

getEstimatedAvgWidthPerOutputRecord

public float getEstimatedAvgWidthPerOutputRecord()

Description copied from interface: EstimateProvider

Gets the estimated number of bytes per record.

Specified by:

getEstimatedAvgWidthPerOutputRecord in interface EstimateProvider

Returns:

The estimated number of bytes per record.

getRequiredGlobalProps

public RequestedGlobalProperties getRequiredGlobalProps()

setRequiredGlobalProps

public void setRequiredGlobalProps(RequestedGlobalProperties requiredGlobalProps)

getRequiredLocalProps

public RequestedLocalProperties getRequiredLocalProps()

setRequiredLocalProps

public void setRequiredLocalProps(RequestedLocalProperties requiredLocalProps)

getGlobalProperties

public GlobalProperties getGlobalProperties()

getLocalProperties

public LocalProperties getLocalProperties()

adjustGlobalPropertiesForFullParallelismChange

public void adjustGlobalPropertiesForFullParallelismChange()

swapUnionNodes

public void swapUnionNodes(PlanNode newUnionNode)

Utility method used while swapping binary union nodes for n-ary union nodes.

getMaxDepth

public int getMaxDepth()

toString

public String toString()

Overrides:

toString in class Object

clone

public Channel clone()

Overrides:

clone in class Object