Package org.apache.flink.autoscaler

Class JobVertexScaler<KEY,Context extends JobAutoScalerContext<KEY>>

java.lang.Object

org.apache.flink.autoscaler.JobVertexScaler<KEY,Context>

public class JobVertexScaler<KEY,Context extends JobAutoScalerContext<KEY>>
extends java.lang.Object

Component responsible for computing vertex parallelism based on the scaling metrics.

Field Summary

Fields

Modifier and Type	Field	Description
protected static java.lang.String	INEFFECTIVE_MESSAGE_FORMAT
protected static java.lang.String	INEFFECTIVE_SCALING

Constructor Summary

Constructors

Constructor	Description
JobVertexScaler(AutoScalerEventHandler<KEY,Context> autoScalerEventHandler)

Method Summary

Modifier and Type	Method	Description
int	computeScaleTargetParallelism(Context context, org.apache.flink.runtime.jobgraph.JobVertexID vertex, java.util.Collection<ShipStrategy> inputShipStrategies, java.util.Map<ScalingMetric,EvaluatedScalingMetric> evaluatedMetrics, java.util.SortedMap<java.time.Instant,ScalingSummary> history, java.time.Duration restartTime)
protected static int	scale(int currentParallelism, java.util.Collection<ShipStrategy> inputShipStrategies, int maxParallelism, double scaleFactor, int parallelismLowerLimit, int parallelismUpperLimit)	Computing the newParallelism.
protected void	setClock(java.time.Clock clock)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

INEFFECTIVE_SCALING

@VisibleForTesting
protected static final java.lang.String INEFFECTIVE_SCALING

See Also:

Constant Field Values

INEFFECTIVE_MESSAGE_FORMAT

@VisibleForTesting
protected static final java.lang.String INEFFECTIVE_MESSAGE_FORMAT

See Also:

Constant Field Values

Constructor Detail

JobVertexScaler

public JobVertexScaler(AutoScalerEventHandler<KEY,Context> autoScalerEventHandler)

Method Detail

computeScaleTargetParallelism

public int computeScaleTargetParallelism(Context context,
                                         org.apache.flink.runtime.jobgraph.JobVertexID vertex,
                                         java.util.Collection<ShipStrategy> inputShipStrategies,
                                         java.util.Map<ScalingMetric,EvaluatedScalingMetric> evaluatedMetrics,
                                         java.util.SortedMap<java.time.Instant,ScalingSummary> history,
                                         java.time.Duration restartTime)

scale

@VisibleForTesting
protected static int scale(int currentParallelism,
                           java.util.Collection<ShipStrategy> inputShipStrategies,
                           int maxParallelism,
                           double scaleFactor,
                           int parallelismLowerLimit,
                           int parallelismUpperLimit)

Computing the newParallelism. In general, newParallelism = currentParallelism * scaleFactor. But we limit newParallelism between parallelismLowerLimit and min(parallelismUpperLimit, maxParallelism).

Also, in order to ensure the data is evenly spread across subtasks, we try to adjust the parallelism for source and keyed vertex such that it divides the maxParallelism without a remainder.

setClock

@VisibleForTesting
protected void setClock(java.time.Clock clock)