public class GradientDescent extends IterativeSolver
For each labeled vector in a mini batch the gradient is computed and added to a partial gradient. The partial gradients are then summed and divided by the size of the batches. The average gradient is then used to updated the weight values, including regularization.
At the moment, the whole partition is used for SGD, making it effectively a batch gradient descent. Once a sampling operator has been introduced, the algorithm can be optimized
The parameters to tune the algorithm are:
Solver.LossFunction
for the loss function to be used,
Solver.RegularizationPenaltyValue
for the regularization penalty.
Solver.RegularizationConstant
for the regularization parameter,
IterativeSolver.Iterations
for the maximum number of iteration,
IterativeSolver.LearningRate
for the learning rate used.
IterativeSolver.ConvergenceThreshold
when provided the algorithm will
stop the iterations if the relative change in the value of the objective
function between successive iterations is is smaller than this value.
IterativeSolver.LearningRateMethodValue
determines functional form of
effective learning rate.
IterativeSolver.ConvergenceThreshold$, IterativeSolver.Iterations$, IterativeSolver.LearningRate$, IterativeSolver.LearningRateMethodValue$
Solver.LossFunction$, Solver.RegularizationConstant$, Solver.RegularizationPenaltyValue$
Constructor and Description |
---|
GradientDescent() |
Modifier and Type | Method and Description |
---|---|
static GradientDescent |
apply() |
static DataSet<WeightVector> |
createInitialWeightsDS(scala.Option<DataSet<WeightVector>> initialWeights,
DataSet<LabeledVector> data) |
static DataSet<WeightVector> |
createInitialWeightVector(DataSet<Object> dimensionDS) |
DataSet<WeightVector> |
optimize(DataSet<LabeledVector> data,
scala.Option<DataSet<WeightVector>> initialWeights)
Provides a solution for the given optimization problem
|
DataSet<WeightVector> |
optimizeWithConvergenceCriterion(DataSet<LabeledVector> dataPoints,
DataSet<WeightVector> initialWeightsDS,
int numberOfIterations,
RegularizationPenalty regularizationPenalty,
double regularizationConstant,
double learningRate,
double convergenceThreshold,
LossFunction lossFunction,
LearningRateMethod.LearningRateMethodTrait learningRateMethod) |
DataSet<WeightVector> |
optimizeWithoutConvergenceCriterion(DataSet<LabeledVector> data,
DataSet<WeightVector> initialWeightsDS,
int numberOfIterations,
RegularizationPenalty regularizationPenalty,
double regularizationConstant,
double learningRate,
LossFunction lossFunction,
LearningRateMethod.LearningRateMethodTrait optimizationMethod) |
static ParameterMap |
parameters() |
static IterativeSolver |
setConvergenceThreshold(double convergenceThreshold) |
static IterativeSolver |
setIterations(int iterations) |
static IterativeSolver |
setLearningRateMethod(LearningRateMethod.LearningRateMethodTrait learningRateMethod) |
static Solver |
setLossFunction(LossFunction lossFunction) |
static Solver |
setRegularizationConstant(double regularizationConstant) |
static Solver |
setRegularizationPenalty(RegularizationPenalty regularizationPenalty) |
static IterativeSolver |
setStepsize(double stepsize) |
Vector |
takeStep(Vector weightVector,
Vector gradient,
RegularizationPenalty regularizationPenalty,
double regularizationConstant,
double learningRate)
Calculates the new weights based on the gradient
|
MAX_DLOSS, setConvergenceThreshold, setIterations, setLearningRateMethod, setStepsize
createInitialWeightsDS, createInitialWeightVector, setLossFunction, setRegularizationConstant, setRegularizationPenalty
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
parameters
public static GradientDescent apply()
public static ParameterMap parameters()
public static DataSet<WeightVector> createInitialWeightsDS(scala.Option<DataSet<WeightVector>> initialWeights, DataSet<LabeledVector> data)
public static DataSet<WeightVector> createInitialWeightVector(DataSet<Object> dimensionDS)
public static Solver setLossFunction(LossFunction lossFunction)
public static Solver setRegularizationConstant(double regularizationConstant)
public static Solver setRegularizationPenalty(RegularizationPenalty regularizationPenalty)
public static IterativeSolver setIterations(int iterations)
public static IterativeSolver setStepsize(double stepsize)
public static IterativeSolver setConvergenceThreshold(double convergenceThreshold)
public static IterativeSolver setLearningRateMethod(LearningRateMethod.LearningRateMethodTrait learningRateMethod)
public DataSet<WeightVector> optimize(DataSet<LabeledVector> data, scala.Option<DataSet<WeightVector>> initialWeights)
optimize
in class IterativeSolver
data
- A Dataset of LabeledVector (label, features) pairsinitialWeights
- The initial weights that will be optimizedpublic DataSet<WeightVector> optimizeWithConvergenceCriterion(DataSet<LabeledVector> dataPoints, DataSet<WeightVector> initialWeightsDS, int numberOfIterations, RegularizationPenalty regularizationPenalty, double regularizationConstant, double learningRate, double convergenceThreshold, LossFunction lossFunction, LearningRateMethod.LearningRateMethodTrait learningRateMethod)
public DataSet<WeightVector> optimizeWithoutConvergenceCriterion(DataSet<LabeledVector> data, DataSet<WeightVector> initialWeightsDS, int numberOfIterations, RegularizationPenalty regularizationPenalty, double regularizationConstant, double learningRate, LossFunction lossFunction, LearningRateMethod.LearningRateMethodTrait optimizationMethod)
public Vector takeStep(Vector weightVector, Vector gradient, RegularizationPenalty regularizationPenalty, double regularizationConstant, double learningRate)
weightVector
- The weights to be updatedgradient
- The gradient according to which we will update the weightsregularizationPenalty
- The regularization penalty to applyregularizationConstant
- The regularization parameterlearningRate
- The effective step size for this iterationCopyright © 2014–2018 The Apache Software Foundation. All rights reserved.