public abstract class GradientDescent extends IterativeSolver
For each labeled vector in a mini batch the gradient is computed and added to a partial gradient. The partial gradients are then summed and divided by the size of the batches. The average gradient is then used to updated the weight values, including regularization.
At the moment, the whole partition is used for SGD, making it effectively a batch gradient descent. Once a sampling operator has been introduced, the algorithm can be optimized
The parameters to tune the algorithm are:
Solver.LossFunction
for the loss function to be used,
Solver.RegularizationConstant
for the regularization parameter,
IterativeSolver.Iterations
for the maximum number of iteration,
IterativeSolver.LearningRate
for the learning rate used.
IterativeSolver.ConvergenceThreshold
when provided the algorithm will
stop the iterations if the relative change in the value of the objective
function between successive iterations is is smaller than this value.
IterativeSolver.LearningRateMethodValue
determines functional form of
effective learning rate.
IterativeSolver.ConvergenceThreshold$, IterativeSolver.Iterations$, IterativeSolver.LearningRate$, IterativeSolver.LearningRateMethodValue$
Solver.LossFunction$, Solver.RegularizationConstant$
Constructor and Description |
---|
GradientDescent() |
Modifier and Type | Method and Description |
---|---|
DataSet<WeightVector> |
optimize(DataSet<LabeledVector> data,
scala.Option<DataSet<WeightVector>> initialWeights)
Provides a solution for the given optimization problem
|
DataSet<WeightVector> |
optimizeWithConvergenceCriterion(DataSet<LabeledVector> dataPoints,
DataSet<WeightVector> initialWeightsDS,
int numberOfIterations,
double regularizationConstant,
double learningRate,
double convergenceThreshold,
LossFunction lossFunction,
LearningRateMethod.LearningRateMethodTrait learningRateMethod) |
DataSet<WeightVector> |
optimizeWithoutConvergenceCriterion(DataSet<LabeledVector> data,
DataSet<WeightVector> initialWeightsDS,
int numberOfIterations,
double regularizationConstant,
double learningRate,
LossFunction lossFunction,
LearningRateMethod.LearningRateMethodTrait optimizationMethod) |
abstract Vector |
takeStep(Vector weightVector,
Vector gradient,
double regularizationConstant,
double learningRate)
Calculates the new weights based on the gradient
|
MAX_DLOSS, setConvergenceThreshold, setIterations, setLearningRateMethod, setStepsize
createInitialWeightsDS, createInitialWeightVector, setLossFunction, setRegularizationConstant
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
parameters
public DataSet<WeightVector> optimize(DataSet<LabeledVector> data, scala.Option<DataSet<WeightVector>> initialWeights)
public DataSet<WeightVector> optimizeWithConvergenceCriterion(DataSet<LabeledVector> dataPoints, DataSet<WeightVector> initialWeightsDS, int numberOfIterations, double regularizationConstant, double learningRate, double convergenceThreshold, LossFunction lossFunction, LearningRateMethod.LearningRateMethodTrait learningRateMethod)
public DataSet<WeightVector> optimizeWithoutConvergenceCriterion(DataSet<LabeledVector> data, DataSet<WeightVector> initialWeightsDS, int numberOfIterations, double regularizationConstant, double learningRate, LossFunction lossFunction, LearningRateMethod.LearningRateMethodTrait optimizationMethod)
Copyright © 2014–2017 The Apache Software Foundation. All rights reserved.