|
mlpack
master
|
Gradient Descent is a technique to minimize a function. More...
Public Member Functions | |
| GradientDescent (FunctionType &function, const double stepSize=0.01, const size_t maxIterations=100000, const double tolerance=1e-5) | |
| Construct the Gradient Descent optimizer with the given function and parameters. More... | |
| const FunctionType & | Function () const |
| Get the instantiated function to be optimized. More... | |
| FunctionType & | Function () |
| Modify the instantiated function. More... | |
| size_t | MaxIterations () const |
| Get the maximum number of iterations (0 indicates no limit). More... | |
| size_t & | MaxIterations () |
| Modify the maximum number of iterations (0 indicates no limit). More... | |
| double | Optimize (arma::mat &iterate) |
| Optimize the given function using gradient descent. More... | |
| double | StepSize () const |
| Get the step size. More... | |
| double & | StepSize () |
| Modify the step size. More... | |
| double | Tolerance () const |
| Get the tolerance for termination. More... | |
| double & | Tolerance () |
| Modify the tolerance for termination. More... | |
Private Attributes | |
| FunctionType & | function |
| The instantiated function. More... | |
| size_t | maxIterations |
| The maximum number of allowed iterations. More... | |
| double | stepSize |
| The step size for each example. More... | |
| double | tolerance |
| The tolerance for termination. More... | |
Gradient Descent is a technique to minimize a function.
To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient of the function at the current point, producing the following update scheme:
![\[ A_{j + 1} = A_j + \alpha \nabla F(A) \]](form_34.png)
where 
is a parameter which specifies the step size. 
is the function being optimized. The algorithm continues until 
reaches the maximum number of iterations—or when an update produces an improvement within a certain tolerance 
. That is,
![\[ | F(A_{j + 1}) - F(A_j) | < \epsilon. \]](form_39.png)
The parameter 
is specified by the tolerance parameter to the constructor.
For Gradient Descent to work, a FunctionType template parameter is required. This class must implement the following function:
double Evaluate(const arma::mat& coordinates); void Gradient(const arma::mat& coordinates, arma::mat& gradient);
| FunctionType | Decomposable objective function type to be minimized. |
Definition at line 53 of file gradient_descent.hpp.
| mlpack::optimization::GradientDescent< FunctionType >::GradientDescent | ( | FunctionType & | function, |
| const double | stepSize = 0.01, |
||
| const size_t | maxIterations = 100000, |
||
| const double | tolerance = 1e-5 |
||
| ) |
Construct the Gradient Descent optimizer with the given function and parameters.
The defaults here are not necessarily good for the given problem, so it is suggested that the values used be tailored to the task at hand.
| function | Function to be optimized (minimized). |
| stepSize | Step size for each iteration. |
| maxIterations | Maximum number of iterations allowed (0 means no limit). |
| tolerance | Maximum absolute tolerance to terminate algorithm. |
|
inline |
Get the instantiated function to be optimized.
Definition at line 84 of file gradient_descent.hpp.
|
inline |
Modify the instantiated function.
Definition at line 86 of file gradient_descent.hpp.
|
inline |
Get the maximum number of iterations (0 indicates no limit).
Definition at line 94 of file gradient_descent.hpp.
References mlpack::optimization::GradientDescent< FunctionType >::maxIterations.
|
inline |
Modify the maximum number of iterations (0 indicates no limit).
Definition at line 96 of file gradient_descent.hpp.
References mlpack::optimization::GradientDescent< FunctionType >::maxIterations.
| double mlpack::optimization::GradientDescent< FunctionType >::Optimize | ( | arma::mat & | iterate | ) |
Optimize the given function using gradient descent.
The given starting point will be modified to store the finishing point of the algorithm, and the final objective value is returned.
| iterate | Starting point (will be modified). |
|
inline |
Get the step size.
Definition at line 89 of file gradient_descent.hpp.
References mlpack::optimization::GradientDescent< FunctionType >::stepSize.
|
inline |
Modify the step size.
Definition at line 91 of file gradient_descent.hpp.
References mlpack::optimization::GradientDescent< FunctionType >::stepSize.
|
inline |
Get the tolerance for termination.
Definition at line 99 of file gradient_descent.hpp.
References mlpack::optimization::GradientDescent< FunctionType >::tolerance.
|
inline |
Modify the tolerance for termination.
Definition at line 101 of file gradient_descent.hpp.
References mlpack::optimization::GradientDescent< FunctionType >::tolerance.
|
private |
The instantiated function.
Definition at line 105 of file gradient_descent.hpp.
|
private |
The maximum number of allowed iterations.
Definition at line 111 of file gradient_descent.hpp.
Referenced by mlpack::optimization::GradientDescent< FunctionType >::MaxIterations().
|
private |
The step size for each example.
Definition at line 108 of file gradient_descent.hpp.
Referenced by mlpack::optimization::GradientDescent< FunctionType >::StepSize().
|
private |
The tolerance for termination.
Definition at line 114 of file gradient_descent.hpp.
Referenced by mlpack::optimization::GradientDescent< FunctionType >::Tolerance().
1.8.11
