This is a class for the sparse autoencoder objective function. More...

Public Member Functions
	SparseAutoencoderFunction (const arma::mat &data, const size_t visibleSize, const size_t hiddenSize, const double lambda=0.0001, const double beta=3, const double rho=0.01)
	Construct the sparse autoencoder objective function with the given parameters. More...

void	Beta (const double b)
	Sets the KL divergence parameter. More...

double	Beta () const
	Gets the KL divergence parameter. More...

double	Evaluate (const arma::mat &parameters) const
	Evaluates the objective function of the sparse autoencoder model using the given parameters. More...

const arma::mat &	GetInitialPoint () const
	Return the initial point for the optimization. More...

void	Gradient (const arma::mat &parameters, arma::mat &gradient) const
	Evaluates the gradient values of the objective function given the current set of parameters. More...

void	HiddenSize (const size_t hidden)
	Sets size of the hidden layer. More...

size_t	HiddenSize () const
	Gets the size of the hidden layer. More...

const arma::mat	InitializeWeights ()
	Initializes the parameters of the model to suitable values. More...

void	Lambda (const double l)
	Sets the L2-regularization parameter. More...

double	Lambda () const
	Gets the L2-regularization parameter. More...

void	Rho (const double r)
	Sets the sparsity parameter. More...

double	Rho () const
	Gets the sparsity parameter. More...

void	Sigmoid (const arma::mat &x, arma::mat &output) const
	Returns the elementwise sigmoid of the passed matrix, where the sigmoid function of a real number 'x' is [1 / (1 + exp(-x))]. More...

void	VisibleSize (const size_t visible)
	Sets size of the visible layer. More...

size_t	VisibleSize () const
	Gets size of the visible layer. More...

Private Attributes
double	beta
	KL divergence parameter. More...

const arma::mat &	data
	The matrix of data points. More...

size_t	hiddenSize
	Size of the hidden layer. More...

arma::mat	initialPoint
	Initial parameter vector. More...

double	lambda
	L2-regularization parameter. More...

double	rho
	Sparsity parameter. More...

size_t	visibleSize
	Size of the visible layer. More...

Detailed Description

This is a class for the sparse autoencoder objective function.

It can be used to create learning models like self-taught learning, stacked autoencoders, conditional random fields (CRFs), and so forth.

Definition at line 26 of file sparse_autoencoder_function.hpp.

Constructor & Destructor Documentation

mlpack::nn::SparseAutoencoderFunction::SparseAutoencoderFunction	(	const arma::mat &	data,
		const size_t	visibleSize,
		const size_t	hiddenSize,
		const double	lambda = `0.0001`,
		const double	beta = `3`,
		const double	rho = `0.01`
	)

Construct the sparse autoencoder objective function with the given parameters.

Parameters

data	The data matrix.
visibleSize	Size of input vector expected at the visible layer.
hiddenSize	Size of input vector expected at the hidden layer.
lambda	L2-regularization parameter.
beta	KL divergence parameter.
rho	Sparsity parameter.

Member Function Documentation

void mlpack::nn::SparseAutoencoderFunction::Beta ( const double b )

inline

Sets the KL divergence parameter.

Definition at line 124 of file sparse_autoencoder_function.hpp.

References beta.

double mlpack::nn::SparseAutoencoderFunction::Beta ( ) const

inline

Gets the KL divergence parameter.

Definition at line 130 of file sparse_autoencoder_function.hpp.

References beta.

double mlpack::nn::SparseAutoencoderFunction::Evaluate ( const arma::mat & parameters ) const

Evaluates the objective function of the sparse autoencoder model using the given parameters.

The cost function has terms for the reconstruction error, regularization cost and the sparsity cost. The objective function takes a low value when the model is able to reconstruct the data well using weights which are low in value and when the average activations of neurons in the hidden layers agrees well with the sparsity parameter 'rho'.

Parameters

parameters Current values of the model parameters.

const arma::mat& mlpack::nn::SparseAutoencoderFunction::GetInitialPoint ( ) const

inline

Return the initial point for the optimization.

Definition at line 85 of file sparse_autoencoder_function.hpp.

References initialPoint.

void mlpack::nn::SparseAutoencoderFunction::Gradient	(	const arma::mat &	parameters,
		arma::mat &	gradient
	)		const

Evaluates the gradient values of the objective function given the current set of parameters.

The function performs a feedforward pass and computes the error in reconstructing the data points. It then uses the backpropagation algorithm to compute the gradient values.

Parameters