A sparse autoencoder is a neural network whose aim to learn compressed representations of the data, typically for dimensionality reduction, with a constraint on the activity of the neurons in the network. More...

Public Member Functions
	SparseAutoencoder (const arma::mat &data, const size_t visibleSize, const size_t hiddenSize, const double lambda=0.0001, const double beta=3, const double rho=0.01)
	Construct the sparse autoencoder model with the given training data. More...

	SparseAutoencoder (OptimizerType< SparseAutoencoderFunction > &optimizer)
	Construct the sparse autoencoder model with the given training data. More...

void	Beta (const double b)
	Sets the KL divergence parameter. More...

double	Beta () const
	Gets the KL divergence parameter. More...

void	GetNewFeatures (arma::mat &data, arma::mat &features)
	Transforms the provided data into the representation learned by the sparse autoencoder. More...

void	HiddenSize (const size_t hidden)
	Sets size of the hidden layer. More...

size_t	HiddenSize () const
	Gets the size of the hidden layer. More...

void	Lambda (const double l)
	Sets the L2-regularization parameter. More...

double	Lambda () const
	Gets the L2-regularization parameter. More...

void	Rho (const double r)
	Sets the sparsity parameter. More...

double	Rho () const
	Gets the sparsity parameter. More...

void	Sigmoid (const arma::mat &x, arma::mat &output) const
	Returns the elementwise sigmoid of the passed matrix, where the sigmoid function of a real number 'x' is [1 / (1 + exp(-x))]. More...

void	VisibleSize (const size_t visible)
	Sets size of the visible layer. More...

size_t	VisibleSize () const
	Gets size of the visible layer. More...

Private Attributes
double	beta
	KL divergence parameter. More...

size_t	hiddenSize
	Size of the hidden layer. More...

double	lambda
	L2-regularization parameter. More...

arma::mat	parameters
	Parameters after optimization. More...

double	rho
	Sparsity parameter. More...

size_t	visibleSize
	Size of the visible layer. More...

Detailed Description

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>
class mlpack::nn::SparseAutoencoder< OptimizerType >

A sparse autoencoder is a neural network whose aim to learn compressed representations of the data, typically for dimensionality reduction, with a constraint on the activity of the neurons in the network.

Sparse autoencoders can be stacked together to learn a hierarchy of features, which provide a better representation of the data for classification. This is a method used in the recently developed field of deep learning. More technical details about the model can be found on the following webpage:

http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Tutorial

An example of how to use the interface is shown below:

arma::mat data; // Data matrix.
const size_t vSize = 64; // Size of visible layer, depends on the data.
const size_t hSize = 25; // Size of hidden layer, depends on requirements.
// Train the model using default options.
SparseAutoencoder encoder1(data, vSize, hSize);
const size_t numBasis = 5; // Parameter required for L-BFGS algorithm.
const size_t numIterations = 100; // Maximum number of iterations.
// Use an instantiated optimizer for the training.
SparseAutoencoderFunction saf(data, vSize, hSize);
L_BFGS<SparseAutoencoderFunction> optimizer(saf, numBasis, numIterations);
SparseAutoencoder<L_BFGS> encoder2(optimizer);
arma::mat features1, features2; // Matrices for storing new representations.
// Get new representations from the trained models.
encoder1.GetNewFeatures(data, features1);
encoder2.GetNewFeatures(data, features2);

This implementation allows the use of arbitrary mlpack optimizers via the OptimizerType template parameter.

Template Parameters

OptimizerType The optimizer to use; by default this is L-BFGS. Any mlpack optimizer can be used here.

Definition at line 68 of file sparse_autoencoder.hpp.

Constructor & Destructor Documentation

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

mlpack::nn::SparseAutoencoder< OptimizerType >::SparseAutoencoder	(	const arma::mat &	data,
		const size_t	visibleSize,
		const size_t	hiddenSize,
		const double	lambda = `0.0001`,
		const double	beta = `3`,
		const double	rho = `0.01`
	)

Construct the sparse autoencoder model with the given training data.

This will train the model. The parameters 'lambda', 'beta' and 'rho' can be set optionally. Changing these parameters will have an effect on regularization and sparsity of the model.

Parameters

data	Input data with each column as one example.
visibleSize	Size of input vector expected at the visible layer.
hiddenSize	Size of input vector expected at the hidden layer.
lambda	L2-regularization parameter.
beta	KL divergence parameter.
rho	Sparsity parameter.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

mlpack::nn::SparseAutoencoder< OptimizerType >::SparseAutoencoder ( OptimizerType< SparseAutoencoderFunction > & optimizer )

Construct the sparse autoencoder model with the given training data.

This will train the model. This overload takes an already instantiated optimizer and uses it to train the model. The optimizer should hold an instantiated SparseAutoencoderFunction object for the function to operate upon. This option should be preferred when the optimizer options are to be changed.

Parameters

optimizer Instantiated optimizer with instantiated error function.

Member Function Documentation

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

void mlpack::nn::SparseAutoencoder< OptimizerType >::Beta ( const double b )

inline

Sets the KL divergence parameter.

Definition at line 160 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::beta.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

double mlpack::nn::SparseAutoencoder< OptimizerType >::Beta ( ) const

inline

Gets the KL divergence parameter.

Definition at line 166 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::beta.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

void mlpack::nn::SparseAutoencoder< OptimizerType >::GetNewFeatures	(	arma::mat &	data,
		arma::mat &	features
	)

Transforms the provided data into the representation learned by the sparse autoencoder.

The function basically performs a feedforward computation using the learned weights, and returns the hidden layer activations.

Parameters

data	Matrix of the provided data.
features	The hidden layer representation of the provided data.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

void mlpack::nn::SparseAutoencoder< OptimizerType >::HiddenSize ( const size_t hidden )

inline

Sets size of the hidden layer.

Definition at line 136 of file sparse_autoencoder.hpp.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

size_t mlpack::nn::SparseAutoencoder< OptimizerType >::HiddenSize ( ) const

inline

Gets the size of the hidden layer.

Definition at line 142 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::hiddenSize.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

void mlpack::nn::SparseAutoencoder< OptimizerType >::Lambda ( const double l )

inline

Sets the L2-regularization parameter.

Definition at line 148 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::lambda.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

double mlpack::nn::SparseAutoencoder< OptimizerType >::Lambda ( ) const

inline

Gets the L2-regularization parameter.

Definition at line 154 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::lambda.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

void mlpack::nn::SparseAutoencoder< OptimizerType >::Rho ( const double r )

inline

Sets the sparsity parameter.

Definition at line 172 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::rho.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

double mlpack::nn::SparseAutoencoder< OptimizerType >::Rho ( ) const

inline

Gets the sparsity parameter.

Definition at line 178 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::rho.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

void mlpack::nn::SparseAutoencoder< OptimizerType >::Sigmoid	(	const arma::mat &	x,
		arma::mat &	output
	)		const

inline

Returns the elementwise sigmoid of the passed matrix, where the sigmoid function of a real number 'x' is [1 / (1 + exp(-x))].

Parameters

x	Matrix of real values for which we require the sigmoid activation.

Definition at line 118 of file sparse_autoencoder.hpp.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

void mlpack::nn::SparseAutoencoder< OptimizerType >::VisibleSize ( const size_t visible )

inline

Sets size of the visible layer.

Definition at line 124 of file sparse_autoencoder.hpp.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

size_t mlpack::nn::SparseAutoencoder< OptimizerType >::VisibleSize ( ) const

inline

Gets size of the visible layer.

Definition at line 130 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::visibleSize.

Member Data Documentation

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

double mlpack::nn::SparseAutoencoder< OptimizerType >::beta

private

KL divergence parameter.

Definition at line 193 of file sparse_autoencoder.hpp.

Referenced by mlpack::nn::SparseAutoencoder< OptimizerType >::Beta().

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

size_t mlpack::nn::SparseAutoencoder< OptimizerType >::hiddenSize

private

Size of the hidden layer.

Definition at line 189 of file sparse_autoencoder.hpp.

Referenced by mlpack::nn::SparseAutoencoder< OptimizerType >::HiddenSize().

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

double mlpack::nn::SparseAutoencoder< OptimizerType >::lambda

private

L2-regularization parameter.

Definition at line 191 of file sparse_autoencoder.hpp.

Referenced by mlpack::nn::SparseAutoencoder< OptimizerType >::Lambda().

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

arma::mat mlpack::nn::SparseAutoencoder< OptimizerType >::parameters

private

Parameters after optimization.

Definition at line 185 of file sparse_autoencoder.hpp.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

double mlpack::nn::SparseAutoencoder< OptimizerType >::rho

private

Sparsity parameter.

Definition at line 195 of file sparse_autoencoder.hpp.

Referenced by mlpack::nn::SparseAutoencoder< OptimizerType >::Rho().

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>

size_t mlpack::nn::SparseAutoencoder< OptimizerType >::visibleSize

private

Size of the visible layer.

Definition at line 187 of file sparse_autoencoder.hpp.

Referenced by mlpack::nn::SparseAutoencoder< OptimizerType >::VisibleSize().

The documentation for this class was generated from the following file:

src/mlpack/methods/sparse_autoencoder/sparse_autoencoder.hpp

Public Member Functions

Private Attributes

Detailed Description

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> class mlpack::nn::SparseAutoencoder< OptimizerType >

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>
class mlpack::nn::SparseAutoencoder< OptimizerType >