An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom. More...

Public Member Functions
template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>
	LocalCoordinateCoding (const arma::mat &data, const size_t atoms, const double lambda, const size_t maxIterations=0, const double tolerance=0.01, const DictionaryInitializer &initializer=DictionaryInitializer())
	Set the parameters to LocalCoordinateCoding, and train the dictionary. More...

	LocalCoordinateCoding (const size_t atoms=0, const double lambda=0.0, const size_t maxIterations=0, const double tolerance=0.01)
	Set the parameters to LocalCoordinateCoding. More...

size_t	Atoms () const
	Get the number of atoms. More...

size_t &	Atoms ()
	Modify the number of atoms. More...

const arma::mat &	Dictionary () const
	Accessor for dictionary. More...

arma::mat &	Dictionary ()
	Mutator for dictionary. More...

void	Encode (const arma::mat &data, arma::mat &codes)
	Code each point via distance-weighted LARS. More...

double	Lambda () const
	Get the L1 regularization parameter. More...

double &	Lambda ()
	Modify the L1 regularization parameter. More...

size_t	MaxIterations () const
	Get the maximum number of iterations. More...

size_t &	MaxIterations ()
	Modify the maximum number of iterations. More...

double	Objective (const arma::mat &data, const arma::mat &codes, const arma::uvec &adjacencies) const
	Compute objective function given the list of adjacencies. More...

void	OptimizeDictionary (const arma::mat &data, const arma::mat &codes, const arma::uvec &adjacencies)
	Learn dictionary by solving linear system. More...

template<typename Archive >
void	Serialize (Archive &ar, const unsigned int)
	Serialize the model. More...

double	Tolerance () const
	Get the objective tolerance. More...

double &	Tolerance ()
	Modify the objective tolerance. More...

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>
void	Train (const arma::mat &data, const DictionaryInitializer &initializer=DictionaryInitializer())
	Run local coordinate coding. More...

Private Attributes
size_t	atoms
	Number of atoms in dictionary. More...

arma::mat	dictionary
	Dictionary (columns are atoms). More...

double	lambda
	l1 regularization term. More...

size_t	maxIterations
	Maximum number of iterations during training. More...

double	tolerance
	Tolerance for main objective. More...

Detailed Description

An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom.

Let d be the number of dimensions in the original space, m the number of training points, and k the number of atoms in the dictionary (the dimension of the learned feature space). The training data X is a d-by-m matrix where each column is a point and each row is a dimension. The dictionary D is a d-by-k matrix, and the sparse codes matrix Z is a k-by-m matrix. This program seeks to minimize the objective: min_{D,Z} ||X - D Z||_{Fro}^2

lambda sum_{i=1}^m sum_{j=1}^k dist(X_i,D_j)^2 Z_i^j where lambda > 0.

This problem is solved by an algorithm that alternates between a dictionary learning step and a sparse coding step. The dictionary learning step updates the dictionary D by solving a linear system (note that the objective is a positive definite quadratic program). The sparse coding step involves solving a large number of weighted l1-norm regularized linear regression problems problems; this can be done efficiently using LARS, an algorithm that can solve the LASSO (paper below).

The papers are listed below.

@incollection{NIPS2009_0719,
  title = {Nonlinear Learning using Local Coordinate Coding},
  author = {Kai Yu and Tong Zhang and Yihong Gong},
  booktitle = {Advances in Neural Information Processing Systems 22},
  editor = {Y. Bengio and D. Schuurmans and J. Lafferty and C. K. I. Williams
      and A. Culotta},
  pages = {2223--2231},
  year = {2009}
}

@article{efron2004least,
  title={Least angle regression},
  author={Efron, B. and Hastie, T. and Johnstone, I. and Tibshirani, R.},
  journal={The Annals of statistics},
  volume={32},
  number={2},
  pages={407--499},
  year={2004},
  publisher={Institute of Mathematical Statistics}
}

Definition at line 79 of file lcc.hpp.

Constructor & Destructor Documentation

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

mlpack::lcc::LocalCoordinateCoding::LocalCoordinateCoding	(	const arma::mat &	data,
		const size_t	atoms,
		const double	lambda,
		const size_t	maxIterations = `0`,
		const double	tolerance = `0.01`,
		const DictionaryInitializer &	initializer = `DictionaryInitializer()`
	)

Set the parameters to LocalCoordinateCoding, and train the dictionary.

This constructor will also initialize the dictionary using the given DictionaryInitializer before training.

If you want to initialize the dictionary to a custom matrix, consider either writing your own DictionaryInitializer class (with void Initialize(const arma::mat& data, arma::mat& dictionary) function), or call the constructor that does not take a data matrix, then call Dictionary() to set the dictionary matrix to a matrix of your choosing, and then call Train() with sparse_coding::NothingInitializer (i.e. Train<sparse_coding::NothingInitializer>(data)).

Parameters

data	Data matrix.
atoms	Number of atoms in dictionary.
lambda	Regularization parameter for weighted l1-norm penalty.
maxIterations	Maximum number of iterations for training (0 runs until convergence).
tolerance	Tolerance for the objective function.

mlpack::lcc::LocalCoordinateCoding::LocalCoordinateCoding	(	const size_t	atoms = `0`,
		const double	lambda = `0.0`,
		const size_t	maxIterations = `0`,
		const double	tolerance = `0.01`
	)

Set the parameters to LocalCoordinateCoding.

This constructor will not train the model, and a subsequent call to Train() will be required before the model can encode points with Encode(). The default values for atoms and lambda should be changed if you intend to train the model!

Parameters

atoms	Number of atoms in dictionary.
lambda	Regularization parameter for weighted l1-norm penalty.
maxIterations	Maximum number of iterations for training (0 runs until convergence).
tolerance	Tolerance for the objective function.

Member Function Documentation

size_t mlpack::lcc::LocalCoordinateCoding::Atoms ( ) const

inline

Get the number of atoms.

Definition at line 174 of file lcc.hpp.

References atoms.

size_t& mlpack::lcc::LocalCoordinateCoding::Atoms ( )

inline

Modify the number of atoms.

Definition at line 176 of file lcc.hpp.

References atoms.

const arma::mat& mlpack::lcc::LocalCoordinateCoding::Dictionary ( ) const

inline

Accessor for dictionary.

Definition at line 179 of file lcc.hpp.

References dictionary.

arma::mat& mlpack::lcc::LocalCoordinateCoding::Dictionary ( )

inline

Mutator for dictionary.

Definition at line 181 of file lcc.hpp.

References dictionary.

void mlpack::lcc::LocalCoordinateCoding::Encode	(	const arma::mat &	data,
		arma::mat &	codes
	)

Code each point via distance-weighted LARS.

Parameters

data	Matrix containing points to encode.
codes	Output matrix to store codes in.

double mlpack::lcc::LocalCoordinateCoding::Lambda ( ) const

inline

Get the L1 regularization parameter.

Definition at line 184 of file lcc.hpp.

References lambda.

double& mlpack::lcc::LocalCoordinateCoding::Lambda ( )

inline

Modify the L1 regularization parameter.

Definition at line 186 of file lcc.hpp.

References lambda.

size_t mlpack::lcc::LocalCoordinateCoding::MaxIterations ( ) const

inline

Get the maximum number of iterations.

Definition at line 189 of file lcc.hpp.

References maxIterations.

size_t& mlpack::lcc::LocalCoordinateCoding::MaxIterations ( )

inline

Modify the maximum number of iterations.

Definition at line 191 of file lcc.hpp.

References maxIterations.

double mlpack::lcc::LocalCoordinateCoding::Objective	(	const arma::mat &	data,
		const arma::mat &	codes,
		const arma::uvec &	adjacencies
	)		const

Compute objective function given the list of adjacencies.

void mlpack::lcc::LocalCoordinateCoding::OptimizeDictionary	(	const arma::mat &	data,
		const arma::mat &	codes,
		const arma::uvec &	adjacencies
	)

Learn dictionary by solving linear system.

Parameters

adjacencies Indices of entries (unrolled column by column) of the coding matrix Z that are non-zero (the adjacency matrix for the bipartite graph of points and atoms)

template<typename Archive >

void mlpack::lcc::LocalCoordinateCoding::Serialize	(	Archive &	ar,
		const unsigned	int
	)

Serialize the model.

Referenced by Tolerance().

double mlpack::lcc::LocalCoordinateCoding::Tolerance ( ) const

inline

Get the objective tolerance.

Definition at line 194 of file lcc.hpp.

References tolerance.

double& mlpack::lcc::LocalCoordinateCoding::Tolerance ( )

inline

Modify the objective tolerance.

Definition at line 196 of file lcc.hpp.

References Serialize(), and tolerance.

template<typename DictionaryInitializer = sparse_coding::DataDependentRandomInitializer>

void mlpack::lcc::LocalCoordinateCoding::Train	(	const arma::mat &	data,
		const DictionaryInitializer &	initializer = `DictionaryInitializer()`
	)

Run local coordinate coding.

Parameters

nIterations	Maximum number of iterations to run algorithm.
objTolerance	Tolerance of objective function. When the objective function changes by a value lower than this tolerance, the optimization terminates.

Member Data Documentation

size_t mlpack::lcc::LocalCoordinateCoding::atoms

private

Number of atoms in dictionary.

Definition at line 204 of file lcc.hpp.

Referenced by Atoms().

arma::mat mlpack::lcc::LocalCoordinateCoding::dictionary

private

Dictionary (columns are atoms).

Definition at line 207 of file lcc.hpp.

Referenced by Dictionary().

double mlpack::lcc::LocalCoordinateCoding::lambda

private

l1 regularization term.

Definition at line 210 of file lcc.hpp.

Referenced by Lambda().

size_t mlpack::lcc::LocalCoordinateCoding::maxIterations

private

Maximum number of iterations during training.

Definition at line 213 of file lcc.hpp.

Referenced by MaxIterations().

double mlpack::lcc::LocalCoordinateCoding::tolerance

private

Tolerance for main objective.

Definition at line 215 of file lcc.hpp.

Referenced by Tolerance().

The documentation for this class was generated from the following file:

src/mlpack/methods/local_coordinate_coding/lcc.hpp

Public Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation