Implementation of a standard feed forward network. More...

Public Types
using	NetworkType = FFN< OutputLayerType, InitializationRuleType >
	Convenience typedef for the internal model construction. More...

Public Member Functions
	FFN (OutputLayerType &&outputLayer=OutputLayerType(), InitializationRuleType initializeRule=InitializationRuleType())
	Create the FFN object with the given predictors and responses set (this is the set that is used to train the network) and the given optimizer. More...

	FFN (const arma::mat &predictors, const arma::mat &responses, OutputLayerType &&outputLayer=OutputLayerType(), InitializationRuleType initializeRule=InitializationRuleType())
	Create the FFN object with the given predictors and responses set (this is the set that is used to train the network) and the given optimizer. More...

	~FFN ()
	Destructor to release allocated memory. More...

template<class LayerType , class... Args>
void	Add (Args...args)

void	Add (LayerTypes layer)

double	Evaluate (const arma::mat &parameters, const size_t i, const bool deterministic=true)
	Evaluate the feedforward network with the given parameters. More...

void	Gradient (const arma::mat &parameters, const size_t i, arma::mat &gradient)
	Evaluate the gradient of the feedforward network with the given parameters, and with respect to only one point in the dataset. More...

size_t	NumFunctions () const
	Return the number of separable functions (the number of predictor points). More...

const arma::mat &	Parameters () const
	Return the initial point for the optimization. More...

arma::mat &	Parameters ()
	Modify the initial point for the optimization. More...

void	Predict (arma::mat &predictors, arma::mat &responses)
	Predict the responses to a given set of predictors. More...

template<typename Archive >
void	Serialize (Archive &ar, const unsigned int)
	Serialize the model. More...

template<template< typename > class OptimizerType = mlpack::optimization::RMSprop>
void	Train (const arma::mat &predictors, const arma::mat &responses, OptimizerType< NetworkType > &optimizer)
	Train the feedforward network on the given input data using the given optimizer. More...

template<template< typename > class OptimizerType = mlpack::optimization::RMSprop>
void	Train (const arma::mat &predictors, const arma::mat &responses)
	Train the feedforward network on the given input data. More...

Private Member Functions
void	Backward ()
	The Backward algorithm (part of the Forward-Backward algorithm). More...

void	Forward (arma::mat &&input)
	The Forward algorithm (part of the Forward-Backward algorithm). More...

void	Gradient ()
	Iterate through all layer modules and update the the gradient using the layer defined optimizer. More...

void	ResetDeterministic ()
	Reset the module status by setting the current deterministic parameter for all modules that implement the Deterministic function. More...

void	ResetGradients (arma::mat &gradient)
	Reset the gradient for all modules that implement the Gradient function. More...

void	ResetParameters ()
	Reset the module infomration (weights/parameters). More...

Private Attributes
arma::mat	currentInput
	THe current input of the forward/backward pass. More...

arma::mat	currentTarget
	THe current target of the forward/backward pass. More...

DeleteVisitor	deleteVisitor
	Locally-stored delete visitor. More...

arma::mat	delta
	Locally-stored delta object. More...

DeltaVisitor	deltaVisitor
	Locally-stored delta visitor. More...

bool	deterministic
	The current evaluation mode (training or testing). More...

arma::mat	error
	The current error for the backward pass. More...

arma::mat	gradient
	Locally-stored gradient parameter. More...

size_t	height
	The input height. More...

InitializationRuleType	initializeRule
	Instantiated InitializationRule object for initializing the network parameter. More...

arma::mat	inputParameter
	Locally-stored input parameter object. More...

std::vector< LayerTypes >	network
	Locally-stored model modules. More...

size_t	numFunctions
	The number of separable functions (the number of predictor points). More...

OutputHeightVisitor	outputHeightVisitor
	Locally-stored output height visitor. More...

OutputLayerType	outputLayer
	Instantiated outputlayer used to evaluate the network. More...

arma::mat	outputParameter
	Locally-stored output parameter object. More...

OutputParameterVisitor	outputParameterVisitor
	Locally-stored output parameter visitor. More...

OutputWidthVisitor	outputWidthVisitor
	Locally-stored output width visitor. More...

arma::mat	parameter
	Matrix of (trained) parameters. More...

arma::mat	predictors
	The matrix of data points (predictors). More...

bool	reset
	Indicator if we already trained the model. More...

ResetVisitor	resetVisitor
	Locally-stored reset visitor. More...

arma::mat	responses
	The matrix of responses to the input data points. More...

WeightSizeVisitor	weightSizeVisitor
	Locally-stored weight size visitor. More...

size_t	width
	The input width. More...

Detailed Description

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>
class mlpack::ann::FFN< OutputLayerType, InitializationRuleType >

Implementation of a standard feed forward network.

Template Parameters

OutputLayerType	The output layer type used to evaluate the network.
InitializationRuleType	Rule used to initialize the weight matrix.

Definition at line 43 of file ffn.hpp.

Member Typedef Documentation

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

using mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::NetworkType = FFN<OutputLayerType, InitializationRuleType>

Convenience typedef for the internal model construction.

Definition at line 47 of file ffn.hpp.

Constructor & Destructor Documentation

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::FFN	(	OutputLayerType &&	outputLayer = `OutputLayerType()`,
		InitializationRuleType	initializeRule = `InitializationRuleType()`
	)

Create the FFN object with the given predictors and responses set (this is the set that is used to train the network) and the given optimizer.

Optionally, specify which initialize rule and performance function should be used.

Parameters

outputLayer	Output layer used to evaluate the network.
initializeRule	Optional instantiated InitializationRule object for initializing the network parameter.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::FFN	(	const arma::mat &	predictors,
		const arma::mat &	responses,
		OutputLayerType &&	outputLayer = `OutputLayerType()`,
		InitializationRuleType	initializeRule = `InitializationRuleType()`
	)

Create the FFN object with the given predictors and responses set (this is the set that is used to train the network) and the given optimizer.

Optionally, specify which initialize rule and performance function should be used.

Parameters

predictors	Input training variables.
responses	Outputs results from input training variables.
outputLayer	Output layer used to evaluate the network.
initializeRule	Optional instantiated InitializationRule object for initializing the network parameter.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::~FFN ( )

Destructor to release allocated memory.

Member Function Documentation

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

template<class LayerType , class... Args>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Add ( Args... args )

inline

Definition at line 162 of file ffn.hpp.

References mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::network.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Add ( LayerTypes layer )

inline

Definition at line 169 of file ffn.hpp.

References mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::network.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Backward ( )

private

The Backward algorithm (part of the Forward-Backward algorithm).

Computes backward pass for module.

Referenced by mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Parameters().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

double mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Evaluate	(	const arma::mat &	parameters,
		const size_t	i,
		const bool	deterministic = `true`
	)

Evaluate the feedforward network with the given parameters.

This function is usually called by the optimizer to train the model.

Parameters

parameters	Matrix model parameters.
i	Index of point to use for objective function evaluation.
deterministic	Whether or not to train or test the model. Note some layer act differently in training or testing mode.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Forward ( arma::mat && input )

private

The Forward algorithm (part of the Forward-Backward algorithm).

Computes forward probabilities for each module.

Parameters

input Data sequence to compute probabilities for.

Referenced by mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Parameters().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Gradient	(	const arma::mat &	parameters,
		const size_t	i,
		arma::mat &	gradient
	)

Evaluate the gradient of the feedforward network with the given parameters, and with respect to only one point in the dataset.

This is useful for optimizers such as SGD, which require a separable objective function.

Parameters

parameters	Matrix of the model parameters to be optimized.
i	Index of points to use for objective function gradient evaluation.
gradient	Matrix to output gradient into.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Gradient ( )

private

Iterate through all layer modules and update the the gradient using the layer defined optimizer.

Referenced by mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Parameters().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

size_t mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::NumFunctions ( ) const

inline

Return the number of separable functions (the number of predictor points).

Definition at line 172 of file ffn.hpp.

References mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::numFunctions.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

const arma::mat& mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Parameters ( ) const

inline

Return the initial point for the optimization.

Definition at line 175 of file ffn.hpp.

References mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::parameter.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

arma::mat& mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Parameters ( )

inline

Modify the initial point for the optimization.

Definition at line 177 of file ffn.hpp.

References mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Backward(), mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Forward(), mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Gradient(), mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::gradient, mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::parameter, mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::ResetDeterministic(), mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::ResetGradients(), mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::ResetParameters(), and mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Serialize().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Predict	(	arma::mat &	predictors,
		arma::mat &	responses
	)

Predict the responses to a given set of predictors.

The responses will reflect the output of the given output layer as returned by the output layer function.

Parameters

predictors	Input predictors.
responses	Matrix to put output predictions of responses into.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::ResetDeterministic ( )

private

Reset the module status by setting the current deterministic parameter for all modules that implement the Deterministic function.

Referenced by mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Parameters().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::ResetGradients ( arma::mat & gradient )

private

Reset the gradient for all modules that implement the Gradient function.

Referenced by mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Parameters().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::ResetParameters ( )

private

Reset the module infomration (weights/parameters).

Referenced by mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Parameters().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

template<typename Archive >

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Serialize	(	Archive &	ar,
		const unsigned	int
	)

Serialize the model.

Referenced by mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Parameters().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

template<template< typename > class OptimizerType = mlpack::optimization::RMSprop>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Train	(	const arma::mat &	predictors,
		const arma::mat &	responses,
		OptimizerType< NetworkType > &	optimizer
	)

Train the feedforward network on the given input data using the given optimizer.

This will use the existing model parameters as a starting point for the optimization. If this is not what you want, then you should access the parameters vector directly with Parameters() and modify it as desired.

Template Parameters

OptimizerType Type of optimizer to use to train the model.

Parameters

predictors	Input training variables.
responses	Outputs results from input training variables.
optimizer	Instantiated optimizer used to train the model.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

template<template< typename > class OptimizerType = mlpack::optimization::RMSprop>

void mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Train	(	const arma::mat &	predictors,
		const arma::mat &	responses
	)

Train the feedforward network on the given input data.

By default, the RMSprop optimization algorithm is used, but others can be specified (such as mlpack::optimization::SGD).

This will use the existing model parameters as a starting point for the optimization. If this is not what you want, then you should access the parameters vector directly with Parameters() and modify it as desired.

Template Parameters

OptimizerType Type of optimizer to use to train the model.

Parameters

predictors	Input training variables.
responses	Outputs results from input training variables.

Member Data Documentation

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

arma::mat mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::currentInput

private

THe current input of the forward/backward pass.

Definition at line 256 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

arma::mat mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::currentTarget

private

THe current target of the forward/backward pass.

Definition at line 259 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

DeleteVisitor mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::deleteVisitor

private

Locally-stored delete visitor.

Definition at line 280 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

arma::mat mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::delta

private

Locally-stored delta object.

Definition at line 286 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

DeltaVisitor mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::deltaVisitor

private

Locally-stored delta visitor.

Definition at line 262 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

bool mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::deterministic

private

The current evaluation mode (training or testing).

Definition at line 283 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

arma::mat mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::error

private

The current error for the backward pass.

Definition at line 253 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

arma::mat mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::gradient

private

Locally-stored gradient parameter.

Definition at line 295 of file ffn.hpp.

Referenced by mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Parameters().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

size_t mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::height

private

The input height.

Definition at line 232 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

InitializationRuleType mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::initializeRule

private

Instantiated InitializationRule object for initializing the network parameter.

Definition at line 226 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

arma::mat mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::inputParameter

private

Locally-stored input parameter object.

Definition at line 289 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

std::vector<LayerTypes> mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::network

private

Locally-stored model modules.

Definition at line 238 of file ffn.hpp.

Referenced by mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Add().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

size_t mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::numFunctions

private

The number of separable functions (the number of predictor points).

Definition at line 250 of file ffn.hpp.

Referenced by mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::NumFunctions().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

OutputHeightVisitor mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::outputHeightVisitor

private

Locally-stored output height visitor.

Definition at line 274 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

OutputLayerType mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::outputLayer

private

Instantiated outputlayer used to evaluate the network.

Definition at line 222 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

arma::mat mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::outputParameter

private

Locally-stored output parameter object.

Definition at line 292 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

OutputParameterVisitor mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::outputParameterVisitor

private

Locally-stored output parameter visitor.

Definition at line 265 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

OutputWidthVisitor mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::outputWidthVisitor

private

Locally-stored output width visitor.

Definition at line 271 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

arma::mat mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::parameter

private

Matrix of (trained) parameters.

Definition at line 247 of file ffn.hpp.

Referenced by mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::Parameters().

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

arma::mat mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::predictors

private

The matrix of data points (predictors).

Definition at line 241 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

bool mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::reset

private

Indicator if we already trained the model.

Definition at line 235 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

ResetVisitor mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::resetVisitor

private

Locally-stored reset visitor.

Definition at line 277 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

arma::mat mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::responses

private

The matrix of responses to the input data points.

Definition at line 244 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

WeightSizeVisitor mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::weightSizeVisitor

private

Locally-stored weight size visitor.

Definition at line 268 of file ffn.hpp.

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>

size_t mlpack::ann::FFN< OutputLayerType, InitializationRuleType >::width

private

The input width.

Definition at line 229 of file ffn.hpp.

The documentation for this class was generated from the following file:

src/mlpack/methods/ann/ffn.hpp

Public Types

Public Member Functions

Private Member Functions

Private Attributes

Detailed Description

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization> class mlpack::ann::FFN< OutputLayerType, InitializationRuleType >

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization>
class mlpack::ann::FFN< OutputLayerType, InitializationRuleType >