The DropConnect layer is a regularizer that randomly with probability ratio sets the connection values to zero and scales the remaining elements by factor 1 /(1 - ratio). More...

Public Member Functions
	DropConnect ()
	Create the DropConnect object. More...

	DropConnect (const size_t inSize, const size_t outSize, const double ratio=0.5)
	Creates the DropConnect Layer as a Linear Object that takes input size, output size and ratio as parameter. More...

	~DropConnect ()

template<typename eT >
void	Backward (arma::Mat< eT > &&input, arma::Mat< eT > &&gy, arma::Mat< eT > &&g)
	Ordinary feed backward pass of the DropConnect layer. More...

OutputDataType const &	Delta () const
	Get the delta. More...

OutputDataType &	Delta ()
	Modify the delta. More...

bool	Deterministic () const
	The value of the deterministic parameter. More...

bool &	Deterministic ()
	Modify the value of the deterministic parameter. More...

template<typename eT >
void	Forward (arma::Mat< eT > &&input, arma::Mat< eT > &&output)
	Ordinary feed forward pass of the DropConnect layer. More...

template<typename eT >
void	Gradient (arma::Mat< eT > &&input, arma::Mat< eT > &&error, arma::Mat< eT > &&)
	Calculate the gradient using the output delta and the input activation. More...

OutputDataType const &	Gradient () const
	Get the gradient. More...

OutputDataType &	Gradient ()
	Modify the gradient. More...

InputDataType const &	InputParameter () const
	Get the input parameter. More...

InputDataType &	InputParameter ()
	Modify the input parameter. More...

std::vector< LayerTypes > &	Model ()
	Get the model modules. More...

OutputDataType const &	OutputParameter () const
	Get the output parameter. More...

OutputDataType &	OutputParameter ()
	Modify the output parameter. More...

OutputDataType const &	Parameters () const
	Get the parameters. More...

OutputDataType &	Parameters ()
	Modify the parameters. More...

double	Ratio () const
	The probability of setting a value to zero. More...

void	Ratio (const double r)
	Modify the probability of setting a value to zero. More...

template<typename Archive >
void	Serialize (Archive &ar, const unsigned int)
	Serialize the layer. More...

Private Attributes
LayerTypes	baseLayer
	Locally-stored layer module. More...

OutputDataType	delta
	Locally-stored delta object. More...

OutputDataType	denoise
	Denoise mask for the weights. More...

bool	deterministic
	If true dropout and scaling is disabled, see notes above. More...

OutputDataType	gradient
	Locally-stored gradient object. More...

InputDataType	inputParameter
	Locally-stored input parameter object. More...

OutputDataType	mask
	Locally-stored mask object. More...

std::vector< LayerTypes >	network
	Locally-stored network modules. More...

OutputDataType	outputParameter
	Locally-stored output parameter object. More...

OutputDataType	parameters
	Locally-stored weight object. More...

double	ratio
	The probability of setting a value to zero. More...

double	scale
	The scale fraction. More...

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::DropConnect< InputDataType, OutputDataType >

The DropConnect layer is a regularizer that randomly with probability ratio sets the connection values to zero and scales the remaining elements by factor 1 /(1 - ratio).

The output is scaled with 1 / (1 - p) when deterministic is false. In the deterministic mode(during testing), the layer just computes the output. The output is computed according to the input layer. If no input layer is given, it will take a linear layer as default.

Note: During training you should set deterministic to false and during testing you should set deterministic to true.

For more information, see the following.

@inproceedings{WanICML2013,
  title={Regularization of Neural Networks using DropConnect},
  booktitle = {Proceedings of the 30th International Conference on Machine
               Learning(ICML - 13)},
  author = {Li Wan and Matthew Zeiler and Sixin Zhang and Yann L. Cun and
            Rob Fergus},
  year = {2013}
}

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 62 of file dropconnect.hpp.