The PReLU activation function, defined by (where alpha is trainable) More...

Public Member Functions
	PReLU (const double user_alpha=0.03)
	Create the PReLU object using the specified parameters. More...

double const &	Alpha () const
	Get the non zero gradient. More...

double &	Alpha ()
	Modify the non zero gradient. More...

template<typename DataType >
void	Backward (const DataType &&input, DataType &&gy, DataType &&g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f. More...

OutputDataType const &	Delta () const
	Get the delta. More...

OutputDataType &	Delta ()
	Modify the delta. More...

template<typename InputType , typename OutputType >
void	Forward (const InputType &&input, OutputType &&output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

template<typename eT >
void	Gradient (const arma::Mat< eT > &&input, arma::Mat< eT > &&error, arma::Mat< eT > &&gradient)
	Calculate the gradient using the output delta and the input activation. More...

OutputDataType const &	Gradient () const
	Get the gradient. More...

OutputDataType &	Gradient ()
	Modify the gradient. More...

InputDataType const &	InputParameter () const
	Get the input parameter. More...

InputDataType &	InputParameter ()
	Modify the input parameter. More...

OutputDataType const &	OutputParameter () const
	Get the output parameter. More...

OutputDataType &	OutputParameter ()
	Modify the output parameter. More...

OutputDataType const &	Parameters () const
	Get the parameters. More...

OutputDataType &	Parameters ()
	Modify the parameters. More...

void	Reset ()

template<typename Archive >
void	Serialize (Archive &ar, const unsigned int)
	Serialize the layer. More...

Private Member Functions
double	Deriv (const double x)
	Computes the first derivative of the parametric ReLU function. More...

template<typename InputType , typename OutputType >
void	Deriv (const InputType &x, OutputType &y)
	Computes the first derivative of the PReLU function. More...

double	Fn (const double x)
	Computes the parametric ReLU function. More...

template<typename eT >
void	Fn (const arma::Mat< eT > &x, arma::Mat< eT > &y)
	Computes the parametric ReLU function using a dense matrix as input. More...

Private Attributes
OutputDataType	alpha
	Leakyness Parameter object. More...

OutputDataType	delta
	Locally-stored delta object. More...

OutputDataType	gradient
	Locally-stored gradient object. More...

InputDataType	inputParameter
	Locally-stored input parameter object. More...

OutputDataType	outputParameter
	Locally-stored output parameter object. More...

double	user_alpha
	Leakyness Parameter given by user in the range 0 < alpha < 1. More...

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::PReLU< InputDataType, OutputDataType >

The PReLU activation function, defined by (where alpha is trainable)

$\begin{eqnarray*} f(x) &=& \max(x, alpha*x) \\ f'(x) &=& \left\{ \begin{array}{lr} 1 & : x > 0 \\ alpha & : x \le 0 \end{array} \right. \end{eqnarray*}$

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 45 of file parametric_relu.hpp.

Constructor & Destructor Documentation

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

mlpack::ann::PReLU< InputDataType, OutputDataType >::PReLU ( const double user_alpha = 0.03 )

Create the PReLU object using the specified parameters.

The non zero gradient can be adjusted by specifying tha parameter alpha in the range 0 to 1. Default (alpha = 0.03). This parameter is trainable.

Parameters

alpha Non zero gradient

Member Function Documentation

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

double const& mlpack::ann::PReLU< InputDataType, OutputDataType >::Alpha ( ) const

inline

Get the non zero gradient.

Definition at line 123 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::alpha.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

double& mlpack::ann::PReLU< InputDataType, OutputDataType >::Alpha ( )

inline

Modify the non zero gradient.

Definition at line 125 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::alpha, and mlpack::ann::PReLU< InputDataType, OutputDataType >::Serialize().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

template<typename DataType >

void mlpack::ann::PReLU< InputDataType, OutputDataType >::Backward	(	const DataType &&	input,
		DataType &&	gy,
		DataType &&	g
	)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f.

Using the results from the feed forward pass.

Parameters

input	The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType const& mlpack::ann::PReLU< InputDataType, OutputDataType >::Delta ( ) const

inline

Get the delta.

Definition at line 113 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::delta.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType& mlpack::ann::PReLU< InputDataType, OutputDataType >::Delta ( )

inline

Modify the delta.

Definition at line 115 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::delta.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

double mlpack::ann::PReLU< InputDataType, OutputDataType >::Deriv ( const double x )

inlineprivate

Computes the first derivative of the parametric ReLU function.

Parameters

x	Input data.

Returns: f'(x)

Definition at line 165 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::alpha.

Referenced by mlpack::ann::PReLU< InputDataType, OutputDataType >::Deriv().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

template<typename InputType , typename OutputType >

void mlpack::ann::PReLU< InputDataType, OutputDataType >::Deriv	(	const InputType &	x,
		OutputType &	y
	)

inlineprivate

Computes the first derivative of the PReLU function.

Parameters

y	Input activations.
x	The resulting derivatives.

Definition at line 178 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::Deriv().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

double mlpack::ann::PReLU< InputDataType, OutputDataType >::Fn ( const double x )

inlineprivate

Computes the parametric ReLU function.

Parameters

x	Input data.

Returns: f(x).

Definition at line 140 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::alpha.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

template<typename eT >

void mlpack::ann::PReLU< InputDataType, OutputDataType >::Fn	(	const arma::Mat< eT > &	x,
		arma::Mat< eT > &	y
	)

inlineprivate

Computes the parametric ReLU function using a dense matrix as input.

Parameters

x	Input data.
y	The resulting output activation.

Definition at line 152 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::alpha.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

template<typename InputType , typename OutputType >

void mlpack::ann::PReLU< InputDataType, OutputDataType >::Forward	(	const InputType &&	input,
		OutputType &&	output
	)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

template<typename eT >

void mlpack::ann::PReLU< InputDataType, OutputDataType >::Gradient	(	const arma::Mat< eT > &&	input,
		arma::Mat< eT > &&	error,
		arma::Mat< eT > &&	gradient
	)

Calculate the gradient using the output delta and the input activation.

Parameters

input	The input parameter used for calculating the gradient.
error	The calculated error.
gradient	The calculated gradient.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType const& mlpack::ann::PReLU< InputDataType, OutputDataType >::Gradient ( ) const

inline

Get the gradient.

Definition at line 118 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::gradient.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType& mlpack::ann::PReLU< InputDataType, OutputDataType >::Gradient ( )

inline

Modify the gradient.

Definition at line 120 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::gradient.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

InputDataType const& mlpack::ann::PReLU< InputDataType, OutputDataType >::InputParameter ( ) const

inline

Get the input parameter.

Definition at line 103 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::inputParameter.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

InputDataType& mlpack::ann::PReLU< InputDataType, OutputDataType >::InputParameter ( )

inline

Modify the input parameter.

Definition at line 105 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::inputParameter.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType const& mlpack::ann::PReLU< InputDataType, OutputDataType >::OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 108 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::outputParameter.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType& mlpack::ann::PReLU< InputDataType, OutputDataType >::OutputParameter ( )

inline

Modify the output parameter.

Definition at line 110 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::outputParameter.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType const& mlpack::ann::PReLU< InputDataType, OutputDataType >::Parameters ( ) const

inline

Get the parameters.

Definition at line 98 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::alpha.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType& mlpack::ann::PReLU< InputDataType, OutputDataType >::Parameters ( )

inline

Modify the parameters.

Definition at line 100 of file parametric_relu.hpp.

References mlpack::ann::PReLU< InputDataType, OutputDataType >::alpha.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

void mlpack::ann::PReLU< InputDataType, OutputDataType >::Reset ( )

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

template<typename Archive >

void mlpack::ann::PReLU< InputDataType, OutputDataType >::Serialize	(	Archive &	ar,
		const unsigned	int
	)

Serialize the layer.

Referenced by mlpack::ann::PReLU< InputDataType, OutputDataType >::Alpha().

Member Data Documentation

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType mlpack::ann::PReLU< InputDataType, OutputDataType >::alpha

private

Leakyness Parameter object.

Definition at line 198 of file parametric_relu.hpp.

Referenced by mlpack::ann::PReLU< InputDataType, OutputDataType >::Alpha(), mlpack::ann::PReLU< InputDataType, OutputDataType >::Deriv(), mlpack::ann::PReLU< InputDataType, OutputDataType >::Fn(), and mlpack::ann::PReLU< InputDataType, OutputDataType >::Parameters().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType mlpack::ann::PReLU< InputDataType, OutputDataType >::delta

private

Locally-stored delta object.

Definition at line 189 of file parametric_relu.hpp.

Referenced by mlpack::ann::PReLU< InputDataType, OutputDataType >::Delta().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType mlpack::ann::PReLU< InputDataType, OutputDataType >::gradient

private

Locally-stored gradient object.

Definition at line 201 of file parametric_relu.hpp.

Referenced by mlpack::ann::PReLU< InputDataType, OutputDataType >::Gradient().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

InputDataType mlpack::ann::PReLU< InputDataType, OutputDataType >::inputParameter

private

Locally-stored input parameter object.

Definition at line 192 of file parametric_relu.hpp.

Referenced by mlpack::ann::PReLU< InputDataType, OutputDataType >::InputParameter().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType mlpack::ann::PReLU< InputDataType, OutputDataType >::outputParameter

private

Locally-stored output parameter object.

Definition at line 195 of file parametric_relu.hpp.

Referenced by mlpack::ann::PReLU< InputDataType, OutputDataType >::OutputParameter().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

double mlpack::ann::PReLU< InputDataType, OutputDataType >::user_alpha

private

Leakyness Parameter given by user in the range 0 < alpha < 1.

Definition at line 204 of file parametric_relu.hpp.

The documentation for this class was generated from the following file:

src/mlpack/methods/ann/layer/parametric_relu.hpp

Public Member Functions

Private Member Functions

Private Attributes

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat> class mlpack::ann::PReLU< InputDataType, OutputDataType >

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::PReLU< InputDataType, OutputDataType >