An implementation of a lstm network layer. More...

Public Member Functions
	LSTM ()
	Create the LSTM object. More...

	LSTM (const size_t inSize, const size_t outSize, const size_t rho)
	Create the LSTM layer object using the specified parameters. More...

template<typename eT >
void	Backward (const arma::Mat< eT > &&, arma::Mat< eT > &&gy, arma::Mat< eT > &&g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...

OutputDataType const &	Delta () const
	Get the delta. More...

OutputDataType &	Delta ()
	Modify the delta. More...

bool	Deterministic () const
	The value of the deterministic parameter. More...

bool &	Deterministic ()
	Modify the value of the deterministic parameter. More...

template<typename eT >
void	Forward (arma::Mat< eT > &&input, arma::Mat< eT > &&output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

template<typename eT >
void	Gradient (arma::Mat< eT > &&input, arma::Mat< eT > &&, arma::Mat< eT > &&)

OutputDataType const &	Gradient () const
	Get the gradient. More...

OutputDataType &	Gradient ()
	Modify the gradient. More...

InputDataType const &	InputParameter () const
	Get the input parameter. More...

InputDataType &	InputParameter ()
	Modify the input parameter. More...

std::vector< LayerTypes > &	Model ()
	Get the model modules. More...

OutputDataType const &	OutputParameter () const
	Get the output parameter. More...

OutputDataType &	OutputParameter ()
	Modify the output parameter. More...

OutputDataType const &	Parameters () const
	Get the parameters. More...

OutputDataType &	Parameters ()
	Modify the parameters. More...

size_t	Rho () const
	Get the maximum number of steps to backpropagate through time (BPTT). More...

size_t &	Rho ()
	Modify the maximum number of steps to backpropagate through time (BPTT). More...

template<typename Archive >
void	Serialize (Archive &ar, const unsigned int)
	Serialize the layer. More...

Private Attributes
size_t	backwardStep
	Locally-stored number of backward steps. More...

arma::mat	cellActivationError
	Locally-stored cell activation error. More...

LayerTypes	cellActivationModule
	Locally-stored cell activation module. More...

LayerTypes	cellModule
	Locally-stored cell module. More...

std::vector< arma::mat >	cellParameter
	Locally-stored cell parameters. More...

OutputDataType	delta
	Locally-stored delta object. More...

DeltaVisitor	deltaVisitor
	Locally-stored delta visitor. More...

bool	deterministic
	If true dropout and scaling is disabled, see notes above. More...

arma::mat	forgetGateError
	Locally-stored foget gate error. More...

LayerTypes	forgetGateModule
	Locally-stored forget gate module. More...

size_t	forwardStep
	Locally-stored number of forward steps. More...

OutputDataType	gradient
	Locally-stored gradient object. More...

size_t	gradientStep
	Locally-stored number of gradient steps. More...

LayerTypes	hiddenStateModule
	Locally-stored hidden state module. More...

LayerTypes	input2GateModule
	Locally-stored input 2 gate module. More...

LayerTypes	inputGateModule
	Locally-stored input gate module. More...

InputDataType	inputParameter
	Locally-stored input parameter object. More...

size_t	inSize
	Locally-stored number of input units. More...

std::vector< LayerTypes >	network
	Locally-stored list of network modules. More...

std::vector< arma::mat >	outParameter
	Locally-stored output parameters. More...

LayerTypes	output2GateModule
	Locally-stored output 2 gate module. More...

LayerTypes	outputGateModule
	Locally-stored output gate module. More...

OutputDataType	outputParameter
	Locally-stored output parameter object. More...

OutputParameterVisitor	outputParameterVisitor
	Locally-stored output parameter visitor. More...

size_t	outSize
	Locally-stored number of output units. More...

arma::mat	prevCell
	Locally-stored previous cell state. More...

arma::mat	prevError
	Locally-stored previous error. More...

arma::mat	prevOutput
	Locally-stored previous output. More...

size_t	rho
	Number of steps to backpropagate through time (BPTT). More...

OutputDataType	weights
	Locally-stored weight object. More...

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::LSTM< InputDataType, OutputDataType >

An implementation of a lstm network layer.

This class allows specification of the type of the activation functions used for the gates and cells and also of the type of the function used to initialize and update the peephole weights.

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 51 of file layer_types.hpp.

Constructor & Destructor Documentation

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

mlpack::ann::LSTM< InputDataType, OutputDataType >::LSTM ( )

Create the LSTM object.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

mlpack::ann::LSTM< InputDataType, OutputDataType >::LSTM	(	const size_t	inSize,
		const size_t	outSize,
		const size_t	rho
	)

Create the LSTM layer object using the specified parameters.

Parameters

inSize	The number of input units.
outSize	The number of output units.
rho	Maximum number of steps to backpropagate through time (BPTT).

Member Function Documentation

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

template<typename eT >

void mlpack::ann::LSTM< InputDataType, OutputDataType >::Backward	(	const arma::Mat< eT > &&	,
		arma::Mat< eT > &&	gy,
		arma::Mat< eT > &&	g
	)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters

input	The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType const& mlpack::ann::LSTM< InputDataType, OutputDataType >::Delta ( ) const

inline

Get the delta.

Definition at line 123 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::delta.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType& mlpack::ann::LSTM< InputDataType, OutputDataType >::Delta ( )

inline

Modify the delta.

Definition at line 125 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::delta.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

bool mlpack::ann::LSTM< InputDataType, OutputDataType >::Deterministic ( ) const

inline

The value of the deterministic parameter.

Definition at line 98 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::deterministic.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

bool& mlpack::ann::LSTM< InputDataType, OutputDataType >::Deterministic ( )

inline

Modify the value of the deterministic parameter.

Definition at line 100 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::deterministic.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

template<typename eT >

void mlpack::ann::LSTM< InputDataType, OutputDataType >::Forward	(	arma::Mat< eT > &&	input,
		arma::Mat< eT > &&	output
	)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

template<typename eT >

void mlpack::ann::LSTM< InputDataType, OutputDataType >::Gradient	(	arma::Mat< eT > &&	input,
		arma::Mat< eT > &&	,
		arma::Mat< eT > &&
	)

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType const& mlpack::ann::LSTM< InputDataType, OutputDataType >::Gradient ( ) const

inline

Get the gradient.

Definition at line 128 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::gradient.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType& mlpack::ann::LSTM< InputDataType, OutputDataType >::Gradient ( )

inline

Modify the gradient.

Definition at line 130 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::gradient.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

InputDataType const& mlpack::ann::LSTM< InputDataType, OutputDataType >::InputParameter ( ) const

inline

Get the input parameter.

Definition at line 113 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::inputParameter.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

InputDataType& mlpack::ann::LSTM< InputDataType, OutputDataType >::InputParameter ( )

inline

Modify the input parameter.

Definition at line 115 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::inputParameter.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

std::vector<LayerTypes>& mlpack::ann::LSTM< InputDataType, OutputDataType >::Model ( )

inline

Get the model modules.

Definition at line 133 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::network, and mlpack::ann::LSTM< InputDataType, OutputDataType >::Serialize().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType const& mlpack::ann::LSTM< InputDataType, OutputDataType >::OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 118 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::outputParameter.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType& mlpack::ann::LSTM< InputDataType, OutputDataType >::OutputParameter ( )

inline

Modify the output parameter.

Definition at line 120 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::outputParameter.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType const& mlpack::ann::LSTM< InputDataType, OutputDataType >::Parameters ( ) const

inline

Get the parameters.

Definition at line 108 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::weights.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType& mlpack::ann::LSTM< InputDataType, OutputDataType >::Parameters ( )

inline

Modify the parameters.

Definition at line 110 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::weights.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

size_t mlpack::ann::LSTM< InputDataType, OutputDataType >::Rho ( ) const

inline

Get the maximum number of steps to backpropagate through time (BPTT).

Definition at line 103 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::rho.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

size_t& mlpack::ann::LSTM< InputDataType, OutputDataType >::Rho ( )

inline

Modify the maximum number of steps to backpropagate through time (BPTT).

Definition at line 105 of file lstm.hpp.

References mlpack::ann::LSTM< InputDataType, OutputDataType >::rho.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

template<typename Archive >

void mlpack::ann::LSTM< InputDataType, OutputDataType >::Serialize	(	Archive &	ar,
		const unsigned	int
	)

Serialize the layer.

Referenced by mlpack::ann::LSTM< InputDataType, OutputDataType >::Model().

Member Data Documentation

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

size_t mlpack::ann::LSTM< InputDataType, OutputDataType >::backwardStep

private

Locally-stored number of backward steps.

Definition at line 198 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

arma::mat mlpack::ann::LSTM< InputDataType, OutputDataType >::cellActivationError

private

Locally-stored cell activation error.

Definition at line 213 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

LayerTypes mlpack::ann::LSTM< InputDataType, OutputDataType >::cellActivationModule

private

Locally-stored cell activation module.

Definition at line 183 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

LayerTypes mlpack::ann::LSTM< InputDataType, OutputDataType >::cellModule

private

Locally-stored cell module.

Definition at line 180 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

std::vector<arma::mat> mlpack::ann::LSTM< InputDataType, OutputDataType >::cellParameter

private

Locally-stored cell parameters.

Definition at line 204 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType mlpack::ann::LSTM< InputDataType, OutputDataType >::delta

private

Locally-stored delta object.

Definition at line 222 of file lstm.hpp.

Referenced by mlpack::ann::LSTM< InputDataType, OutputDataType >::Delta().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

DeltaVisitor mlpack::ann::LSTM< InputDataType, OutputDataType >::deltaVisitor

private

Locally-stored delta visitor.

Definition at line 189 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

bool mlpack::ann::LSTM< InputDataType, OutputDataType >::deterministic

private

If true dropout and scaling is disabled, see notes above.

Definition at line 219 of file lstm.hpp.

Referenced by mlpack::ann::LSTM< InputDataType, OutputDataType >::Deterministic().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

arma::mat mlpack::ann::LSTM< InputDataType, OutputDataType >::forgetGateError

private

Locally-stored foget gate error.

Definition at line 216 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

LayerTypes mlpack::ann::LSTM< InputDataType, OutputDataType >::forgetGateModule

private

Locally-stored forget gate module.

Definition at line 174 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

size_t mlpack::ann::LSTM< InputDataType, OutputDataType >::forwardStep

private

Locally-stored number of forward steps.

Definition at line 195 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType mlpack::ann::LSTM< InputDataType, OutputDataType >::gradient

private

Locally-stored gradient object.

Definition at line 225 of file lstm.hpp.

Referenced by mlpack::ann::LSTM< InputDataType, OutputDataType >::Gradient().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

size_t mlpack::ann::LSTM< InputDataType, OutputDataType >::gradientStep

private

Locally-stored number of gradient steps.

Definition at line 201 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

LayerTypes mlpack::ann::LSTM< InputDataType, OutputDataType >::hiddenStateModule

private

Locally-stored hidden state module.

Definition at line 171 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

LayerTypes mlpack::ann::LSTM< InputDataType, OutputDataType >::input2GateModule

private

Locally-stored input 2 gate module.

Definition at line 162 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

LayerTypes mlpack::ann::LSTM< InputDataType, OutputDataType >::inputGateModule

private

Locally-stored input gate module.

Definition at line 168 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

InputDataType mlpack::ann::LSTM< InputDataType, OutputDataType >::inputParameter

private

Locally-stored input parameter object.

Definition at line 228 of file lstm.hpp.

Referenced by mlpack::ann::LSTM< InputDataType, OutputDataType >::InputParameter().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

size_t mlpack::ann::LSTM< InputDataType, OutputDataType >::inSize

private

Locally-stored number of input units.

Definition at line 144 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

std::vector<LayerTypes> mlpack::ann::LSTM< InputDataType, OutputDataType >::network

private

Locally-stored list of network modules.

Definition at line 192 of file lstm.hpp.

Referenced by mlpack::ann::LSTM< InputDataType, OutputDataType >::Model().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

std::vector<arma::mat> mlpack::ann::LSTM< InputDataType, OutputDataType >::outParameter

private

Locally-stored output parameters.

Definition at line 207 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

LayerTypes mlpack::ann::LSTM< InputDataType, OutputDataType >::output2GateModule

private

Locally-stored output 2 gate module.

Definition at line 165 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

LayerTypes mlpack::ann::LSTM< InputDataType, OutputDataType >::outputGateModule

private

Locally-stored output gate module.

Definition at line 177 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType mlpack::ann::LSTM< InputDataType, OutputDataType >::outputParameter

private

Locally-stored output parameter object.

Definition at line 231 of file lstm.hpp.

Referenced by mlpack::ann::LSTM< InputDataType, OutputDataType >::OutputParameter().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputParameterVisitor mlpack::ann::LSTM< InputDataType, OutputDataType >::outputParameterVisitor

private

Locally-stored output parameter visitor.

Definition at line 186 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

size_t mlpack::ann::LSTM< InputDataType, OutputDataType >::outSize

private

Locally-stored number of output units.

Definition at line 147 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

arma::mat mlpack::ann::LSTM< InputDataType, OutputDataType >::prevCell

private

Locally-stored previous cell state.

Definition at line 159 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

arma::mat mlpack::ann::LSTM< InputDataType, OutputDataType >::prevError

private

Locally-stored previous error.

Definition at line 210 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

arma::mat mlpack::ann::LSTM< InputDataType, OutputDataType >::prevOutput

private

Locally-stored previous output.

Definition at line 156 of file lstm.hpp.

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

size_t mlpack::ann::LSTM< InputDataType, OutputDataType >::rho

private

Number of steps to backpropagate through time (BPTT).

Definition at line 150 of file lstm.hpp.

Referenced by mlpack::ann::LSTM< InputDataType, OutputDataType >::Rho().

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>

OutputDataType mlpack::ann::LSTM< InputDataType, OutputDataType >::weights

private

Locally-stored weight object.

Definition at line 153 of file lstm.hpp.

Referenced by mlpack::ann::LSTM< InputDataType, OutputDataType >::Parameters().

The documentation for this class was generated from the following files:

src/mlpack/methods/ann/layer/layer_types.hpp
src/mlpack/methods/ann/layer/lstm.hpp

Public Member Functions

Private Attributes

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat> class mlpack::ann::LSTM< InputDataType, OutputDataType >

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::LSTM< InputDataType, OutputDataType >