mlpack 3.4.2
Public Member Functions | List of all members
PReLU< InputDataType, OutputDataType > Class Template Reference

The PReLU activation function, defined by (where alpha is trainable) More...

#include <parametric_relu.hpp>

Public Member Functions

 PReLU (const double userAlpha=0.03)
 Create the PReLU object using the specified parameters. More...
 
double & Alpha ()
 Modify the non zero gradient. More...
 
double const & Alpha () const
 Get the non zero gradient. More...
 
template<typename DataType >
void Backward (const DataType &input, const DataType &gy, DataType &g)
 Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f. More...
 
OutputDataType & Delta ()
 Modify the delta. More...
 
OutputDataType const & Delta () const
 Get the delta. More...
 
template<typename InputType , typename OutputType >
void Forward (const InputType &input, OutputType &output)
 Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...
 
OutputDataType & Gradient ()
 Modify the gradient. More...
 
OutputDataType const & Gradient () const
 Get the gradient. More...
 
template<typename eT >
void Gradient (const arma::Mat< eT > &input, const arma::Mat< eT > &error, arma::Mat< eT > &gradient)
 Calculate the gradient using the output delta and the input activation. More...
 
OutputDataType & OutputParameter ()
 Modify the output parameter. More...
 
OutputDataType const & OutputParameter () const
 Get the output parameter. More...
 
OutputDataType & Parameters ()
 Modify the parameters. More...
 
OutputDataType const & Parameters () const
 Get the parameters. More...
 
void Reset ()
 
template<typename Archive >
void serialize (Archive &ar, const unsigned int)
 Serialize the layer. More...
 

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::PReLU< InputDataType, OutputDataType >

The PReLU activation function, defined by (where alpha is trainable)

\begin{eqnarray*} f(x) &=& \max(x, alpha*x) \\ f'(x) &=& \left\{ \begin{array}{lr} 1 & : x > 0 \\ alpha & : x \le 0 \end{array} \right. \end{eqnarray*}

Template Parameters
InputDataTypeType of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataTypeType of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 45 of file parametric_relu.hpp.

Constructor & Destructor Documentation

◆ PReLU()

PReLU ( const double  userAlpha = 0.03)

Create the PReLU object using the specified parameters.

The non zero gradient can be adjusted by specifying tha parameter alpha in the range 0 to 1. Default (alpha = 0.03). This parameter is trainable.

Parameters
userAlphaNon zero gradient

Member Function Documentation

◆ Alpha() [1/2]

double & Alpha ( )
inline

Modify the non zero gradient.

Definition at line 120 of file parametric_relu.hpp.

◆ Alpha() [2/2]

double const & Alpha ( ) const
inline

Get the non zero gradient.

Definition at line 118 of file parametric_relu.hpp.

◆ Backward()

void Backward ( const DataType &  input,
const DataType &  gy,
DataType &  g 
)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f.

Using the results from the feed forward pass.

Parameters
inputThe propagated input activation.
gyThe backpropagated error.
gThe calculated gradient.

◆ Delta() [1/2]

OutputDataType & Delta ( )
inline

Modify the delta.

Definition at line 110 of file parametric_relu.hpp.

◆ Delta() [2/2]

OutputDataType const & Delta ( ) const
inline

Get the delta.

Definition at line 108 of file parametric_relu.hpp.

◆ Forward()

void Forward ( const InputType &  input,
OutputType &  output 
)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters
inputInput data used for evaluating the specified function.
outputResulting output activation.

◆ Gradient() [1/3]

OutputDataType & Gradient ( )
inline

Modify the gradient.

Definition at line 115 of file parametric_relu.hpp.

◆ Gradient() [2/3]

OutputDataType const & Gradient ( ) const
inline

Get the gradient.

Definition at line 113 of file parametric_relu.hpp.

◆ Gradient() [3/3]

void Gradient ( const arma::Mat< eT > &  input,
const arma::Mat< eT > &  error,
arma::Mat< eT > &  gradient 
)

Calculate the gradient using the output delta and the input activation.

Parameters
inputThe input parameter used for calculating the gradient.
errorThe calculated error.
gradientThe calculated gradient.

◆ OutputParameter() [1/2]

OutputDataType & OutputParameter ( )
inline

Modify the output parameter.

Definition at line 105 of file parametric_relu.hpp.

◆ OutputParameter() [2/2]

OutputDataType const & OutputParameter ( ) const
inline

Get the output parameter.

Definition at line 103 of file parametric_relu.hpp.

◆ Parameters() [1/2]

OutputDataType & Parameters ( )
inline

Modify the parameters.

Definition at line 100 of file parametric_relu.hpp.

◆ Parameters() [2/2]

OutputDataType const & Parameters ( ) const
inline

Get the parameters.

Definition at line 98 of file parametric_relu.hpp.

◆ Reset()

void Reset ( )

◆ serialize()

void serialize ( Archive &  ar,
const unsigned int   
)

Serialize the layer.


The documentation for this class was generated from the following file: