mlpack 3.4.2
Public Member Functions | List of all members
PositionalEncoding< InputDataType, OutputDataType > Class Template Reference

Positional Encoding injects some information about the relative or absolute position of the tokens in the sequence. More...

#include <positional_encoding.hpp>

Public Member Functions

 PositionalEncoding ()
 Create PositionalEncoding object. More...
 
 PositionalEncoding (const size_t embedDim, const size_t maxSequenceLength)
 Create the PositionalEncoding layer object using the specified parameters. More...
 
template<typename eT >
void Backward (const arma::Mat< eT > &, const arma::Mat< eT > &gy, arma::Mat< eT > &g)
 Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...
 
OutputDataType & Delta ()
 Modify the delta. More...
 
OutputDataType const & Delta () const
 Get the delta. More...
 
InputDataType const & Encoding () const
 Get the positional encoding vector. More...
 
template<typename eT >
void Forward (const arma::Mat< eT > &input, arma::Mat< eT > &output)
 Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...
 
InputDataType & InputParameter ()
 Modify the input parameter. More...
 
InputDataType const & InputParameter () const
 Get the input parameter. More...
 
OutputDataType & OutputParameter ()
 Modify the output parameter. More...
 
OutputDataType const & OutputParameter () const
 Get the output parameter. More...
 
template<typename Archive >
void serialize (Archive &ar, const unsigned int)
 Serialize the layer. More...
 

Detailed Description

template<typename InputDataType = arma::mat, typename OutputDataType = arma::mat>
class mlpack::ann::PositionalEncoding< InputDataType, OutputDataType >

Positional Encoding injects some information about the relative or absolute position of the tokens in the sequence.

The input and the output have the same shape: (embedDim * maxSequenceLength, batchSize). The embeddings are stored consequently.

Template Parameters
InputDataTypeType of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataTypeType of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 37 of file positional_encoding.hpp.

Constructor & Destructor Documentation

◆ PositionalEncoding() [1/2]

Create PositionalEncoding object.

◆ PositionalEncoding() [2/2]

PositionalEncoding ( const size_t  embedDim,
const size_t  maxSequenceLength 
)

Create the PositionalEncoding layer object using the specified parameters.

Parameters
embedDimThe length of the embedding vector.
maxSequenceLengthNumber of tokens in each sequence.

Member Function Documentation

◆ Backward()

void Backward ( const arma::Mat< eT > &  ,
const arma::Mat< eT > &  gy,
arma::Mat< eT > &  g 
)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters
*(input) The propagated input activation.
gyThe backpropagated error.
gThe calculated gradient.

◆ Delta() [1/2]

OutputDataType & Delta ( )
inline

Modify the delta.

Definition at line 91 of file positional_encoding.hpp.

◆ Delta() [2/2]

OutputDataType const & Delta ( ) const
inline

Get the delta.

Definition at line 89 of file positional_encoding.hpp.

◆ Encoding()

InputDataType const & Encoding ( ) const
inline

Get the positional encoding vector.

Definition at line 94 of file positional_encoding.hpp.

◆ Forward()

void Forward ( const arma::Mat< eT > &  input,
arma::Mat< eT > &  output 
)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters
inputInput data used for evaluating the specified function.
outputResulting output activation.

◆ InputParameter() [1/2]

InputDataType & InputParameter ( )
inline

Modify the input parameter.

Definition at line 81 of file positional_encoding.hpp.

◆ InputParameter() [2/2]

InputDataType const & InputParameter ( ) const
inline

Get the input parameter.

Definition at line 79 of file positional_encoding.hpp.

◆ OutputParameter() [1/2]

OutputDataType & OutputParameter ( )
inline

Modify the output parameter.

Definition at line 86 of file positional_encoding.hpp.

◆ OutputParameter() [2/2]

OutputDataType const & OutputParameter ( ) const
inline

Get the output parameter.

Definition at line 84 of file positional_encoding.hpp.

◆ serialize()

void serialize ( Archive &  ar,
const unsigned int   
)

Serialize the layer.


The documentation for this class was generated from the following file: