mlpack 3.4.2
Static Public Member Functions | List of all members
GELUFunction Class Reference

The GELU function, defined by. More...

#include <gelu_function.hpp>

Static Public Member Functions

static double Deriv (const double y)
 Computes the first derivative of the GELU function. More...
 
template<typename InputVecType , typename OutputVecType >
static void Deriv (const InputVecType &y, OutputVecType &x)
 Computes the first derivatives of the GELU function. More...
 
static double Fn (const double x)
 Computes the GELU function. More...
 
template<typename InputVecType , typename OutputVecType >
static void Fn (const InputVecType &x, OutputVecType &y)
 Computes the GELU function. More...
 

Detailed Description

The GELU function, defined by.

\begin{eqnarray*} f(x) = 0.5 * x * {1 + tanh[(2/pi)^(1/2) * (x + 0.044715 * x^3)]} \\ f'(x) = 0.5 * tanh(0.0356774 * x^3) + 0.797885 * x) + (0.0535161x^3 + 0.398942 * x) * sech^2(0.0356774 * x^3+0.797885 * x) + 0.5\\ \end{eqnarray*}

Definition at line 31 of file gelu_function.hpp.

Member Function Documentation

◆ Deriv() [1/2]

static double Deriv ( const double  y)
inlinestatic

Computes the first derivative of the GELU function.

Parameters
yInput data.
Returns
f'(x)

Definition at line 65 of file gelu_function.hpp.

◆ Deriv() [2/2]

static void Deriv ( const InputVecType &  y,
OutputVecType &  x 
)
inlinestatic

Computes the first derivatives of the GELU function.

Parameters
yInput data.
xThe resulting derivatives.

Definition at line 80 of file gelu_function.hpp.

◆ Fn() [1/2]

static double Fn ( const double  x)
inlinestatic

Computes the GELU function.

Parameters
xInput data.
Returns
f(x).

Definition at line 40 of file gelu_function.hpp.

References M_PI.

◆ Fn() [2/2]

static void Fn ( const InputVecType &  x,
OutputVecType &  y 
)
inlinestatic

Computes the GELU function.

Parameters
xInput data.
yThe resulting output activation.

Definition at line 53 of file gelu_function.hpp.

References M_PI.


The documentation for this class was generated from the following file: