Soboleva modified hyperbolic tangent

Short description: Mathematical activation function in data analysis

The Soboleva modified hyperbolic tangent, also known as (parametric) Soboleva modified hyperbolic tangent activation function ([P]SMHTAF), is a special S-shaped function based on the hyperbolic tangent, given by

Equation	Left tail control	Right tail control
[math]\displaystyle{ \operatorname{smht}x = \frac {e^{ax} - e^{-bx}} {e^{cx} + e^{-dx}}. }[/math]

History

This function was originally proposed as "modified hyperbolic tangent" by Ukrainian scientist Elena V. Soboleva (Елена В. Соболева) as a utility function for multi-objective optimization and choice modelling in decision-making.

Practical usage

The function has since been introduced into neural network theory and practice.

It was also used in economics for modelling consumption and investment, to approximate current-voltage characteristics of field-effect transistors and light-emitting diodes, to design antenna feeders, and analyze plasma temperatures and densities in the divertor region of fusion reactors.

Sensitivity to parameters

Derivative of the function is defined by the formula:

[math]\displaystyle{ \operatorname{smht}'(x) \doteq \frac {ae^{ax} + be^{-bx}} {e^{cx} + e^{-dx}} - \operatorname{smht}(x)\frac {ce^{cx} - de^{-dx}} {e^{cx} + e^{-dx}} }[/math]

The following conditions are keeping the function limited on y-axes: a ≤ c, b ≤ d.

A family of recurrence-generated parametric Soboleva modified hyperbolic tangent activation functions (NPSMHTAF, FPSMHTAF) was studied with parameters a = c and b = d. It is worth noting that in this case, the function is not sensitive to flipping the left and right-sides parameters:

Equation	Left prevalence	Right prevalence
[math]\displaystyle{ \frac {e^{ax} - e^{-bx}} {e^{ax} + e^{-bx}} = \frac {e^{bx} - e^{-ax}} {e^{bx} + e^{-ax}} }[/math]

The function is sensitive to ratio of the denominator coefficients and often is used without coefficients in the numerator:

Equation	Basic chart	Scaled function
[math]\displaystyle{ \operatorname{ssht}x = \frac {e^{x} - e^{-x}} {e^{\alpha x} + e^{-\beta x}}. }[/math] Extremum estimates: [math]\displaystyle{ x_{\min} = \frac {1} {2} \ln \frac {\beta-1} {\beta+1}; }[/math] [math]\displaystyle{ x_{\max} = \frac {1} {2} \ln \frac {\alpha+1} {\alpha-1}. }[/math]	Error creating thumbnail: Unable to save thumbnail to destination

With parameters a = b = c = d = 1 the modified hyperbolic tangent function reduces to the conventional tanh(x) function, whereas for a = b = 1 and c = d = 0, the term becomes equal to sinh(x).

See also

• Activation function
• e (mathematical constant)
• Equal incircles theorem, based on sinh
• Hausdorff distance
• Inverse hyperbolic functions
• List of integrals of hyperbolic functions
• Poinsot's spirals
• Sigmoid function

Further reading

• "A Note on the New Activation Function of Gompertz Type". Biomath Communications (Faculty of Mathematics and Informatics, University of Plovdiv "Paisii Hilendarski", Plovdiv, Bulgaria / Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, Sofia, Bulgaria: Biomath Forum (BF)) 4 (2). 2017. doi:10.11145/10.11145/bmc.2017.10.201. ISSN 2367-5233. http://www.biomathforum.org/biomath/index.php/conference/article/view/1070. Retrieved 2020-06-19.  (20 pages) [6]