Ridge function

Not to be confused with ridge regression.

In mathematics, a ridge function is any function $f:{\mathbb{ℝ}}^d\to \mathbb{ℝ}$ that can be written as the composition of an univariate function $g:\mathbb{ℝ}\to \mathbb{ℝ}$, that is called a profile function, with an affine transformation, given by a direction vector $a\in {\mathbb{ℝ}}^d$ with shift $b\in \mathbb{ℝ}$.

Then, the ridge function reads $f(x)=g(x^{\mathrm{\top }}a+b)$ for $x\in {\mathbb{ℝ}}^d$.

Coinage of the term 'ridge function' is often attributed to B.F. Logan and L.A. Shepp.

Relevance

A ridge function is not susceptible to the curse of dimensionality, making it an instrumental tool in various estimation problems. This is a direct result of the fact that ridge functions are constant in $d-1$ directions: Let $a_1,\dots ,a_{d-1}$ be $d-1$ independent vectors that are orthogonal to $a$, such that these vectors span $d-1$ dimensions. Then

$f(\mathit{x}+\sum _{k=1}^{d-1}{c}_k{\mathit{a}}_k)=g(\mathit{x}\cdot \mathit{a}+\sum _{k=1}^{d-1}{c}_k{\mathit{a}}_k\cdot \mathit{a})=g(\mathit{x}\cdot \mathit{a}+\sum _{k=1}^{d-1}{c}_{k}0)=g(\mathit{x}\cdot \mathit{a})=f(\mathit{x})$

for all $c_i\in \mathbb{ℝ},1\le i<d$. In other words, any shift of $\mathit{x}$ in a direction perpendicular to $\mathit{a}$ does not change the value of $f$.

Ridge functions play an essential role in amongst others projection pursuit, generalized linear models, and as activation functions in neural networks. For a survey on ridge functions, see. For books on ridge functions, see.