# Relevance vector machine

__: Machine learning technique__

**Short description**Machine learning and data mining |
---|

In mathematics, a **Relevance Vector Machine (RVM)** is a machine learning technique that uses Bayesian inference to obtain parsimonious solutions for regression and probabilistic classification.^{[1]}
The RVM has an identical functional form to the support vector machine, but provides probabilistic classification.

It is actually equivalent to a Gaussian process model with covariance function:

- [math]\displaystyle{ k(\mathbf{x},\mathbf{x'}) = \sum_{j=1}^N \frac{1}{\alpha_j} \varphi(\mathbf{x},\mathbf{x}_j)\varphi(\mathbf{x}',\mathbf{x}_j) }[/math]

where [math]\displaystyle{ \varphi }[/math] is the kernel function (usually Gaussian), [math]\displaystyle{ \alpha_j }[/math] are the variances of the prior on the weight vector
[math]\displaystyle{ w \sim N(0,\alpha^{-1}I) }[/math], and [math]\displaystyle{ \mathbf{x}_1,\ldots,\mathbf{x}_N }[/math] are the input vectors of the training set.^{[2]}

Compared to that of support vector machines (SVM), the Bayesian formulation of the RVM avoids the set of free parameters of the SVM (that usually require cross-validation-based post-optimizations). However RVMs use an expectation maximization (EM)-like learning method and are therefore at risk of local minima. This is unlike the standard sequential minimal optimization (SMO)-based algorithms employed by SVMs, which are guaranteed to find a global optimum (of the convex problem).

The relevance vector machine was patented in the United States by Microsoft (patent expired September 4, 2019).^{[3]}

## See also

- Kernel trick
- Platt scaling: turns an SVM into a probability model

## References

- ↑ Tipping, Michael E. (2001). "Sparse Bayesian Learning and the Relevance Vector Machine".
*Journal of Machine Learning Research***1**: 211–244. http://jmlr.csail.mit.edu/papers/v1/tipping01a.html. - ↑ Candela, Joaquin Quiñonero (2004). "Sparse Probabilistic Linear Models and the RVM".
*Learning with Uncertainty - Gaussian Processes and Relevance Vector Machines*(PDF) (Ph.D.). Technical University of Denmark. Retrieved April 22, 2016. - ↑ Michael E. Tipping, "Relevance vector machine", US patent 6633857

## Software

- dlib C++ Library
- The Kernel-Machine Library
- rvmbinary: R package for binary classification
- scikit-rvm
- fast-scikit-rvm, rvm tutorial

## External links

- Tipping's webpage on Sparse Bayesian Models and the RVM
- A Tutorial on RVM by Tristan Fletcher
- Applied tutorial on RVM
- Comparison of RVM and SVM

Original source: https://en.wikipedia.org/wiki/Relevance vector machine.
Read more |