Probabilistic programming language

From HandWiki

A probabilistic programming language (PPL) is a programming language designed to describe probabilistic models and then perform inference in those models. PPLs are closely related to graphical models and Bayesian networks, but are more expressive and flexible.[1] Probabilistic programming represents an attempt to "[unify] general purpose programming with probabilistic modeling."[2]

Probabilistic reasoning is a foundational technology of machine learning. It is used by companies such as Google, Amazon.com and Microsoft. Probabilistic reasoning has been used for predicting stock prices, recommending movies, diagnosing computers, detecting cyber intrusions and image detection.[3]

PPLs often extend from a basic language. The choice of underlying basic language depends on the similarity of the model to the basic language's ontology, as well as commercial considerations and personal preference. For instance, Dimple[4] and Chimple[5] are based on Java, Infer.NET is based on .NET framework,[6] while PRISM extends from Prolog.[7] However, some PPLs such as WinBUGS and Stan offer a self-contained language, with no obvious origin in another language.[8][9]

Several PPLs are in active development, including some in beta test.

Relational

A probabilistic relational programming language (PRPL) is a PPL specially designed to describe and infer with probabilistic relational models (PRMs).

A PRM is usually developed with a set of algorithms for reducing, inference about and discovery of concerned distributions, which are embedded into the corresponding PRPL.

Probabilistic programming

Probabilistic programming creates systems that help make decisions in the face of uncertainty. Probabilistic reasoning combines knowledge of a situation with the laws of probability. Until recently, probabilistic reasoning systems have been limited in scope, and have not successfully addressed real world situations. Probabilistic programming is a new approach that makes probabilistic reasoning systems easier to build and more widely applicable.[10] Reasoning about variables as probability distributions causes difficulties for novice programmers, but these difficulties can be addressed through use of Bayesian network visualisations and graphs of variable distributions embedded within the source code editor.[11]

Applications

In 2015, a 50-line PPL computer vision program was used to generate 3D models of human faces based on 2D images of those faces. The program used inverse graphics as the basis of its inferencing.[3] This made possible "in 50 lines of code what used to take thousands [whereas their experiments used their] probabilistic programming language they call Picture, which is an extension of Julia language, another language developed at MIT".[12][13] A paper on the Picture language, shown at the 2015 Computer Vision and Pattern Recognition conference, was awarded "Best Paper Honorable Mention".[14]

List of probabilistic programming languages

Name Extends from Host language
Analytica[15] C++
bayesloop[16][17] Python Python
Venture[18] Scheme C++
Probabilistic-C[19] C C
Anglican[20] Clojure Clojure
IBAL[21] OCaml
BayesDB[22] SQLite, Python
PRISM[7] B-Prolog
Infer.NET[6] .NET Framework .NET Framework
dimple[4] MATLAB, Java
chimple[5] MATLAB, Java
BLOG[23] Java
PSQL[24] SQL
BUGS[8]
FACTORIE[25] Scala
PMTK[26] MATLAB MATLAB
Alchemy[27] C++
Dyna[28] Prolog
Figaro[29] Scala
Church[30] Scheme Various: JavaScript, Scheme
ProbLog[31] Prolog Python, Jython
ProBT[32] C++, Python
Stan[9] C++
Hakaru[33] Haskell Haskell
BAli-Phy (software)[34] Haskell C++
ProbCog[35] Java, Python
Gamble[36] Racket
PWhile[37] While Python
Tuffy[38] Java
PyMC3[39] Python, Theano Python
greta[40] TensorFlow R
pomegranate[41] Python Python
Lea[42] Python Python
WebPPL[43] JavaScript JavaScript
Picture[3] Julia Julia
Turing.jl[44] Julia Julia
Troll[45] Moscow ML
Edward[46] TensorFlow Python
TensorFlow Probability[47] TensorFlow Python
Edward2[48] TensorFlow Probability Python
Pyro[49] PyTorch Python
Saul[50] Scala Scala
RankPL[51] Java
Birch[52] C++

See also

Notes

  1. ↑ "Probabilistic programming does in 50 lines of code what used to take thousands". phys.org. April 13, 2015. http://phys.org/news/2015-04-probabilistic-lines-code-thousands.html. Retrieved 2015-04-13. 
  2. ↑ "Probabilistic Programming". probabilistic-programming.org. http://probabilistic-programming.org/wiki/Home. 
  3. ↑ 3.0 3.1 3.2 "Short probabilistic programming machine-learning code replaces complex programs for computer-vision tasks". KurzweilAI. April 13, 2015. http://www.kurzweilai.net/short-probabilistic-programming-machine-learning-code-replaces-complex-programs-for-computer-vision-tasks. 
  4. ↑ 4.0 4.1 "Dimple Home Page". analog.com. https://github.com/analog-garage/dimple. 
  5. ↑ 5.0 5.1 "Chimple Home Page". analog.com. https://github.com/analog-garage/chimple. 
  6. ↑ 6.0 6.1 "Infer.NET". microsoft.com. Microsoft. http://research.microsoft.com/en-us/um/cambridge/projects/infernet/. 
  7. ↑ 7.0 7.1 "PRISM: PRogramming In Statistical Modeling". http://rjida.meijo-u.ac.jp/prism/. 
  8. ↑ 8.0 8.1 "The BUGS Project - MRC Biostatistics Unit". cam.ac.uk. http://www.mrc-bsu.cam.ac.uk/bugs/. 
  9. ↑ 9.0 9.1 "Stan". mc-stan.org. http://mc-stan.org/. 
  10. ↑ Pfeffer, Avrom (2014), Practical Probabilistic Programming, Manning Publications. p.28. ISBN:978-1 6172-9233-0
  11. ↑ Gorinova, Maria I.; Sarkar, Advait; Blackwell, Alan F.; Syme, Don (2016-01-01). "A Live, Multiple-Representation Probabilistic Programming Environment for Novices". Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. CHI '16 (New York, NY, USA: ACM): 2533–2537. doi:10.1145/2858036.2858221. ISBN 9781450333627. http://doi.acm.org/10.1145/2858036.2858221. 
  12. ↑ Hardesty, Larry (April 13, 2015). "Graphics in reverse". http://news.mit.edu/2015/better-probabilistic-programming-0413. 
  13. ↑ "MIT shows off machine-learning script to make CREEPY HEADS". https://www.theregister.co.uk/2015/04/14/mit_shows_off_machinelearning_script_to_make_creepy_heads/. 
  14. ↑ "CVPR 2015 Webpage - Awards". http://www.pamitc.org/cvpr15/awards.php. 
  15. ↑ "Analytica-- A Probabilistic Modeling Language". lumina.com. http://www.analytica.com. 
  16. ↑ "bayesloop: Probabilistic programming framework that facilitates objective model selection for time-varying parameter models". http://bayesloop.com/. 
  17. ↑ "GitHub -- bayesloop". https://github.com/christophmark/bayesloop. 
  18. ↑ "Venture -- a general-purpose probabilistic programming platform". mit.edu. http://probcomp.csail.mit.edu/venture/. 
  19. ↑ "Probabilistic C". ox.ac.uk. http://www.robots.ox.ac.uk/~brooks/probabilistic-c/. 
  20. ↑ "The Anglican Probabilistic Programming System". ox.ac.uk. http://www.robots.ox.ac.uk/~fwood/anglican/. 
  21. ↑ "IBAL Home Page". Archived from the original on December 26, 2010. https://web.archive.org/web/20101226131239/http://www.eecs.harvard.edu/~avi/IBAL/. 
  22. ↑ "BayesDB on SQLite. A Bayesian database table for querying the probable implications of data as easily as SQL databases query the data itself". GitHub. https://github.com/probcomp/bayeslite. 
  23. ↑ "Bayesian Logic (BLOG)". mit.edu. Archived from the original on June 16, 2011. https://web.archive.org/web/20110616214423/http://people.csail.mit.edu/milch/blog/. 
  24. ↑ "PSQL: A query language for probabilistic relational data". Data & Knowledge Engineering 28: 107–120. doi:10.1016/S0169-023X(98)00015-9. https://dx.doi.org/10.1016/S0169-023X(98)00015-9. 
  25. ↑ "Factorie - Probabilistic programming with imperatively-defined factor graphs - Google Project Hosting". google.com. http://code.google.com/p/factorie/. 
  26. ↑ "PMTK3 - probabilistic modeling toolkit for Matlab/Octave, version 3 - Google Project Hosting". google.com. http://code.google.com/p/pmtk3/. 
  27. ↑ "Alchemy - Open Source AI". washington.edu. http://alchemy.cs.washington.edu/. 
  28. ↑ "Dyna". http://www.dyna.org/. 
  29. ↑ "Charles River Analytics - Probabilistic Modeling Services". cra.com. http://www.cra.com/figaro. 
  30. ↑ "Church". mit.edu. http://projects.csail.mit.edu/church/wiki/Church. 
  31. ↑ "ProbLog: Probabilistic Programming". http://dtai.cs.kuleuven.be/problog. 
  32. ↑ ProbaYes. "ProbaYes - Ensemble, nous valorisations vos données". probayes.com. http://www.probayes.com/fr/Bayesian-Programming-Book/downloads/. 
  33. ↑ "Hakaru Home Page". hakaru-dev.github.io/. https://hakaru-dev.github.io/. 
  34. ↑ "BAli-Phy Home Page". bali-phy.org. http://www.bali-phy.org/. 
  35. ↑ "ProbCog". GitHub. https://github.com/opcode81/ProbCog/wiki/Features. 
  36. ↑ Culpepper, Ryan (January 17, 2017). "gamble: Probabilistic Programming". https://github.com/rmculpepper/gamble. 
  37. ↑ "PWhile Compiler". GitHub. https://github.com/zz5013/pwCompiler. 
  38. ↑ "Tuffy: A Scalable Markov Logic Inference Engine". stanford.edu. http://i.stanford.edu/hazy/tuffy/home. 
  39. ↑ PyMC devs. "PyMC3". pymc-devs.github.io. https://pymc-devs.github.io/pymc3/. 
  40. ↑ "greta: simple and scalable statistical modelling in R". https://greta-dev.github.io/greta/. 
  41. ↑ "Home — pomegranate 0.10.0 documentation" (in en). https://pomegranate.readthedocs.io/en/latest/index.html. 
  42. ↑ "Lea Home Page". bitbucket.org. https://bitbucket.org/piedenis/lea. 
  43. ↑ "WebPPL Home Page". github.com/probmods/webppl. http://dippl.org/. 
  44. ↑ "The Turing language for probabilistic programming". https://github.com/yebai/Turing.jl. 
  45. ↑ "Troll dice roller and probability calculator". http://topps.diku.dk/torbenm/troll.msp. 
  46. ↑ "Edward – Home". http://edwardlib.org/. 
  47. ↑ TensorFlow (2018-04-11). "Introducing TensorFlow Probability". https://medium.com/tensorflow/introducing-tensorflow-probability-dca4c304e245. 
  48. ↑ "'Edward2' TensorFlow Probability module" (in en). https://github.com/tensorflow/probability/tree/master/tensorflow_probability/python/edward2. 
  49. ↑ "Pyro" (in en). http://pyro.ai. 
  50. ↑ "CogComp - Home". https://cogcomp.org/page/software_view/Saul. 
  51. ↑ Rienstra, Tjitze (2018-01-18), RankPL: A qualitative probabilistic programming language based on ranking theory, https://github.com/tjitze/RankPL, retrieved 2018-01-18 
  52. ↑ "Probabilistic Programming in Birch". http://birch-lang.org/. 

External links




Retrieved from "https://handwiki.org/wiki/index.php?title=Probabilistic_programming_language&oldid=122546"