View source for Automatic Adjoint Differentiation Compiler

{{Short description|Run-time graph compiler for C++}}

Automatic Adjoint Differentiation [[Compiler]] ('''AADC''') is a [[Just-in-time compilation|Just-In-Time]] (JIT) compiler designed for [[Automatic differentiation|AAD]] that avoids inefficiencies introduced by traditional high-level programming languages<ref>https://www.intel.com/content/www/us/en/developer/articles/technical/new-approach-parallel-automatic-differentiation.html</ref> for repetitive calculations such as [[Monte Carlo method|Monte-Carlo]], [[Value at risk|VaR]], or Stress-Testing. It was first presented in 2019 at [https://www.youtube.com/watch?v=jHc62F4QmwQ&ab_channel=Matlogica Intel HPC Workshop]. 

In addition to [[Automatic differentiation|AAD]], the library is used to speed up 'forward' repetitive calculations, such as Monte-Carlo simulations required in [[XVA]] calculations <ref>https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/xva-pricing-application-financial-services-white-papers.pdf</ref> and for machine learning in [[Time series|time series]] analysis where it outperformed traditional Tensorflow LSTM implementation both in terms of prediction accuracy and training speed<ref>https://arxiv.org/abs/2207.03577</ref>. 

==Technology overview==
Traditionally, algorithmic adjoint differentiation is implemented through the use of a memory tape (Wengert list). [[Expression templates]] or [[Operator overloading|operator overloading]] are typically used to extract the sequence of elementary operations for each pass through an algorithm to record the tape. The tape is then used to evaluate the derivatives of interest by applying the chain rule in reverse for each elementary operation (see reverse accumulation). In practice, the use of tape slows down the program since for every iteration, the CPU needs to interpret the sequence of operations held in tape, rather than executing such operations directly.

AADC solution uses  [[Operator overloading|operator overloading]] approach to record elementary operations during program execution and instead of using tape, AADC generates optimised machine code<ref>https://ria.ua.pt/bitstream/10773/26588/1/wilmott.pdf</ref> at runtime. This machine code will be applied to a large set of data points, and the compiler will convert scalar operations to full [[Single instruction, multiple data|SIMD]]<ref>https://arxiv.org/pdf/1901.04200.pdf</ref> vector operations and process four AVX2 or eight [[AVX-512]] data samples in parallel. Since the recording happens only for one input sample the resulting recorded function is thread-safe and turns non-multithread-safe code into code that can be safely executed on multicore systems<ref>https://www.intel.com/content/www/us/en/developer/articles/technical/new-approach-parallel-automatic-differentiation.html#inpage-nav-2</ref><ref>https://meetings.siam.org/sess/dsp_talk.cfm?p=112373</ref>. Due to these properties, AADC allows to calculate all the differentials faster than the primal program <ref>https://wilmott.com/aad-breaking-the-primal-barrier/ </ref>

== References ==
<!-- Inline citations added to your article will automatically display here. See en.wikipedia.org/wiki/WP:REFB for instructions on how to add citations. -->
{{reflist}}
{{Sourceattribution|Automatic Adjoint Differentiation Compiler}}