Aggregative Contingent Estimation (ACE) Program

From HandWiki

Aggregative Contingent Estimation (ACE) was a program of the Office of Incisive Analysis (OIA) at the Intelligence Advanced Research Projects Activity (IARPA).[1][2] The program ran from June 2010 until June 2015.[3]


The broad program announcement for ACE was published on June 30, 2010.[4] ACE funded the Aggregative Contingent Estimation System (ACES) website and interface on July 15, 2011.[5] They funded The Good Judgment Project some time around July 2011.[6] ACE has been covered in The Washington Post ''[7] and Wired Magazine.[8] The program was concluded by late 2015.[9] The program manager was future IARPA director Jason Gaverick Matheny.[10]

Goals and methods

The official website says that the goals of ACE are "to dramatically enhance the accuracy, precision, and timeliness of intelligence forecasts for a broad range of event types, through the development of advanced techniques that elicit, weight, and combine the judgments of many intelligence analysts."[1] The website claims that ACE seeks technical innovations in the following areas:[1]

  • efficient elicitation of probabilistic judgments, including conditional probabilities for contingent events
  • mathematical aggregation of judgments by many individuals, based on factors that may include: past performance, expertise, cognitive style, metaknowledge, and other attributes predictive of accuracy
  • effective representation of aggregated probabilistic forecasts and their distributions.

There is a fair amount of research funded by grants made by the IARPA ACE program.[11]


The ACE has collaborated with partners who compete in its forecasting tournaments. Their most notable partner is The Good Judgment Project from Philip E. Tetlock et al.[12] (winner of a 2013 ACE tournament)[7] ACE also partnered with the ARA to create the Aggregative Contingent Estimation System (ACES).[5]

Data from ACE is fed into another program, called Forecasting Science and Technology (ForeST), which partners with SciCast from George Mason University.[13]


  1. 1.0 1.1 1.2 Matheny, Jason; Rieber, Steve. "Aggregative Contingent Estimation (ACE)". Intelligence Advanced Research Projects Activity. 
  2. "Aggregative Contingent Estimation". Office of the Director of National Intelligence, United States. 
  3. Harbert, Tam (2015-10-19). "IARPA's New Director Wants You to Surprise Him". 
  4. "Aggregative Contingent Estimation System". Federal Business Opportunities. June 30, 2010. 
  5. 5.0 5.1 Hickey, Kathleen (July 15, 2011). "Intell site tests crowdsourcing's ability to predict future". GCN. 
  6. "The idea behind the Good Judgment Project". The Good Judgment Project. July 27, 2011. 
  7. 7.0 7.1 Horowitz, Michael (November 26, 2013). "Good judgment in forecasting international affairs (and an invitation for season 3)". The Washington Post. 
  8. Drummond, Katie (April 22, 2010). "Can Algorithms Find the Best Intelligence Analysts?". Wired Magazine. 
  9. Corrin, Amber (2015-09-23). "IARPA's high-stakes intelligence experiment". 
  10. Marc Prensky (7 August 2012). Brain Gain: Technology and the Quest for Digital Wisdom. St. Martin's Press. p. 260. ISBN 978-1-137-09317-2. "The ACE program manager is Jason Matheny" 
  11. "Google Scholar listing of research funded by IARPA ACE". 
  12. "The Project". The Good Judgment Project. 
  13. Matheny, Jason. "Forecasting Science & Technology (ForeST)". Intelligence Advanced Research Projects Activity. 

External links