Megastudy

From HandWiki

A megastudy or mega-study is a research study in which a large number of different treatments or interventions are tested at the same time, on the same sample or similar samples, using a common outcome measure, and using the same experimental protocol.[1][2][3]

Megastudy examples

  • Exercise encouragement[1]
  • Vaccination nudges[4]
  • Strengthening of democratic attitudes[5]
  • Interventions against climate change[3]

Many-lab studies

The megastudy technique can be combined with the many-labs approach, so that teams of researchers from across the planet conduct the same experiment locally.[3][6]

Megastudy criticisms

  • Statistical power: While the overall megastudy sample size may be large, the sample size per intervention may be relatively small, leading to underpowered designs with wide confidence intervals. As a result, while interventions may be comparable, their relative ranking by outcome measure may be noisy. Increased sample size can help address this issue.[7]
  • Lack of theory: The megastudy technique may be considered a form of "fishing expedition" for what interventions have strongest effect on the outcome measure, without much theory building.[7]

References

  1. 1.0 1.1 Milkman, Katherine L. et al. (2021). "Megastudies improve the impact of applied behavioural science". Nature 600 (7889): 478–483. doi:10.1038/s41586-021-04128-4. PMID 34880497. PMC 8822539. Bibcode2021Natur.600..478M. https://doi.org/10.1038/s41586-021-04128-4. 
  2. Duckworth, Angela L.; Milkman, Katherine L. (2022). "A guide to megastudies". PNAS Nexus (5). doi:10.1093/pnasnexus/pgac214. PMID 36712333. 
  3. 3.0 3.1 3.2 Doell, Kimberly C. (2023). "Megastudies to test the efficacy of behavioural interventions". Nature Reviews Psychology 2 (5): 263. doi:10.1038/s44159-023-00174-z. https://doi.org/10.1038/s44159-023-00174-z. 
  4. Milkman, Katherine L. et al. (2022). "A 680,000-person megastudy of nudges to encourage vaccination in pharmacies". Proceedings of the National Academy of Sciences 119 (6). doi:10.1073/pnas.2115126119. PMID 35105809. Bibcode2022PNAS..11915126M. 
  5. Voelkel, Jan G., et al. "Megastudy identifying successful interventions to strengthen Americans’ democratic attitudes." Northwestern University: Evanston, IL, USA (2022). https://www.ipr.northwestern.edu/documents/working-papers/2022/wp-22-38.pdf
  6. Pavlov, Yuri G.; Adamian, Nika; Appelhoff, Stefan; Arvaneh, Mahnaz; Benwell, Christopher S. Y.; Beste, Christian; Bland, Amy R.; Bradford, Daniel E. et al. (2021-11-01). "#EEGManyLabs: Investigating the replicability of influential EEG experiments". Cortex 144: 213–229. doi:10.1016/j.cortex.2021.03.013. ISSN 0010-9452. PMID 33965167. https://www.sciencedirect.com/science/article/pii/S0010945221001106. 
  7. 7.0 7.1 "Megastudy scepticism" (in en). 2022-05-27. https://www.jasoncollins.blog/posts/megastudy-scepticism.