Social:Seriousness check
The seriousness check is a technique that can be used in online research (also called Internet-based research, Web-based research, Web-based experiments) to improve data quality. Nowadays, many scientific studies with human participants are conducted online and are accessible to a large diversity of participants. Nonetheless, many people just want to look at the different pages of the questionnaire, instead of giving carefully chosen answers to the questions.[1] The seriousness check addresses this problem: In this approach the respondents are asked about the seriousness of their participation or for a probability estimate that they will complete the entire study or experiment.[2][3][4] Thus, by using the seriousness check irrelevant data entries can be easily identified and be excluded from the data analysis.
Method
Seriousness checks can be implemented both before [4] and after [6] participation in the study. However, it has been shown that the seriousness check is a good predictor of dropout rates when implemented in the first page of the experiment.[1]
There are Web-based tools, e.g. WEXTOR,[7] that implement the seriousness check by default.
Impact
Several studies have shown that performing a seriousness check at the start of a study best predicts motivation and dropout probability.[1][8] It was observed that of those answering “I would like to look at the pages only" around 75% will drop, while of those answering “I would like to seriously participate now" only ca. 10-15% will drop during the study. Overall, about 30-50% of visitors will fail the seriousness check, i.e. answer “I would like to look at the pages only".[1] Moreover, it was found that emphasizing seriousness increased information seeking in participants and the time they spent on the study.[9] Following up on this, it was shown that motivation and self-reported seriousness significantly predict several data quality indicators [10]
References
- ↑ 1.0 1.1 1.2 1.3 Reips, U.-D. (2009). Internet experiments: Methods, guidelines, metadata. Human Vision and Electronic Imaging XIV, Proc. SPIE, 7240, 724008. https://doi.org/10.1117/12.823416
- ↑ Musch, J., & Klauer, K. C. (2002). Psychological experimenting on the World Wide Web: Investigating content effects in syllogistic reasoning. In B. Batinic, U.-D. Reips, & M. Bosnjak (Eds.), Online social sciences (pp. 181–212). Hogrefe & Huber Publishers.
- ↑ Reips, U.-D. (2007). The methodology of Internet-based experiments. In A. Joinson, K. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford Handbook of Internet Psychology (pp. 373-390). Oxford University Press.
- ↑ 4.0 4.1 Reips, U.-D. (2002). Standards for internet-based experimenting. Experimental Psychology, 49(4), 243–256. https://doi.org/10.1026/1618-3169.49.4.243
- ↑ Reips, U.-D., & Krantz, J. H. (2010). Conducting true experiments on the Web. In S. D. Gosling & J. A. Johnson (Eds.), Advanced methods for conducting online behavioral research (p. 193–216). American Psychological Association. https://doi.org/10.1037/12076-01
- ↑ Aust, F., Diedenhofen, B., Ullrich, S., & Musch, J. (2013). Seriousness checks are useful to improve data validity in online research. Behavior Research Methods, 45(2), 527–535. https://doi.org/10.3758/s13428-012-0265-2
- ↑ Reips, U.-D.; Neuhaus, C. (2002). "WEXTOR: A web-based tool for generating and visualizing experimental designs and procedures". Behavior Research Methods, Instruments, and Computers 34 (2): 234–240. doi:10.3758/BF03195449. PMID 12109018. https://link.springer.com/content/pdf/10.3758/BF03195449.pdf.
- ↑ Reips, U.-D. (2008). How Internet-mediated research changes science. In A. Barak (Ed.), Psychological aspects of cyberspace: Theory, research, applications (p. 268–294). Cambridge University Press. https://doi.org/10.1017/CBO9780511813740.013
- ↑ Bayram, A. Burcu (2018). "Serious subjects: A test of the seriousness technique to increase participant motivation in political science experiments". Research & Politics 5 (2): 205316801876745. doi:10.1177/2053168018767453. ISSN 2053-1680.
- ↑ Verbree, Anne-Roos; Toepoel, Vera; Perada, Dominique (2020). "The Effect of Seriousness and Device Use on Data Quality" (in en). Social Science Computer Review 38 (6): 720–738. doi:10.1177/0894439319841027. ISSN 0894-4393.
 |  |
