Salta al menu principale di navigazione Salta al contenuto principale Salta al piè di pagina del sito
Ricerche di Psicologia/Psychological Research Journal - Open Access

Articoli

Ultimi articoli

Unraveling the impact of prolific: Exploring the presence of bias when studying psychological constructs related to the use of technology

DOI
https://doi.org/10.3280/rip2025oa22334
Inviata
27 marzo 2026
Pubblicato
27-03-2026

Abstract

This study investigates potential biases in data collected through online paid platforms, specifically Prolific, compared to traditional snowball convenience sampling, when studying psychological constructs related to technology. The researchers analyzed data from a separate study examining ChatGPT adoption among Italian university students, using both Prolific and snowball sampling methods. Three technology-related constructs were examined: technology self-efficacy (SE), technology anxiety (AX), and social norms related to technology use (SN). Analysis of covariance revealed a significant difference between groups in technology self-efficacy (SE), with Prolific participants exhibiting higher scores. No significant differences were found in AX and SN. The findings highlight potential selection bias, suggesting Prolific may attract participants with higher technological confidence. It is important to stress that this difference may also reflect a combination of self-selection into the platform, the presence of financial compensation, and the inherent biases of convenience snowball samples. The study underscores the importance of considering such recruitment-related biases when generalizing research findings, particularly in technology-related studies, and emphasizes the need for transparency in reporting sampling methods.

Riferimenti bibliografici

  1. Aburbeian, A. M., Owda, A. Y., & Owda, M. (2022). A technology acceptance model survey of the Metaverse prospects. Ai, 3(2), 285-302. Doi: 10.3390/ai3020018.
  2. Albert, D. A., & Smilek, D. (2023). Comparing attentional disengagement between Prolific and MTurk samples. Scientific Reports, 13(1), 20574. Doi: 10.1038/s41598-023-46048-5.
  3. Arechar, A. A., & Rand, D. G. (2021). Turking in the time of COVID. Behavior research methods, 53(6), 2591-2595. Doi: 10.3758/s13428-021-01588-4.
  4. Bock, O., Baetge, I., & Nicklisch, A. (2014). hroot: Hamburg registration and organization online tool. European Economic Review, 71, 117-120. Doi: 10.1016/j.euroecorev.2014.07.003.
  5. Buchanan, T. (2018). Personality biases in different types of ‘internet samples’ can influence research outcomes. Computers in Human Behavior, 86, 235-244. Doi: 10.1016/j.chb.2018.05.002.
  6. Buhrmester, M. D., Talaifar, S., & Gosling, S. D. (2018). An evaluation of Amazon’s Mechanical Turk, its rapid rise, and its effective use. Perspectives on psychological science, 13(2), 149-154. Doi: 10.1177/1745691617706516.
  7. Caliciuri, R., Milani, M., Sorgente, A., Rodriguez, R. & Lanz, M. (under review). Exploration of ChatGPT adoption among Italian university students: An application of the Technology Acceptance Model. Current Psychology.
  8. Cheung, J. H., Burns, D. K., Sinclair, R. R., & Sliter, M. (2017). Amazon Mechanical Turk in organizational psychology: An evaluation and practical recommendations. Journal of Business and Psychology, 32, 347-361. Doi: 10.1007/s10869-016-9458-5.
  9. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS quarterly, 319-340. Doi: 10.2307/249008.
  10. Douglas, B. D., Ewell, P. J., & Brauer, M. (2023). Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA. Plos one, 18(3), e0279720. Doi: 10.1371/journal.pone.0279720.
  11. Eisinga, R., Grotenhuis, M. T., & Pelzer, B. (2013). The reliability of a two-item scale: Pearson, Cronbach, or Spearman-Brown?. International journal of public health, 58, 637-642. Doi: 10.1007/s00038-012-0416-3.
  12. Goodman, J. K., & Paolacci, G. (2017). Crowdsourcing consumer research. Journal of Consumer Research, 44(1), 196-210. Doi: 10.1093/jcr/ucx047.
  13. Greiner, B. (2015). Subject pool recruitment procedures: organizing experiments with ORSEE. Journal of the Economic Science Association, 1(1), 114-125. Doi: 10.1007/s40881-015-0004-4.
  14. Guner, H., & Acarturk, C. (2020). The use and acceptance of ICT by senior citizens: a comparison of technology acceptance model (TAM) for elderly and young adults. Universal Access in the Information Society, 19, 311-330. Doi: 10.1007/s10209-018-0642-4.
  15. Hauser, D. J., & Schwarz, N. (2016). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior research methods, 48, 400-407. Doi: 10.3758/s13428-015-0578-z.
  16. IBM Corp. (2023). IBM SPSS Statistics for Windows, Version 29.0.1.0. Armonk, NY: IBM Corp.
  17. Jamieson, J. (2004). Analysis of covariance (ANCOVA) with difference scores. International Journal of Psychophysiology, 52(3), 277-283. Doi: 10.1016/j.ijpsycho.2003.12.009.
  18. Newman, A., Bavik, Y. L., Mount, M., & Shao, B. (2021). Data collection via online platforms: Challenges and recommendations for future research. Applied Psychology, 70(3), 1380-1402. Doi: 10.1111/apps.12302.
  19. Ni, A., & Cheung, A. (2023). Understanding secondary students’ continuance intention to adopt AI-powered intelligent tutoring system for English learning. Education and Information Technologies, 28(3), 3191-3216. Doi: 10.1007/s10639-022-11305-z.
  20. Miller, G. A., & Chapman, J. P. (2001). Misunderstanding analysis of covariance. Journal of Abnormal Psychology, 110(1), 40-48. Doi: 10.1037/0021-843X.110.1.40.
  21. Marreiros, H., Tonin, M., Vlassopoulos, M., & Schraefel, M. C. (2017). “Now that you mention it”: A survey experiment on information, inattention and online privacy. Journal of Economic Behavior & Organization, 140, 1-17. Doi: 10.1016/j.jebo.2017.03.024.
  22. Ministry of Education, University, and Research. (2019). Ministerial Decree No. 1156, December 19, 2019. -- https://www.miur.gov.it/-/decreto-ministeriale-n-1156-del-19-dicembre-2019.
  23. Palan, S., & Schitter, C. (2018). Prolific. Ac ‒ A subject pool for online experiments. Journal of behavioral and experimental finance, 17, 22-27. Doi: 10.1016/j.jbef.2017.12.004.
  24. Peer, E., Vosgerau, J., & Acquisti, A. (2014). Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior research methods, 46, 1023-1031. Doi: 10.3758/s13428-013-0434-y.
  25. Stanley, P. (2022). Analysis of covariance: A useful tool for the pharmacologist to reduce variation and improve precision using fewer animals. British Journal of Pharmacology, 179(14), 3645-3650. Doi: 10.1111/bph.15855.
  26. Walter, S. L., Seibert, S. E., Goering, D., & O’Boyle, E. H. (2019). A tale of two sample sources: Do results from online panel data and conventional data converge?. Journal of Business and Psychology, 34, 425-452. Doi: 10.1007/s10869-018-9552-y.