Skip to main navigation menu Skip to main content Skip to site footer
FrancoAngeli Journals

Articles/Articoli

Vol. 15 No. 2 (2024): Artificial Intelligence in Schools and University Education: Risks and Opportunities

Thematic analysis of college students’ perceptions of ChatGPT use in the university setting

DOI
https://doi.org/10.3280/ess2-2024oa18422
Submitted
settembre 5, 2024
Published
2025-01-31

Abstract

The study carried out aimed to analyse the advantages and disadvantages of perspective, and then focus on the qualitative analysis of students’ perceptions of the strengths and weaknesses of using ChatGPT in the university context. It took place within a broader process initiated within the Docimology workshop developed at the University of Palermo ‒ course of studies in Pedagogical Sciences - in the academic year 2023/2024 with 160 students, in which the aim was to verify whether targeted teaching actions, built within the theoretical framework of formative assessment, could increase students’ levels of awareness, sensitizing them to the possible risks of erosion of critical thinking and verification of the accuracy of the sources consulted. The qualitative thematic analysis, conducted at the end of the educational intervention, reveals the need to promote a critical and conscious use of ChatGPT, supported by docimological and metacognitive skills.

References

  1. Albanese M., Compagno G. (2022). La valutazione delle attività neurodidattiche. Fondamenti, tecniche e strumenti. Roma: Anicia.
  2. Albanese M., Compagno G. (2023). La prospettiva neurodidattica nella valutazione delle competenze metodologico-didattiche dell’insegnante. In Ricerche in Neuroscienze Educative 2023 Il futuro prossimo dell’educazione nell’universo digitale (pp. 8-9). Edizioni Universitarie Romane.
  3. Albanese M., Fiorello E., and Compagno G. (2024). Mind in Mind Lab: evaluation tools and critical analysis. Italian Journal of Health Education, Sport and Inclu-sive Didactics, 8(3).
  4. Allal L., Cardinet I. and Perrenoud Ph. (1979) (Eds.) L’évaluation formative dans l’enseignement différenciée, Berne/Franckfor: Peter Lang.
  5. Black P., Wiliam D. (1998). Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice, 5(1): 7-74, DOI: 10.1080/0969595980050102.
  6. Braun V., Clarke V. (2019). Reflecting on reflexive thematic analysis. Qualitative research in sport, exercise and health, 11(4): 589-597.
  7. Braun V., Clarke V., and Gray D. (Eds.) (2017). Collecting qualitative data: A practical guide to textual, media and virtual techniques. Cambridge University Press.
  8. Braun V., Clarke V., and Hayfield N. (2019). A starting point for your journey, not a map: Nikki Hayfield in conversation with Virginia Braun and Victoria Clarke about Thematic Analysis. In Qualitative Research in Psychology, pp. 1-22.
  9. Buchberger B (2023). Is ChatGPT Smarter than Master’s Applicants?. Research Institute for Symbolic Computation: Linz, Austria.
  10. Calonghi L. (1983). Valutare: risultati docimologici e indicazioni per la scheda. Novara: Istituto geografico De Agostini.
  11. Cañadas L. (2023). Contribution of formative assessment for developing teaching competences in teacher education. European Journal of Teacher Education, 46(3): 516-532.
  12. Clarke V., Braun V. (2016). Thematic analysis. In: E. Lyons, A. Coyle (eds). Analysing qualitative data in psychology (2nd ed.) (pp. 84-103). London: Sage.
  13. Clarke V., Braun V. (2013). Teaching thematic analysis: Overcoming challenges and developing strategies for effective learning. The psychologist, 26(2): 120-123.
  14. Cosi A., Voltas N., Lázaro-Cantabrana J. L., Morales P., Calvo M., Molina S. and Qui-roga M. Á. (2020). Formative assessment at university through digital technology tools. Profesorado, revista de currículum y formación del profesorado, 24(1): 164-183.
  15. Deeley S. J. (2018). Using technology to facilitate effective assessment for learning and feedback in higher education. Assessment and Evaluation in Higher Education, 43(3): 439-448. DOI: 10.1080/02602938.2017.1356906.
  16. Earl A. M. (2003) Assessment as Learning: Using Classroom Assessment to Maximize Student Learning, Corwin Press. Educational Research, 68(3): 249-276.
  17. Fuchs K. (2023). Exploring the opportunities and challenges of NLP models in higher education: Is Chat GPT a blessing or a course?. Front. Educ., 8, 1166682.
  18. Grange T., Patera S. (2021). Formative evaluation to support the development of the deep dimension of acting with competence. A case study. Education Sciences & Society - Open Access, 12(2). DOI: 10.3280/ess2-2021oa12430.
  19. Hattie J. (2009). Visible Learning. A synthesis of over 800 meta-analyses relating to achievement. London-New York, NY: Routledge.
  20. Hattie J., Timperley H. (2007). The power of feedback. Review of educational research, 77(1): 81-112.
  21. Heitink M. C., Van der Kleij F. M., Veldkamp B. P., Schildkamp K., and Kippers W. B. (2016). A systematic review of prerequisites for implementing assessment for learning in classroom practice. Educational research review, 17: 50-62. DOI: 10.1016/j.edurev.2015.12.002.
  22. Heritage M. (2010). Formative Assessment and Next-Generation Assessment Systems: Are We Losing an Opportunity?. Council of Chief State School Officers.
  23. Irons A., Elkington S. (2021). Enhancing learning through formative assessment and feedback. Routledge.
  24. Javaid M., Haleem A., Singh R.P., Kahn S., and Khan I.H. (2023), Unlocking the opportunities through ChatGPT Tool towards ameliorating the education system. Bench Council Trans. Benchmarks Stand. Eva, 3: 100-115.
  25. Lo Chung Kwan. (2023). What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature. Education Sciences, 13(4), 410. DOI: 10.3390/educsci13040410.
  26. Lui A. M., Andrade H. L. (2022). The next black box of formative assessment: A model of the internal mechanisms of feedback processing. In Frontiers in Educa-tion (7, 751548). Frontiers.
  27. Montenegro-Rueda M., Fernández-Cerero J., Fernández-Batanero J.M., López and Meneses E. (2023). Impact of the Implementation of ChatGPT in Education: A Systematic Review. Computers, 12, 153. DOI: 10.3390/computers12080153.
  28. Morris R., Perry T., and Wardle L. (2021). Formative assessment and feedback for learning in higher education: A systematic review. Review of Education, 9(3), e3292.
  29. Oranga J. (2023). Benefits of Artificial Intelligence (ChatGPT) in Education and Learning: Is Chat GPT Helpful?. International Review of Practical Innovation, Technology and Green Energy (IRPITAGE), 3(3): 46-50. DOI: 10.54443/irpitage.v3i3.1250.
  30. Pagani V. (2020). Dare voce ai dati. L’analisi dei dati testuali nella ricerca educativa (pp. 1-230). Edizioni Junior.
  31. Patton M.Q. (2002). Qualitative evaluation and research methods (3rd ed.). Newbury Park: Sage.
  32. Scriven M. (1991). Prose and cons about goal-free evaluation. Evaluation Practice, 12(1): 55-62.
  33. Stiggins R. J. (2002). Assessment crisis: The absence of assessment for learning. Phi Delta Kappan, 83(10): 758-765.
  34. Strzelecki A. (2023). To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interact. Learn. Environ., 1-14.
  35. Terry G., Hayfield N., Clarke V., and Braun V. (2017). Thematic analysis. In: C. Willig, W. Rogers (eds.). The SAGE handbook of qualitative research in psychology, (pp. 17-37), London: Sage.
  36. Vannini I. (2019). La qualità nella didattica: metodologie e strumenti di progetta-zione e valutazione. Edizioni Centro Studi Erickson.
  37. Vertecchi B. (1976). La valutazione formativa. Torino: Loescher.
  38. Wang T., Lund B.D., Marengo A., Pagano A., Mannuru N.R., Teel Z.A., and Pange J. (2023). Exploring the Potential Impact of Artificial Intelligence (AI) on International Students in Higher Education: Generative AI, Chatbots, Analytics, and International Student Success. Appl. Sci., 13, 6716.
  39. Zanniello G. (2016). La didattica tra storia e ricerca. Roma: Armando.

Metrics

Metrics Loading ...