Harnessing Generative AI (GenAI) for Automated Feedback in Higher Education: A Systematic Review

Authors

DOI:

https://doi.org/10.24059/olj.v28i3.4593

Keywords:

Generative AI, chatbots, artificial intelligence, higher education, automated feedback, human-ai interaction

Abstract

In this systematic review, we synthesize ten empirical peer-reviewed articles published between 2019 and 2023 that used generative artificial intelligence (GenAI) for automated feedback in higher education. There are significant opportunities and challenges to integrate these tools effectively into learning environments as the demand for timely and personalized feedback grows. We examine the articles based on instructional contexts and system characteristics, identifying critical implementation possibilities for GenAI in automated feedback. Our findings reveal that GenAI provides diverse feedback across various contexts with multiple instructional purposes. GenAI systems can reduce instructor workload by automating routine grading and feedback tasks, allowing educators to focus on more complex teaching responsibilities with augmented capabilities. Additionally, these systems enhance communication, offer cognitive and emotional support, and improve accessibility by creating supportive, stress-free learning environments. Overall, implementing GenAI automated feedback systems improves educational outcomes and creates a more efficient and supportive learning environment for students and instructors. We conclude with future research directions to better integrate GenAI with human instruction by reconsidering instructors’ roles, especially in providing feedback to create more effective educational experiences.

Author Biographies

Sophia Soomin Lee, University of Florida

Sophia "Soomin" Lee is a doctoral student in the Educational Technology program at the University of Florida. She is interested in empowering learners who need assistance to succeed in online learning environments. Her research focuses on integrating emerging technologies, including AI, into online learning environments for effective teaching and learning, particularly in higher education and adult learning contexts. She also explores how technology can transform society's learning systems, focusing on microcredentials.

Robert L. Moore, University of Florida

Robert L. Moore, an Assistant Professor of Educational Technology at the University of Florida, directs the IDEATE Research Lab in the Institute for Advanced Learning Technologies. His research explores how the processes and structures within digital ecologies influence learner experiences, focusing on digital microcredentials and MOOCs as transformative pathways for education and careers. Dr. Moore uses systems and design thinking to identify patterns in learner behaviors and motivations, gaining insight into learners' intentions and how they achieve desired outcomes. He also investigates how technology integration affects the effectiveness of these educational and career pathways.

References

Arksey, H., & O’Malley, L. (2005). Scoping studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616

Bahroun, Z., Anane, C., Ahmed, V., & Zacca, A. (2023). Transforming education: A comprehensive review of generative artificial intelligence in educational settings through bibliometric and content analysis. Sustainability, 15(17), 12983. https://doi.org/10.3390/su151712983

Banihashem, S. K., Noroozi, O., van Ginkel, S., Macfadyen, L. P., & Biemans, H. J. (2022). A systematic review of the role of learning analytics in enhancing feedback practices in higher education. Educational Research Review, 37, Article 100489. https://doi.org/10.1016/j.edurev.2022.100489

Banihashem, S. K., Kerman, N. T., Noroozi, O., Moon, J., & Drachsler, H. (2024). Feedback sources in essay writing: peer-generated or AI-generated feedback? International Journal of Educational Technology in Higher Education, 21(1), 23. https://doi.org/10.1186/s41239-024-00455-4

Bano, M., Zowghi, D., Kearney, M., Schuck, S., & Aubusson, P. (2018). Mobile learning for science and mathematics school education: A systematic review of empirical evidence. Computers & Education, 121, 30–58. https://doi.org/10.1016/j.compedu.2018.02.006

Barrot, J. S. (2023). Using ChatGPT for second language writing: Pitfalls and potentials. Assessing Writing, 57, 100745. https://doi.org/10.1016/j.asw.2023.100745

Bayerlein, L. (2014). Students’ feedback preferences: how do students react to timely and

automatically generated assessment feedback? Assessment & Evaluation in Higher Education, 39(8), 916-931. https://doi.org/10.1080/02602938.2013.870531

Bälter, O., Enström, E., & Klingenberg, B. (2013). The effect of short formative diagnostic web quizzes with minimal feedback. Computers & Education, 60(1), 234-242. https://doi.org/10.1016/j.compedu.2012.08.014

Bozkurt, A., & Bae, H. (2024). May the force be with you Jedi: Balancing the light and dark sides of generative AI in the educational landscape. Online Learning Journal, 28(2), 1-6. https://doi.org/10.24059/olj.v28i2.4563

Cai, Z., Gui, Y., Mao, P., Wang, Z., Hao, X., Fan, X., & Tai, R. H. (2023). The effect of feedback on academic achievement in technology-rich learning environments (TREs): A meta-analytic review. Educational Research Review, 39, 100521. https://doi.org/10.1016/j.edurev.2023.100521

Cavalcanti, A. P., Barbosa, A., Carvalho, R., Freitas, F., Tsai, Y. S., Gašević, D., & Mello, R. F. (2021). Automatic feedback in online learning environments: A systematic literature review. Computers and Education: Artificial Intelligence, 2, 100027. https://doi.org/10.1016/j.caeai.2021.100027

Chaudhry, I. S., Sarwary, S. A. M., El Refae, G. A., & Chabchoub, H. (2023). Time to revisit existing student’s performance evaluation approach in higher education sector in a new era of ChatGPT—a case study. Cogent Education, 10(1), 2210461. https://doi.org/10.1080/2331186X.2023.2210461

Chiu, T. K., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. (2023). Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence, 4, 100118. https://doi.org/10.1016/j.caeai.2022.100118

Conrad, S. S., & Dabbagh, N. (2015). Examining the factors that influence how instructors provide feedback in online learning environments. International Journal of Online Pedagogy and Course Design, 5(4), 47–66. https://doi.org/10.4018/IJOPCD.2015100104

Deeva, G., Bogdanova, D., Serral, E., Snoeck, M., & De Weerdt, J. (2021). A review of automated feedback systems for learners: Classification framework, challenges and opportunities. Computers & Education, 162, 104094. https://doi.org/10.1016/j.compedu.2020.104094

Elsayed, S., & Cakir, D. (2023). Implementation of assessment and feedback in higher education. Acta Pedagogia Asiana, 2(1), 34–42. https://doi.org/10.53623/apga.v2i1.170

Escalante, J., Pack, A., & Barrett, A. (2023). AI-generated feedback on writing: Insights into efficacy and ENL student preference. International Journal of Educational Technology in Higher Education, 20(1), 57. https://doi.org/10.1186/s41239-023-00425-2

Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International, 1–15. https://doi.org/10.1080/14703297.2023.2195846

Hobert, S., & Berens, F. (2023). Developing a digital tutor as an intermediary between students, teaching assistants, and lecturers. Educational Technology Research and Development, 1-22. https://doi.org/10.1007/s11423-023-10293-2

Hu, Y. H., Fu, J. S., & Yeh, H. C. (2023). Developing an early-warning system through robotic process automation: Are intelligent tutoring robots as effective as human teachers? Interactive Learning Environments, 1-14. https://doi.org/10.1080/10494820.2022.2160467

Jasin, J., Ng, H. T., Atmosukarto, I., Iyer, P., Osman, F., Wong, P. Y. K., Pua C. Y., & Cheow, W. S. (2023). The implementation of chatbot-mediated immediacy for synchronous communication in an online chemistry course. Education and Information Technologies, 28(8), 10665-10690. https://doi.org/10.1007/s10639-023-11602-1

Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., Stadler, M., Weller, J., Kuhn, J., & Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. https://doi.org/10.1016/j.lindif.2023.102274

Katz, A., Wei, S., Nanda, G., Brinton, C., & Ohland, M. (2023). Exploring the efficacy of ChatGPT in analyzing student teamwork feedback with an existing taxonomy. arXiv preprint arXiv:2305.11882. https://doi.org/10.48550/arXiv.2305.11882

Koenka, A. C., Linnenbrink-Garcia, L., Moshontz, H., Atkinson, K. M., Sanchez, C. E., & Cooper, H. (2021). A meta-analysis on the impact of grades and comments on academic motivation and achievement: A case for written feedback. Educational Psychology, 41(7), 922-947. https://doi.org/10.1080/01443410.2019.1659939

Li, J., Ren, X., Jiang, X., & Chen, C. H. (2023). Exploring the use of ChatGPT in Chinese language classrooms. International Journal of Chinese Language Teaching, 4(3). https://doi.org/10.46451/ijclt.20230303

Li, X., Ouyang, F., Liu, J., Wei, C., & Chen, W. (2023). Examining the effects of a real-time, knowledge-aware tool for academic writing assessment. Journal of Educational Computing Research, 61(6), 1143-1174. https://doi.org/10.1177/07356331221136889

Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P., Clarke, M., Devereaux, P J, Kleijnen, J., & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine, 151(4). https://doi.org/10.1371/journal.pmed.1000100

Lim, L. A., Gasevic, D., Matcha, W., Ahmad Uzir, N. A., & Dawson, S. (2021, April). Impact of learning analytics feedback on self-regulated learning: Triangulating behavioural logs with students’ recall. In LAK21: 11th International Learning Analytics and Knowledge Conference (pp. 364-374). https://doi.org/10.1145/3448139.3448174

Lim, W. M., Gunasekara, A., Pallant, J. L., Pallant, J. I., & Pechenkina, E. (2023). Generative AI and the future of education: Ragnarök or reformation? A paradoxical perspective from management educators. The International Journal of Management Education, 21(2), 100790. https://doi.org/10.1016/j.ijme.2023.100790

Lin, X., Luterbach, K., Gregory, K. H., & Sconyers, S. E. (2024). A case study investigating the utilization of ChatGPT in online discussions. Online Learning Journal, 28(2), 124-149. https://doi.org/10.24059/olj.v28i2.4407

Memmert, L., Tavanapour, N., & Bittner, E. (2023). Learning by doing: Educators’ perspective on an illustrative tool for AI-generated scaffolding for students in conceptualizing design science research studies. Journal of Information Systems Education, 34(3), 279-292. https://aisel.aisnet.org/jise/vol34/iss3/3

Michel-Villarreal, R., Vilalta-Perdomo, E., Salinas-Navarro, D. E., Thierry-Aguilera, R., & Gerardou, F. S. (2023). Challenges and opportunities of generative AI for higher education as explained by ChatGPT. Education Sciences, 13(9), 856. https://doi.org/10.3390/educsci13090856

Moore, R. L. (2019). The role of data analytics in education: Possiblities and limitations. In B. Khan, J. R. Corbeil, & M. E. Corbeil (Eds.), Responsible analytics and data mining in education: Global perspectives on quality, support, and decision making (pp. 101–118). Routledge. https://doi.org/10.4324/9780203728703-8

Moore, R. L., & Blackmon, S. J. (2022). From the learner’s perspective: A systematic review of MOOC learner experiences (2008–2021). Computers & Education, 190, 1–15. https://doi.org/10.1016/j.compedu.2022.104596

Moore, R. L., Hwang, W., & Moses, J. D. (2024). A systematic review of mobile-based microlearning in adult learner contexts. Educational Technology & Society, 27(1), 137–146. https://doi.org/10.30191/ETS.202401_27(1).SP02

Moore, R. L., Jiang, S., & Abramowitz, B. (2023). What would the matrix do? A systematic review of K-12 AI learning contexts and learner-interface interactions. Journal of Research on Technology in Education, 55(1), 7–20. https://doi.org/10.1080/15391523.2022.2148785

Moore, R. L., & Miller, C. N. (2022). Fostering cognitive presence in online courses: A systematic review (2008-2020). Online Learning, 26(1), 130–149. https://doi.org/10.24059/olj.v26i1.3071

Mulliner, E., & Tucker, M. (2017). Feedback on feedback practice: Perceptions of students and academics. Assessment & Evaluation in Higher Education, 42(2), 266-288. https://doi.org/10.1080/02602938.2015.1103365

Neo, M. (2022). The Merlin Project: Malaysian students’ acceptance of an AI chatbot in their learning process. Turkish Online Journal of Distance Education, 23(3), 31-48. https://doi.org/10.17718/tojde.1137122

Picciano, A. G. (2024). Graduate teacher education students use and evaluate ChatGPT as an essay-writing tool. Online Learning, 28(2), 1-20. https://olj.onlinelearningconsortium.org/index.php/olj/index

Pishchukhina, O., & Allen, A. (2021). Supporting learning in large classes: Online formative assessment and automated feedback. 2021 30th Annual Conference of the European Association for Education in Electrical and Information Engineering (EAEEIE), 1–4. https://doi.org/10.1109/EAEEIE50507.2021.9530953

Powers, F. E., & Moore, R. L. (2021). When failure is an option: A scoping review of failure states in game-based learning. TechTrends, 65(4), 615–625. https://doi.org/10.1007/s11528-021-00606-8

Sales, G. C. (1993). Adapted and adaptive feedback in technology-based instruction. Interactive instruction and feedback, 14, 159-175.

Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153-189. https://doi.org/10.3102/0034654307313795

Smolansky, A., Cram, A., Raduescu, C., Zeivots, S., Huber, E., & Kizilcec, R. F. (2023, July). Educator and student perspectives on the impact of generative AI on assessments in higher education. In Proceedings of the Tenth ACM Conference on Learning@ Scale (pp. 378-382). https://doi.org/10.1145/3573051.3596191

Strzelecki, A. (2023). To use or not to use ChatGPT in higher education? A study of students’ acceptance and use of technology. Interactive learning environments, 1-14. https://doi.org/10.1080/10494820.2023.2209881

Swindell, A., Greeley, L., Farag, A., & Verdone, B. (2024). Against artificial education: Towards an ethical framework for generative artificial intelligence (GenAI) use in education. Online Learning Journal, 28(2), 7-28. https://doi.org/10.24059/olj.v28i2.4438

Van der Kleij, F. M., Feskens, R. C., & Eggen, T. J. (2015). Effects of feedback in a computer-based learning environment on students’ learning outcomes: A meta-analysis. Review of Educational Research, 85(4), 475–511. https://doi.org/10.3102/0034654314564881

Vittorini, P., Menini, S., & Tonelli, S. (2021). An AI-Based system for formative and summative assessment in data science courses. International Journal of Artificial Intelligence in Education, 31(2), 159–185.

https://doi.org/10.1007/s40593-020-00230-2

Wambsganss, T., Janson, A., & Leimeister, J. M. (2022). Enhancing argumentative writing with automated feedback and social comparison nudging. Computers & Education, 191, 104644. https://doi.org/10.1016/j.compedu.2022.104644

Xu, W., & Ouyang, F. (2022). A systematic review of AI role in the educational system based on a proposed conceptual framework. Education and Information Technologies, 27(3), 4195–4223. https://doi.org/10.1007/s10639-021-10774-y

Yan, L., Sha, L., Zhao, L., Li, Y., Martinez‐Maldonado, R., Chen, G., Li, X., Jin, Y., & Gašević, D. (2024). Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology, 55(1), 90–112. https://doi.org/10.1111/bjet.13370

Downloads

Published

2024-09-01

How to Cite

Lee, S. S., & Moore, R. L. (2024). Harnessing Generative AI (GenAI) for Automated Feedback in Higher Education: A Systematic Review. Online Learning, 28(3). https://doi.org/10.24059/olj.v28i3.4593

Issue

Vol. 28 No. 3 (2024): Online Learning

Section

Online and Blended Learning in the Age of Generative AI

License

Copyright (c) 2024 Soomin, Rob

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.

This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.

Author(s) hold copyright in their work, and retain publishing rights without restrictions