Predicting Cognitive Presence in At-Scale Online Learning: MOOC and For-Credit Online Course Environments

Jeonghyun Lee, Farahnaz Soleimani, India Irish, John Hosmer, IV, Meryem Yilmaz Soylu, Roy Finkelberg, Saurabh Chatterjee


In this study, we work towards a strategy to measure and enhance the quality of interactions in discussion forums at scale. We present a machine learning (ML) model which identifies the phase of cognitive presence exhibited by a student’s post and suggest future applications of such a model to help online students develop higher-order thinking. We collect discussion forum transcript data from two online courses: CS1301 (an introductory computer programming MOOC) offered by edX and CS6601 (a graduate course on artificial intelligence) which uses the Piazza online discussion tool. We manually code a random sample of students’ posts based on the Community of Inquiry coding scheme and explore trends in cognitive presence within and across the courses. We further use this coded data to analyze the relationship between students’ observed cognitive presence and course grades. In terms of testing and building an ML model, we use a Bidirectional Encoder Representations from Transformers model that uses a deep learning technique to train large text corpus and fine-tune the language model. Our results suggest that deeper cognitive engagement with course concepts, as expressed by higher cognitive presence, are associated with better learning outcomes for students in both course settings. Our ML approach achieves 92.5% accuracy on the classification task, motivating the use of ML for instructional interventions in online courses. We expect that our research study will not only contribute to extending the literature on cognitive presence but also have a beneficial impact on online instructors or curriculum developers in higher education.


cognitive presence; discussion forums; machine learning; higher education

Full Text:



Al-Shabandar, R., Hussain, A. J., Liatsis, P., & Keight, R. (2019). Detecting at-risk students with early interventions using machine learning techniques. IEEE Access, 7, 149464-149478.

Amemado, D., & Manca, S. (2017). Learning from decades of online distance education: MOOCs and the Community of Inquiry Framework. Journal of e-learning and Knowledge Society, 13(2).

An, H., Shin, S, & Lim, K. (2009). The effects of different instructor facilitation approaches on students’ interactions during asynchronous online discussions. Computers & Education, 53(3), 749-760.

Arisoy, E., Sainath, T. N., Kingsbury, B., & Ramabhadran, B. (2012, June). Deep neural network language models. In Proceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT (pp. 20-28). Association for Computational Linguistics.

Askeroth, J.H., & Richardson, J.C. (2019). Instructor perceptions of quality learning in MOOCs they teach. Online Learning, 23(4), 135-159.

Baglione, S., & Nastanski, M. (2007). The superiority of online discussion: Faculty perceptions. The Quarterly Review of Distance Education, 8(2), 139-150.

Baran, E., & Correia, A. (2009). Student-led facilitation strategies in online discussions. Distance Education, 30(3), 339-361.

Bliuc, A., Ellis, R., Goodyear, P., & Piggott, L. (2009). Learning through face-to-face and online discussions: Associations between students’ conceptions, approaches and academic performance in political science. British Journal of Educational Technology, 41(3), 512-524.

Chapman, D., Storberg-Walker, J., & Stone, S. (2008). Hitting reply: A qualitative study to understand student decisions to respond to online discussion postings. E-Learning and Digital Media, 5(1), 29-39.

Chen, B., Chang, Y. H., Ouyang, F., & Zhou, W. (2018). Fostering student engagement in online discussion through social learning analytics. The Internet and Higher Education, 37, 21-30.

Cheung, W., Hew, K., & Ng, C. (2008). Toward an understanding of why students contribute in asynchronous online discussions. Journal of Educational Computing Research, 38(1), 29-50.

Chi, M. T., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219-243.

Darabi, A., Arrastia, M., Nelson, D., Cornille, T. and Liang, X. (2011). Cognitive presence in asynchronous online learning: A comparison of four discussion strategies. Journal of Computer Assisted Learning, 27(3), 216-227.

deNoyelles, A., Zydney, J., & Chen, B. (2014). Strategies for creating a community of inquiry through online asynchronous discussions. Journal of Online Learning and Teaching, 10(1), 153-165.

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.

Galikyan, I., Admiraal, W., & Kester, L. (2021). MOOC discussion forums: The interplay of the cognitive and the social. Computers & Education, 165, 104133.

Gao, F., Zhang, T., & Franklin, T. (2013). Designing asynchronous online discussion environments: Recent progress and possible future directions. British Journal of Educational Technology, 44(3), 469-483.

Garrison, D. R., & Akyol, Z. (2015). Toward the development of a metacognition construct for communities of inquiry. The Internet and Higher Education, 24, 66–71.

Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7-23.

Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. The Internet and Higher Education, 13(1-2), 5-9.

Guo, P., Saab, N., Wu, L., & Admiraal, W. (2021). The Community of Inquiry perspective on students' social presence, cognitive presence, and academic performance in online project-based learning. Journal of Computer Assisted Learning, 37(5), 1479–1493. https://doi. org/10.1111/jcal.12586

Hayati H., Idrissi M. K., & Bennani S. (2020) Automatic classification for cognitive engagement in online discussion forums: Text mining and machine learning approach. In Bittencourt, I., Cukurova, M., Muldner, K., Luckin, R., & Millán, E. (Eds.), Artificial Intelligence in Education. AIED 2020. Lecture Notes in Computer Science, vol 12164 (pp. 114-118). Springer, Cham.

Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45-58.

Hew, K. F., Hu, X., Qiao, C., & Tang, Y. (2020). What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach. Computers & Education, 145, 103724.

Hu, Y., Donald, C., Giacaman, N., & Zhu, Z. (2020, March). Towards automated analysis of cognitive presence in MOOC discussions: A manual classification study. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge (pp. 135-140).

Irish, I., Finkelberg, R., Nkemelu, D., Gujrania, S., Padiyath, A., Raman, S., Tailor, C., Arriaga, R., & Starner, T. (2020, August). PARQR: Automatic Post Suggestion in the Piazza Online Forum to Support Degree Seeking Online Masters Students. In Proceedings of the Seventh ACM Conference on Learning@ Scale (pp. 125-134).

Kilis, S., & Yildirim, Z. (2019). Posting patterns of students’ social presence, cognitive presence, and teaching presence in online learning. Online Learning, 23(2), 179-195.

Kovanović, V., Joksimović, S., Waters, Z., Gašević, D., Kitto, K., Hatala, M., & Siemens, G. (2016, April). Towards automated content analysis of discussion transcripts: A cognitive presence case. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge (pp. 15-24).

Leitner, P., Khalil, M., & Ebner, M. (2017). Learning analytics in higher education—A literature review. In A. Peña-Ayala (Ed.), Learning analytics: fundaments, applications, and trends (pp. 1-23), Springer.

Mazzolini, M., & Maddison, S. (2007). When to jump in: The role of the instructor in online discussion forums. Computers & Education., 49(2), 193-213.

Nanzi, D., Hamilton, M., & Harland, J. (2012). Evaluating the quality of interaction in asynchronous discussion forums in fully online courses. Distance Education, 33(1), 5-30.

Neto, V., Rolim, V., Cavalcanti, A. P., Lins, R. D., Gasevic, D., & Ferreiramello, R. (2021). Automatic Content Analysis of Online Discussions for Cognitive Presence: A Study of the Generalizability across Educational Contexts. IEEE Transactions on Learning Technologies, 14(3), 299-312.

Pelánek, R. (2020, March). Learning analytics challenges: Trade-offs, methodology, scalability. In Proceedings of the Tenth International Conference on Learning Analytics & Knowledge (pp. 554-558).

Quintana, R. M., Pinto, J. D., & Tan, Y. (2021). What we learned when we compared discussion posts from one MOOC hosted on two platforms. Online Learning, 25(4), 7-24.

Rogers, A., Kovaleva, O., & Rumshisky, A. (2020). A primer in bertology: What we know about how bert works. Transactions of the Association for Computational Linguistics, 8, 842-866.

Sadaf, A., Kim, S. Y., & Wang, Y. (2021). A comparison of cognitive presence, learning, satisfaction, and academic performance in case-based and non-case-based online discussions. American Journal of Distance Education, 35(3), 214-227.

Shea, P., & Bidjerano, T. (2009). Community of inquiry as a theoretical framework to foster “epistemic engagement” and “cognitive presence” in online education. Computers & Education, 52(3), 543- 553.

Sadaf, A., & Olesova, L. (2017). Enhancing cognitive presence in online case discussions with questions based on the practical inquiry model. American Journal of Distance Education, 31(1), 56-69.

Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5), 30-32.

Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98-110.

Zhu, M., Bonk, C. J., & Sari, A.R. (2018). Instructor experiences designing MOOCs in higher education: Pedagogical, resource, and logistical considerations and challenges. Online Learning, 22(4), 203-241.


Copyright (c) 2022 Jeonghyun Lee, Farahnaz Soleimani, India Irish, John Hosmer, IV, Meryem Yilmaz Soylu, Roy Finkelberg, Saurabh Chatterjee

License URL: