Using LMS Log Data to Explore Student Engagement with Coursework Videos




learning analytics, student engagement, LMS, higher education


Driven by the increased availability of Learning Management System data, this study explored its value and sought understanding of student behaviour through the information contained in activity level log data. Specifically, this study examined analytics data to understand students’ engagement with online videos. Learning analytics data from the MoodleTM and Vimeo® platforms were compared. The research also examined the impact of video length on engagement, and how engagement with videos changed over the course of a semester when multiple video resources were used in a course. The comparison in platform learning analytics showed differences in metrics thus offering a caution to users relying on unidimensional metrics. While the results support the notion that log data do provide educators with an opportunity for review, the time and expertise in extracting, handling, and using the data may stifle its widespread adoption.


Alfayez, Z. (2021). Designing Educational Videos for University Websites Based on Students’ Preferences. Online Learning, 25(2).

Au, T., Li, S., & Ma, G. (2003). Applying and evaluating models to predict customer attrition using data mining techniques. Journal of Comparative International Management, 6(1), 10-22.

Axelsen, M., Redmond, P., Heinrich, E., & Henderson, M. (2020). The evolving field of learning analytics research in higher education: From data analysis to theory generation, an agenda for future research. Australasian Journal of Educational Technology, 36(2), 1-7. http://doi:10.14742/ajet.5510

Barua, P. D., Zhou, X., Gururajan, R., & Chan, K. C. (2018). Determination of factors influencing student engagement using a Learning Management System in a tertiary setting. In 2018 IEEE/WIC/ACM International Conference on Web Intelligence (pp. 604-609). Santiago: Institute of Electrical and Electronics Engineers (IEEE).

Beer, C., Clark, K., & Jones, D. (2010). Indicators of engagement. In Proceedings ASCILITE 2010 (pp. 75-86). Sydney: Australasian Society for Computers in Learning in Tertiary Education.

Bodily, R., Graham, C. R., & Bush, M. D. (2017). Online learner engagement: Opportunities and challenges with using data analytics. Educational Technology, 57(1), 10-18.

Bodily, R., Nyland, R., & Wiley, D. (2017). The RISE framework: Using learning analytics to automatically identify open educational resources for continuous improvement. International Review of Research in Open and Distributed Learning, 18(2), 103-122.

Bohan, J., & Stack, N. (2014). Do weekly online assignments promote student engagement and enhance learning in first-year psychology students? Psychology of Education Review, 38(2). Retrieved from

Brame, C. J. (2016). Effective educational videos: Principles and guidelines for maximizing student learning from video content. CBE Life Science Education, 15(4), 1-6. http://doi:10.1187/cbe.16-03-0125

Brozina, C., Knight, D. B., Kinoshita, T., & Johri, A. (2019). Engaged to succeed: Understanding first-year engineering students’ course engagement and performance through analytics. IEEE Access, 7, 163686-163699. http://doi:10.1109/ACCESS.2019.2945873

Bulathwela, S., Pérez-Ortiz, M., Lipani, A., Yilmaz, E., & Shawe-Taylor, J. (2020). Predicting engagement in video lectures. In A. N. Rafferty, J. Whitehill, C. Romero, & V. Cavalli-Sforza (Eds.), Proceedings of the 13th International Conference on Educational Data Mining (pp. 50-60). Montreal: International Educational Data Mining Society.

Carmichael, M., Reid, A.-K., & Karpicke, J. D. (2018). Assessing the impact of educational video on student engagement, critical thinking and learning: The current state of play. A SAGE White Paper. SAGE Publishing.

Casey, K., & Azcona, D. (2017). Utilizing student activity patterns to predict performance. International Journal of Educational Technology in Higher Education, 14(4). http://doi:10.1186/s41239-017-0044-3

Dixson, M. D. (2015). Measuring student engagement in the online course: The Online Student Engagement Scale (OSE). Online Learning, 19(4). http://doi:10.24059/olj.v19i4.561

Fincham, E., Whitelock-Wainwright, A., Kovanović, V., Joksimović, S., van Staalduinen, J.-P., & Gašević, D. (2019). Counting clicks is not enough: Validating a theorized model of engagement in learning analytics. In LAK19: The 9th International Learning Analytics & Knowledge Conference (pp. 501-510). New York: Association for Computing Machinery.

Gašević, D., Dawson, S., & Siemens, G. (2015). Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 64-71.

Gašević, D., Mirriahi, N., Long, P., & Dawson, P. (2014). Editorial: Inaugural Issue of the Journal of Learning Analytics. Journal of Learning Analytics, 1(1), 1-2.

Giannakos, M. N., Sampson, D. G., & Kidziński, Ł. (2016). Introduction to smart learning analytics: foundations and developments in video-based learning. Smart Learning Environments, 3:12.

Glance, D. G., Hugh, P., & Barrett, R. (2014, November 23-26). Attrition patterns amongst participant groups in Massive Open Online Courses. Paper presented at the ASCILITE 2014 (Australasian Society for Computers in Learning in Tertiary Education), Dunedin.

Gómez-Aguilar, D. A., Hernández-García, A., García-Peñalvo, F. J., & Therón, R. (2015). Tap into visual analysis of customization of grouping of activities in elearning. Computers in Human Behavior, 47(June), 60-67. http://doi:10.1016/j.chb.2014.11.001

Greenland, S., & Moore, C. (2014). Patterns of online student enrolment and attrition in Australian open access online education: A preliminary case study. Open Praxis, 6(1), 45-54.

Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of MOOC videos. In L@S '14: Proceedings of the first ACM conference on Learning @ scale conference (pp. 41-50). Atlanta: Association for Computing Machinery.

Henrie, C., Bodily, R., Larsen, R., & Graham, C. R. (2018). Exploring the potential of LMS log data as a proxy measure of student engagement. Journal of Computing in Higher Education, 30, 344-362. http://doi:10.1007/s12528-017-9161-1

Henrie, C., Bodily, R., Manwaring, K. C., & Graham, C. R. (2015). Exploring Intensive Longitudinal Measures of Student Engagement in Blended Learning. International Review of Research in Open and Distributed Learning, 16(3), 133-155.

Hochheimer, C. J., Sabo, R. T., Krist, A. H., Day, T., Cyrus, J., & Woolf, S. H. (2016). Methods for evaluating respondent attrition in web-based surveys. Journal of medical Internet research, 18(11), e301. http://doi:10.2196/jmir.6342

Ismail, S. N., Hamid, S., & Chiroma, H. (2019). The utilization of learning analytics to develop student engagement model in learning management system. Journal of Physics: Conference Series, 1339. http://doi:10.1088/1742-6596/1339/1/012096

Jordon, M. M., & Duckett, N. D. (2018). Universities confront ‘tech disruption’: Perceptions of student engagement online using two learning management systems. The Journal of Public and Professional Sociology, 10(1), Article 4. Retrieved from

Karaksha, A., Grant, G., Anoopkumar-Dukie, S., Nirthanan, S. N., & Davey, A. K. (2013). Student engagement in pharmacology courses using online learning tools. American Journal of Pharmaceutical Education, 77(6), Article 125. http://doi:10.5688/ajpe776125

Kizilcec, R. F., Piech, C., & Schneider, E. (2013). Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In LAK '13: Proceedings of the Third International Conference on Learning Analytics and Knowledge (pp. 170-179). New York: Association for Computing Machinery.

Knestrick, J. M., Wilkinson, M. R., Pellathy, T. P., Lange-Kessler, J., Katz, R., & Compton, P. (2016). Predictors of retention of students in an online nurse practitioner program. The Journal for Nurse Practitioners, 12(9), 635-640. http://doi:10.1016/j.nurpra.2016.06.011

Li, N., Kidziński, Ł., Jermann, P., & Dillenbourg, P. (2015). MOOC video interaction patterns: What do they tell us? . In G. Conole, T. Klobučar, C. Rensing, J. Konert, & E. Lavoué (Eds.), Design for teaching and learning in a networked world. 10th European Conference on Technology Enhanced Learning (pp. 197-210). Cham: Springer International Publishing Switzerland.

Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an ‘‘early warning system’’ for educators: A proof of concept. Computers & Education, 54(2), 588-599. http://doi:10.1016/j.compedu.2009.09.008

Marks, A., Al-Ali, A., & Rietsema, K. (2016). Learning management systems: A shift toward learning and academic analytics. International Journal of Emerging Technologies in Learning, 11(4), 77-82. http://doi:10.3991/ijet.v11i04.5419

Pardo, A., Ellis, R. A., & Calvo, R. A. (2015). Combining observational and experiential data to inform the redesign of learning activities. In LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (pp. 305-309). New York: Association for Computing Machinery.

Poon, L. K. M., Kong, S.-C., Yau, T. S. H., Wong, M., & Ling, M. H. (2017). Learning analytics for monitoring students participation online: Visualizing navigational patterns on learning management system. In S. K. S. Cheung, L.-f. Kwok, W. W. K. Ma, L.-K. Lee, & H. Yang (Eds.), Blended Learning. New Challenges and Innovative Practices. 10th International Conference on Blended Learning (pp. 166-176). Hong Kong: Springer International Publishing.

Rienties, B., Toetenel, L., & Bryan, A. (2015). "Scaling up” learning design: impact of learning design activities on LMS behavior and performance. In LAK '15: Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (pp. 345-319). New York: Association for Computing Machinery.

Ruhanen, L., Whitford, M., & McLennan, C.-L. (2015). Indigenous tourism in Australia: Time for a reality check. Tourism Management, 48, 73-83. http://doi:10.1016/j.tourman.2014.10.017

Sheeran, P., & Webb, T. L. (2016). The intention–behavior gap. Social and Personality Psychology Compass, 10(9), 503-518. http://doi:10.1111/spc3.12265

Sherer, P., & Shea, T. (2011). Using online video to support student learning and engagement. College Teaching, 59(2), 56-59. http://doi:10.1080/87567555.2010.511313

Smith, D. (2010). Valuation of customer relationships: choice, application and results of various attrition analysis methodologies. Business Valuation Review, 29(2), 44-53. http://doi:10.5791/0897-1781-29.2.44

Stewart, M., Stott, T., & Nuttall, A.-M. (2011). Student engagement patterns over the duration of level 1 and level 3 geography modules: Influences on student attendance, performance and use of online resources. Journal of Geography in Higher Education, 35(1), 47-65. http://doi:10.1080/03098265.2010.498880

Venugopal, G., & Jain, R. (2015). Influence of learning management system on student engagement. In 2015 IEEE 3rd International Conference on MOOCs, Innovation and Technology in Education (MITE) (pp. 427-432). Amritsar: Institue of Electrical and Electronics Engineers (IEEE).

Vogt, K. L. (2016). Measuring student engagement using Learning Management Systems. (PhD). Ontario Institute for Studies in Education Leadership, Higher & Adult Education, University of Toronto, Toronto.

Vytasek, J. M., Patzak, A., & Winne, P. H. (2020). Analytics for student engagement. In M. Virvou, E. Alepis, G. A. Tsihrintzis, & L. C. Jain (Eds.), Machine learning paradigms. Advances in learning analytics (pp. 23-48). Cham: Springer Nature Switzerland.

Wang, Z., Bergin, C., & Bergin, D. A. (2014). Measuring engagement in fourth to twelfth grade classrooms: The classroom engagement inventory. School Psychology Quarterly, 29(4), 517-535. http://doi:10.1037/spq0000050

Williams, D., & Whiting, A. (2016). Exploring the relationship between student engagement, twitter, and a learning management system: A study of undergraduate marketing students. International Journal of Teaching and Learning in Higher Education, 28(3), 302-313.

Winstone, N., Bourne, J., Medland, E., Niculescu, I., & Rees, R. (2020). “Check the grade, log out”: students’ engagement with feedback in learning management systems. Assessment & Evaluation in Higher Education, 631-643.

Wu, S., Rizoiu, M.-A., & Xie, L. (2018, June 25-28). Beyond views: Measuring and predicting engagement in online videos. Paper presented at the The 12th International Association for the Advancement of Artificial Intelligence (AAAI) Conference on Web and Social Media (ICWSM-18), California.

Yang, D., Wen, M., & Rose, C. (2014, July 4-7). Peer Influence on attrition in Massive Open Online Courses. Paper presented at the The 7th International Conference on Educational Data Mining, London.

Yukselturk, E., Ozekes, S., & Türel, Y. K. (2014). Predicting dropout student: An application of data mining methods in an online education program. European Journal of Open, Distance and E-Learning, 17(1), 118-133.






Section II