Using Learning Analytics to Understand K–12 Learner Behavior in Online Video-Based Learning

Authors

DOI:

https://doi.org/10.24059/olj.v28i1.3675

Keywords:

learning analytics, ICAP framework, video-based learning, interactive learning environments, active learning

Abstract

This research investigated the potential of learning analytics (LA) as a tool for identifying and evaluating student behaviours associated with active learning within an online learning environment. The study focused on the application of LA for evaluating student engagement in video-based learning – an area of inquiry highlighted in literature as needing further investigation. Results determined that the LA method could identify active learning behaviours, and that LA can play a valuable role in providing information on learner activity in autonomous learning environments. However, LA did not provide a complete picture of learner behaviour and viewing strategies, highlighting the importance of a multi-method approach to research on online learner behaviours. It is anticipated the accessible approach outlined in this study will provide educators with a viable means of using LA techniques to better understand how learners interact with course content and learning objects, greatly assisting the design of online learning programmes.

Author Biographies

Eamon Vale, Macquarie University

Eamon Vale is a PhD candidate in the Macquarie School of Education at Macquarie University. Eamon’s field of research is learning analytics and its potential for informing online teaching and learning decision-making. Eamon is also a senior manager at Keypath Education - an Australian ‘Online Program Management’ (OPM) company partnering with universities across Australia and Southeast Asia.

Garry Falloon, Federation University

Garry Falloon is the Research Professor of Early Years STEM education in the School of Education at Federation University, Victoria. Previously he was Professor of Education in the Macquarie School of Education, and in the Faculty of Education at Waikato University in Hamilton, New Zealand. His background includes 18 years teaching and leadership of primary and secondary schools in New Zealand, Education Foundation Manager at Telecom New Zealand (Spark), working for Microsoft Partners in Learning and as project lead for the New Zealand Government’s Digital Opportunities Projects. Garry has served on numerous advisory and writing panels for eLearning policy and curriculum development, industry and sector advisory boards, and the N.Z Prime Minister's Panel of Experts for Digital Learning.

References

Ahn, J., & McEachin, A. (2017). Student enrolment patterns and achievement in Ohio’s online charter schools. Educational Researcher, 46(1), 44–57. https://doi.org/10.3102/0013189x17692999

Baker Ryan Shaun and Inventado, P. S. (2014). Educational data mining and learning analytics. In B. Larusson Johann Ari and White (Ed.), Learning Analytics: From Research to Practice (pp. 61–75). Springer New York. https://doi.org/10.1007/978-1-4614-3305-7_4

Bonwell, C. C., & Eison, J. A. (1991). Active Learning: Creating excitement in the classroom. 1991 ASHE-ERIC Higher Education Reports. ERIC document: ED336049

Bowers, J., & Kumar, P. (2015). Students’ perceptions of teaching and social presence: A comparative analysis of face-to-face and online learning environments. International Journal of Web-Based Learning and Teaching Technologies (IJWLTT), 10(1), 27–44.

Brown, A., Lawrence, J., Basson, M., & Redmond, P. (2020). A conceptual framework to enhance student online learning and engagement in higher education. Https://Doi.Org/10.1080/07294360.2020.1860912, 41(2), 284–299. https://doi.org/10.1080/07294360.2020.1860912

Chen, Q., Chen, Y., Liu, D., Shi, C., Wu, Y., & Qu, H. (2016). PeakVizor: Visual analytics of peaks in video clickstreams from Massive Open Online Courses. IEEE Transactions on Visualization and Computer Graphics, 22(10), 2315–2330. https://doi.org/10.1109/TVCG.2015.2505305

Chi, M. T. H., Adams, J., Bogusch, E. B., Bruchok, C., Kang, S., Lancaster, M., Levy, R., Li, N., McEldoon, K. L., & Stump, G. S. (2018). Translating the ICAP theory of cognitive engagement into practice. Cognitive Science, 42(6), 1777–1832.

Chi, M. T., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219–243.

Chorianopoulos, K., Giannakos, M. N., Chrisochoides, N., & Reed, S. (2014). Open service for video learning analytics. Proceedings of the 2014 IEEE 14th International Conference on Advanced Learning Technologies, 28–30.

Dodson, S., Roll, I., Fong, M., Yoon, D., Harandi, N. M., & Fels, S. (2018). An active viewing framework for video-based learning. Proceedings of the Fifth Annual ACM Conference on Learning at Scale, 24.

de Freitas, S., Gibson, D., du Plessis, C., Halloran, P., Williams, E., Ambrose, M., Dunwell, I., & Arnab, S. (2015). Foundations of dynamic learning analytics: Using university student data to increase retention. British Journal of Educational Technology, 46(6), 1175–1188. https://doi.org/10.1111/bjet.12212

Ferguson, R., Clow, D., Griffiths, D., & Brasher, A. (2019). Moving forward with learning analytics: Expert views. Journal of Learning Analytics, 6(3), 43–59. https://doi.org/10.18608/jla.2019.63.8

García-Morales, V. J., Garrido-Moreno, A., & Martín-Rojas, R. (2021). The Transformation of Higher Education After the COVID Disruption: Emerging Challenges in an Online Learning Scenario. Frontiers in Psychology, 12, 196. https://doi.org/10.3389/FPSYG.2021.616059/BIBTEX

Giannakos, M. N., Chorianopoulos, K., & Chrisochoides, N. (2015). Making sense of video analytics: Lessons learned from clickstream interactions, attitudes, and learning outcome in a video-assisted course. International Review of Research in Open and Distributed Learning, 16(1), 260–283.

Jivet, I., Scheffel, M., Specht, M., & Drachsler, H. (2018). License to evaluate: Preparing learning analytics dashboards for educational practice. ACM International Conference Proceeding Series, 31–40. https://doi.org/10.1145/3170358.3170421

Kim, J., Guo, P. J., Seaton, D. T., Mitros, P., Gajos, K. Z., & Miller, R. C. (2014). Understanding in-video dropouts and interaction peaks in online lecture videos. Proceedings of the First ACM Conference on Learning@ Scale Conference, 31–40.

Kitto, K., Lupton, M., Davis, K., & Waters, Z. (2017). Designing for student-facing learning analytics. Australasian Journal of Educational Technology, 33(5).

Kizilcec, R. F., & Halawa, S. (2015). Attrition and achievement gaps in online learning. Proceedings of the Second (2015) ACM Conference on Learning@ Scale, 57–66.

Kleftodimos, A., & Evangelidis, G. (2016). Using open source technologies and open internet resources for building an interactive video based learning environment that supports learning analytics. Smart Learning Environments, 3(1), 1-23.

Lagerstrom, L., & Johanes, P., & Ponsukcharoen, U. (2015), The myth of the six-minute rule: Student engagement with online videos. Paper presented at 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 10.18260/p.24895

Lodge, J. M., Horvath, J. C., Horton, A., Kennedy, G., Venema, S., & Dawson, S. (2017). Designing videos for learning: Separating the good from the bad and the ugly. EARLI, Tampere, Finland, 28 August - 2 September 2017.

Ma, J., Han, X., Yang, J., & Cheng, J. (2015). Examining the necessary condition for engagement in an online learning environment based on learning analytics approach: The role of the instructor. The Internet and Higher Education, 24, 26-34.

Macfadyen, L. P., Lockyer, L., & Rienties, B. (2020). Learning Design and Learning Analytics: Snapshot 2020. Journal of Learning Analytics, 7(3), 6–12. https://doi.org/10.18608/jla.2020.73.2

Martin, F., Bacak, J., Polly, D., & Dymes, L. (2021). A systematic review of research on K12 online teaching and learning: Comparison of research from two decades 2000 to 2019. Https://Doi.Org/10.1080/15391523.2021.1940396. https://doi.org/10.1080/15391523.2021.1940396

Mayer, R. E. (2017). Using multimedia for e‐learning. Journal of Computer Assisted Learning, 33(5), 403–423.

McGowan, A., Hanna, P., & Anderson, N. (2016). Teaching programming: Understanding lecture capture YouTube analytics. Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education, 35–40.

Pardo, A. (2014). Designing learning analytics experiences. In B. Larusson, J. Ari, and B. White (Eds.), Learning Analytics: From Research to Practice (pp. 15–38). Springer New York. https://doi.org/10.1007/978-1-4614-3305-7_2

Rosé, C. P., McLaughlin, E. A., Liu, R., & Koedinger, K. R. (2019). Explanatory learner models: Why machine learning (alone) is not the answer. British Journal of Educational Technology, 50(6), 2943–2958. https://doi.org/10.1111/bjet.12858

Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In K. Sawyer (Ed.), Cambridge Handbook of the Learning Sciences (pp. 97-118). Cambridge University Press.

Stepanek, N. R., & Dorn, B. (2017, June). Automated analysis of lecture video engagement using student posts. The proceedings of the International Conference on Artificial Intelligence in Education, 565-569. Springer, Cham.

Stöhr, C., Stathakarou, N., Mueller, F., Nifakos, S., & McGrath, C. (2019). Videos as learning objects in MOOCs: A study of specialist and non-specialist participants’ video activity in MOOCs. British Journal of Educational Technology, 50(1), 166–176. https://doi.org/10.1111/bjet.12623

Swenson, J. A. (2015). Understanding Ethical Concerns in the Design, Application, and Documentation of Learning Analytics in Post-secondary Education (Doctoral dissertation, University of Minnesota).

UNESCO. (2020). With one in five learners kept out of school, UNESCO mobilizes education ministers to face the COVID-19 crisis. https://en.unesco.org/news/one-five-learners-kept-out-school-unesco-mobilizes-education-ministers-face-covid-19-crisis

Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98-110. https://doi.org/10.1016/j.chb.2018.07.027

Winne, P. H. (2018). Theorizing and researching levels of processing in self‐regulated learning. British Journal of Educational Psychology, 88(1), 9–20.

Woodworth, J. L., Raymond, M. E., Chirbas, K., Gonzalez, M., Negassi, Y., Snow, W., & van Donge, C. (2015). Online charter school study 2015. Center for Research on Educational Outcomes. https://credo.stanford.edu/sites/g/files/sbiybj6481/f/online_charter_study_final.pdf

Zilvinskis, J., Willis, J., & Borden, V. M. H. (2017). An overview of learning analytics. New Directions for Higher Education, 179, 9–17. https://doi.org/10.1002/he.20239

Downloads

Published

2024-03-01

Issue

Vol. 28 No. 1 (2024)

Section

Learner Engagement

License

Copyright (c) 2023 Eamon Vale, Garry Falloon

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.

This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.

Author(s) hold copyright in their work, and retain publishing rights without restrictions