Piloting Learning Analytics in a Multidisciplinary Online Program
DOI:
https://doi.org/10.24059/olj.v25i2.2221Keywords:
learning analytics, online learning, online programs, technology implementationAbstract
Learning analytics is a recent innovation that holds promise for improving retention in fully online programs. However, only a few case studies exist to show models for and outcomes of the implementation of learning analytics systems. This paper reports on a learning analytics implementation in a fully online, multidisciplinary program designed for nontraditional students using a pilot planning group with stakeholders from various roles. The processes for selecting reports, creating communication structures, and evaluating outcomes are outlined. Overall, faculty and advisors were positive about the project and found the reports to be helpful. The results suggest that the actions most often triggered by learning analytics reports were emails to students. Evaluation results suggest that the implementation of the learning analytics program and the interventions enacted had a positive impact on student success, though we acknowledge that it is difficult to isolate the impact of the learning analytics tool itself. We also address several challenges that came along with the implementation of learning analytics including understanding the efficacy of interventions, data security, and ethics.
References
Allen, I. E., & Seaman, J. (2014). Grade change: Tracking online education in the United States. Babson Survey Research Group and Quahog Research Group. Retrieved from: http://www.onlinelearningsurvey.com/reports/gradechange.pdf.
Arnold, K.E., & Pistilli, M.D. (2012). Course Signals at Purdue: Using learning analytics to increase student success. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 267–270). New York, NY: ACM. doi:10.1145/2330601.2330666
Arnold, K. E., Lynch, G., Huston, D., Wong, L., Jorn, L., & Olsen, C. W. (2014). Building institutional capacities and competencies for systemic learning analytics initiatives. Proceedings of the Fourth International Conference on Learning Analytics And Knowledge (pp. 257–260). New York, NY: ACM. doi.org/10.1145/2567574.2567593
Beggs, T. A. (2000). Influences and barriers to the adoption of instructional technology. Proceedings of Fifth Annual Mid-South Instructional Technology. Middle Tennessee State University, USA. Retrieved February 20, 2020, from http://www.mtsu.edu/~itconf/proceed00/beggs/beggs.htm
Carver, L. B., Mukherjee, K., & Lucio, R. (2017). Relationship between grades earned and time in online courses. Online Learning, 21(4), 303-313.
Caulfield, M. (2013, November). What the course signals “Kerfuffle” is about, and what it means to you. Retrieved from http://www.educause.edu/blogs/mcaulfield/what-course-sig nals-kerfuffle-about-and-what-it-means-you
Chen, B. (2009). Barriers to adoption of technology-mediated distance education in higher-education institutions. Quarterly Review of Distance Education, 10(4), 333–338.
Cormack, A. (2016). A data protection framework for learning analytics. Journal of Learning Analytics, 91–106. https://doi.org/10.18608/jla.2016.31.6
Dawson, S., Jovanovic, J., Gašević, D., & Pardo, A. (2017). From prediction to impact: Evaluation of a learning analytics retention program. Proceedings of the Seventh International Learning Analytics & Knowledge Conference (pp. 474–478), New York, NY: ACM. https://doi.org/10.1145/3027385.3027405
Gregg, A., Wilson, B., & Parrish, P. (2018). Do no harm: A balanced approach to vendor relationships, learning analytics, and higher education. IDEA. Retrieved February 20, 2020, from https://files.eric.ed.gov/fulltext/ED588354.pdf
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81-112.
Heather, B. (2015). Predicting success: How predictive analytics are transforming student support and success programs. Community College Journal, 86(1), 14–18.
Hernández-Lara, A. B., Perera-Lluna, A., & Serradell-López, E. (2019). Applying learning analytics to students’ interaction in business simulation games. The usefulness of learning analytics to know what students really learn. Computers in Human Behavior, 92, 600–612. https://doi.org/10.1016/j.chb.2018.03.001
Ifenthaler, D., & Schumacher, C. (2016). Student perceptions of privacy principles for learning analytics. Educational Technology Research and Development, 64(5), 923–938.
Jones, K. M. L. (2019). “Just because you can doesn’t mean you should”: Practitioner perceptions of learning analytics ethics. Portal: Libraries and the Academy, 19(3), 407–428.
Kim, D., Yoon, M., Jo, I.-H., & Branch, R. M. (2018). Learning analytics to support self-regulated learning in asynchronous online courses: A case study at a women’s university in South Korea. Computers & Education, 127, 233–251.
Koç, M. (2017). Learning analytics of student participation and achievement in online distance education: A structural equation modeling. Educational Sciences: Theory & Practice, 17(6), 1893–1910.
Lawson, C., Beer, C., Rossi, D., Moore, T., & Fleming, J. (2016). Identification of ‘at risk’ students using learning analytics: The ethical dilemmas of intervention strategies in a higher education institution. Educational Technology Research and Development, 64(5), 957–968.
Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459.
Milliron, M. D., Malcolm, L., & Kil, D. (2014). Insight and action analytics: Three case studies to consider. Research & Practice in Assessment, 9, 70–89.
Mtebe, J. S., & Raisamo, R. (2014). Challenges and instructors’ intention to adopt and use open educational resources in higher education in Tanzania. The International Review of Research in Open and Distributed Learning, 15(1). 249-271.
Ngqulu, N. (2018, July). Investigating the Adoption and the Application of Learning Analytics in South African Higher Education Institutions (Heis). In International Conference on e-Learning (pp. 545-XVI). Academic Conferences International Limited.
Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Educational Technology & Society, 17(4), 49–64.
Porter, W. W., & Graham, C. R. (2016). Institutional drivers and barriers to faculty adoption of blended learning in higher education. British Journal of Educational Technology, 47(4), 748–762.
Porter, W. W., Graham, C. R., Spring, K. A., & Welch, K. R. (2014). Blended learning in higher education: Institutional adoption and implementation. Computers & Education, 75, 185–195.
Rienties, B., Herodotou, C., Olney, T., Schencks, M., & Boroowa, A. (2018). Making sense of learning analytics dashboards: A technology acceptance perspective of 95 teachers. International Review of Research in Open & Distance Learning, 19(5), 186–202.
Scott, K. M. (2013). Does a university teacher need to change e-learning beliefs and practices when using a social networking site? A longitudinal case study. British Journal of Educational Technology, 44(4), 571–580.
Seaman, J. E., Allen, I. E., & Seaman, J. (2018). Grade increase: Tracking distance education in the United States. Babson Survey Research Group. Retrieved from http://onlinelearningsurvey.com/reports/gradeincrease.pdf
Siemens, G., & Baker, R. S. J.d. (2012). Learning analytics and educational data mining: Towards communication and collaboration. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 252-254). New York, NY: ACM.
Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57(10), 1510–1529.
Smith, V. C., Lange, A., & Huston, D. R. (2012). Predictive modeling to forecast student outcomes and drive effective interventions in online community college courses. Journal of Asynchronous Learning Networks, 16, 3–51.
Swenson, J. (2015). Understanding ethical concerns in the design, application, and documentation of learning analytics in post-secondary education (Publication No. 3728165) [Doctoral dissertation, University of Minnesota]. ProQuest Dissertations and Theses Global.
Tanes, Z., Arnold, K. E., King, A. S., & Remnet, M. A. (2011). Using signals for appropriate feedback: Perceptions and practices. Computers & Education, 57(4), 2414-2422.
Wandera, S. (2017). Continuing the conversation about face-to-face, online, and blended learning a meta-analysis of empirical literature 2006-2017. (Publication No. 10637800) [Doctoral dissertation, University of Minnesota]. ProQuest Dissertations and Theses Global.
West, D., Huijser, H., & Heath, D. (2016). Putting an ethical lens on learning analytics. Educational Technology Research and Development, 64(5), 903–922.
Wintrup, J. (2017). Higher education’s panopticon? Learning analytics, ethics and student engagement. Higher Education Policy, 30(1), 87–103.
Zhou, G., & Xu, J. (2007). Adoption of educational technology: How does gender matter?. International Journal of Teaching and Learning in Higher Education, 19(2), 140–153.
Downloads
Published
Issue
Section
License
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
