Learning Analytics Methods, Benefits, and Challenges in Higher Education: A Systematic Literature Review

Sandra Nunn, John T. Avella, Therese Kanai, Mansureh Kebritchi

Abstract


Higher education for the 21st century continues to promote discoveries in the field through learning analytics (LA). The problem is that the rapid embrace of of LA diverts educators’ attention from clearly identifying requirements and implications of using LA in higher education. LA is a promising emerging field, yet higher education stakeholders need to become further familiar with issues related to the use of LA in higher education. Few studies have synthesized previous studies to provide an overview of LA issues in higher education. To address the problem, a systemic literature review was conducted to provide an overview of methods, benefits, and challenges of using LA in higher education. The literature review revealed that LA uses various methods including visual data analysis techniques, social network analysis, semantic, and educational data mining including prediction, clustering, relationship mining, discovery with models, and separation of data for human judgment to analyze data. The benefits include targeted course offerings, curriculum development, student learning outcomes, behavior and process, personalized learning, improved instructor performance, post-educational employment opportunities, and enhanced research in the field of education. Challenges include issues related to data tracking, collection, evaluation, analysis; lack of connection to learning sciences; optimizing learning environments, and ethical and privacy issues. Such a comprehensive overview provides an integrative report for faculty, course developers, and administrators about methods, benefits, and challenges of LA so that they may apply LA more effectively to improve teaching and learning in higher education.

Keywords


Learning analytics, big data, text mining, data analysis

Full Text:

PDF

References


AlShammari, I. A., Aldhafiri, M. D., & Al-Shammari, Z. (2013). A meta-analysis of educational data mining on improvements in learning outcomes. College Student Journal, 47(2), 326-333.

Althubaiti, A., & Alkhazim, M. (2014). Medical colleges in Saudi Arabia: Can we predict graduate numbers? Higher Education Studies, 4(3), 1-8.

Armayor, G.M., & Leonard, S. T. (2010). Graphic strategies for analyzing and interpreting curricular mapping data. American Journal of Pharmaceutical Education, 74(5), 1-10.

Arnold, K. E., & Pistilli, M. D. (2012, April 29). Course signals at Purdue: Using learning analytics to increase student success. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 267-270). New York, NY: ACM. doi: 10.1145/2330601.2330666

Baker, R. (2010). Data mining for education. International Encyclopedia of Education, 7, 112-118.

Baker, R., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1), 3—16.

Bhardwaj, B. K., & Pal, S. (2011). Data mining: A prediction for performance improvement using classification. International Journal of Computer Science and Information Security, 9(4), 136-140.

Bienkowski, M., Feng, M., & Means, B. (2012). Enhancing teaching and learning through educational data mining and learning analytics: An issue brief. U.S. Department of Education, Office of Educational Technology. Washington, D.C. Retrieved from http://www.ed.gov/technology.

Bottles, K., Begoli, E., & Worley, B. (2014). Understanding the pros and cons of big data analytics. Physician Executive, 40(4), 6-12.

Brown, M. (2012). Learning analytics: Moving from concept to practice. EDUCAUSE Learning Initiative. Retrieved from http://net.educause.edu/ir/library/pdf/ELIB1203.pdf

Buckingham Shum, S., & Ferguson, R. (2012). Social learning analytics. Educational Technology & Society, 15(3), 3-26.

Campbell, J. P., De Blois, P. B., & Oblinger, D. G. (2007). Academic analytics: A new tool for a new era. Educause Review, 42(4), 40-57. Retrieved from http://www.educause.edu/ero/article/academic-analytics-new-tool-new-era

Campbell, J. P., & Oblinger, D. G. (2007). Academic analytics. Educause. Retrieved from http://net.educause.edu/ir/library/pdf/pub6101.pdf.

Clow, D. (2012). The learning analytics cycle: Closing the loop effectively. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 134-138). New York, NY: ACM. doi:10.1145/2330601.2330636

Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, 18(6), 683-695. doi:http://dx.doi.org/10.1080/13562517.2013.827653

Cooper, H. (1988). The structure of knowledge synthesis: A taxonomy of literature

reviews. Knowledge in Society, 1, 104-126.

Dawson, S., & Siemens, G. (2014, September). Analytics to literacies: The development of a learning analytics framework for multiliteracies assessment. International Review of Research in Open and Distance Learning, 15(4), 284-305.

DiCerbo, K. E. (2014). Game-based assessment of persistence. Journal of Educational Technology & Society, 17(1), 17-28.

Dietz-Uhler, B., & Hurn, J. E. (2013, Spring). Using learning analytics to predict (and improve) student success: A faculty perspective. Journal of Interactive Online Learning, 12(1), 17-26.

Dyckhoff, A. L., Zielke, D., Bültmann, M., Chatti, M. A., & Schroeder, U. (2012). Design and implementation of a learning analytics toolkit for teachers. Journal of Educational Technology & Society, 15(3), 58-76.

EDUCAUSE. (2010). Next generation learning challenges: Learner analytics premises. EDUCAUSE Publications. Retrieved from http://www.educause.edu/Resources/NextGenerationLearningChalleng/215028

Elias, T. (2011). Learning analytics: Definitions, processes and potential (Report). Retrieved from http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf

Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304—317.

Fournier, H., Kop, R., & Sitlia, H. (2011). The value of learning analytics to networked learning on a personal learning environment. Proceedings of the Fourth International Conference on Learning Analytics and Knowledge (pp. 246-250). New York, NY: ACM. doi:10.1145/2567574.2567613

Grummon, P. T. H. (2009). Trends in higher education. Planning for Higher Education, 37(4), 48-57.

Hsinchun, C., Chiang, R. L., & Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS Quarterly, 36(4), 1165-1188.

Hung, J.L., Hsu, Y.C., & Rice, K. (2012). Integrating data mining in program evaluation of k-12 online education. Educational Technology & Society, 15(3), 27-41.

Hung, J., & Zhang, K. (2012). Examining mobile learning trends 2003-2008: A categorical meta-trend analysis using text mining techniques. Journal of Computing in Higher Education, 24(1), 1-17. doi:http://dx.doi.org/10.1007/s12528-011-9044-9

Jantawan, B., & Tsai, C. (2013). The application of data mining to build classification model for predicting graduate employment. International Journal of Computer Science and Information Security, 11(10), 1-7.

Johnson, L., Levine, A., Smith, R., & Stone, S. (2010). The horizon report: 2010 edition (Report). Retrieved from http://www.nmc.org/pdf/2010-Horizon-Report.pdf

Johnson, L., Smith, R., Willis, H., Levine, A., & Haywood, K., (2011). The horizon report: 2011 edition (Report). Retrieved from https://net.educause.edu/ir/library/pdf/HR2011.pdf

Kay, D., Korn, N., & Oppenheim, C. (2012). Legal, risk and ethical aspects of analytics in higher education (White Paper). Retrieved from http://publications.cetis.ac.uk/wp-content/uploads/2012/11/Legal-Risk-and-Ethical-Aspects-of-Analytics-in-Higher-Education-Vol1-No6.pdf

Kostoglou, V., Vassilakopoulos, M., & Koilias, C. (2013). Higher technological education specialties and graduates' vocational status and prospects. Education & Training, 55(6), 520-537. doi:http://dx.doi.org/10.1108/ET-03-2012-0026

Lias, T. E., & Elias, T. (2011). Learning analytics: The definitions, the processes, and the potential (Report). Retrieved from http://learninganalytics.net/LearningAnalyticsDefinitionsProcessesPotential.pdf

Mardikyan, S., & Badur, B. (2011). Analyzing teaching performance of instructors using data mining techniques. Informatics in Education, 10(2), 245-257.

McNeely, C. L., & Hahm, J. (2014). The big (data) bang: Policy, prospects, and challenges. Review of Policy Research, 31(4), 304-310.

Pardo, A., & Siemens, G. (2014). Ethical and privacy principles for learning analytics. British Journal of Educational Technology, 45(3), 438-450. doi:10.1111/bjet.12152

Pea, R. (2014). The learning analytics workgroup: A report on building the field of learning analytics for personalized learning at scale (Report). Retrieved from https://ed.stanford.edu/sites/default/files/law_report_complete_09-02-2014.pdf

Peer, P., Bule, J., Gros, J. Ž., & Štruc, V. (2013). Building cloud-based biometric services. Informatica, 37(2), 115-122.

Picciano, A.G. (2012). The evolution of big data and learning analytics in American higher education. Journal of Asynchronous Learning Networks, 16 (3), 9-20.

Picciano, A. G. (2014). Big data and learning analytics in blended learning environments: Benefits and concerns. International Journal of Artificial Intelligence and Interactive Multimedia, 2(7), 35-43.

Reyes, J. A. (2015). The skinny on big data in education: Learning analytics simplified. TechTrends, 59(2), 75-79.

Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 40(6), 601-618.

Sayed, M., & Jradi, F. (2014). Biometrics: Effectiveness and applications within the blended learning environment. Computer Engineering and Intelligent Systems, 5(5), 1-9.

Scheffel, M., Drachsler, H., Stoyanov S., & Specht, M. (2014). Quality indicators for learning analytics. Educational Technology & Society, 17(4), 117—132.

Sclater, N. (2014a, September 18). Code of practice “essential” for learning analytics. Retrieved from http://analytics.jiscinvolve.org/wp/2014/09/18/code-of-practice-essential-for-learning-analytics/

Sclater, N. (2014b, November). Code of practice for learning analytics: A literature review of the ethical and legal issues. Retrieved from http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf

Sharda, R., Adomako Asamoah, D., & Ponna, N. (2013). Research and pedagogy in business analytics: Opportunities and illustrative examples. Journal of Computing & Information Technology, 21(3), 171-183. doi:10.2498/cit.1002194

Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57(10), 1510-1529. doi: 10.1177/0002764213479366

Vahdat, M., Ghio, A., Oneto, L., Anguita, D., Funk, M., & Rauterberg, M. (2015). Advances in learning analytics and educational data mining. Proceedings from 2015 European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Belgium. Retrieved from http://www.idemployee.id.tue.nl/g.w.m.rauterberg/publications/ESANN2015paper1.pdf

West, D. M. (2012, September). Data mining, data analytics, and web dashboards. Governance Studies at Brookings, 1-10. Retrieved from Brookings.edu website at: http://www.brookings.edu/~/media/research/files/papers/2012/9/04%20education%20technology%20west/04%20education%20technology%20west.pdf

Xu, B., & Recker, M. (2012). Teaching analytics: A clustering and triangulation study of digital library user data. Journal of Educational Technology & Society, 15(3), 103-115.




DOI: http://dx.doi.org/10.24059/olj.v20i2.790

Bookmark and Share