Reducing learner disorientation in MOOC learning paths: Evidence from a hybrid recommender system
DOI:
https://doi.org/10.24059/olj.v30i3.4956Keywords:
adaptive learning, hybrid recommender system, learning path, learner disorientation, MOOC, personalized learningAbstract
Nonlinear learning, which aligns with natural thinking and addresses individual needs, has great potential for e-Learning. However, personalization in MOOCs through in-course recommenders remains limited, with only 18% adoption. Therefore, this research addresses on reducing learners’ disorientation by personalizing nonlinear learning paths in MOOCs using a hybrid recommender model, combining ontology and PrefixSpan-based sequential pattern mining. A Moodle-based prototype was tested in a quasi-online experiment with three groups: (1) a comparison group using sequential learning without recommendations, (2) a nonlinear learning group using an ontology- based recommender system, and (3) a nonlinear learning group using a hybrid recommender system. Of 209 registrants, 102 actively participated and provided data consent. The results show that the hybrid recommender group had a higher average efficiency than the ontology recommender group, achieved the highest effectiveness in two of three sub-learning outcomes, and scored best on 22 of 30 e-learning satisfaction variables. Hypothesis testing with analysis of covariance (ANCOVA) revealed significant differences between groups in e-Learning usability, Moodle preference, and learner behavior, though efficiency and effectiveness were not significantly different. Visualization using weighted directed graphs uncovered similar forward- linear learning path patterns across all groups, except for variations in the first three modules of the comparison group. These findings underscore the potential of hybrid recommender systems to improve learner satisfaction and usability in MOOCs. Future research should consider some confounding variables and configurations for hybrid recommender systems to maximize their benefits across various educational settings.
References
Aggarwal, C. C. (2016). Recommender Systems. Springer International Publishing Switzerland. https://doi.org/10.1007/978-3-319-29659-3
Alatrash, R., Chatti, M. A., Ain, Q. U., Fang, Y., Joarder, S., & Siepmann, C. (2024). ConceptGCN: Knowledge concept recommendation in MOOCs based on knowledge graph convolutional networks and SBERT. Computers and Education: Artificial Intelligence, 6, 100193. https://doi.org/10.1016/j.caeai.2023.100193
Al-Azawei, A., Parslow, P., & Lundqvist, K. (2016). Investigating the effect of learning styles in a blended e-learning system: An extension of the technology acceptance model (TAM). Australasian Journal of Educational Technology. https://doi.org/10.14742/ajet.2741
Al-Fraihat, D., Joy, M., Masa’deh, R., & Sinclair, J. (2020). Evaluating E-learning systems success: An empirical study. Computers in Human Behavior, 102, 67–86. https://doi.org/10.1016/j.chb.2019.08.004
Aliyu, O. A., Arasanmi, C., & Ekundayo, S. (2019). Do demographic characteristics moderate the acceptance and use of the Moodle learning system among business students? International Journal of Education and Development Using ICT, 15(1). https://www.learntechlib.org/p/209741/
Alnaji, L., & Karsu, Y. (2016). Effect of work status, experience and job sector on e-learning style in hybrid classrooms: GCC case study. European Journal of Business and Management, 8(12), 31–36.
Amin, S., Uddin, M. I., Alarood, A. A., Mashwani, W. K., Alzahrani, A., & Alzahrani, A. O. (2023). Smart e-learning framework for personalized adaptive learning and sequential path recommendations using reinforcement learning. IEEE Access, 11, 89769–89790. https://doi.org/10.1109/ACCESS.2023.3305584
Betoret, F. D. (2007). The influence of students’ and teachers’ thinking styles on student course satisfaction and on their learning process. Educational Psychology, 27(2), 219–234. https://doi.org/10.1080/01443410601066701
Blaschke, L. M., & Hase, S. (2016). Heutagogy: a holistic framework for creating twenty-first-century self-determined learners. In B. Gros, Kinshuk, & M. Maina (Eds.), The Future of ubiquitous learning: learning designs for emerging pedagogies (pp. 25–40). https://doi.org/10.1007/978-3-662-47724-3_2
Buchner, A. (2022). Understanding Moodle plugins. In Moodle 4 Administration (4th ed.). Packt Publishing. https://subscription.packtpub.com/book/web-development/9781801816724/2/ch02lvl1sec08/understanding-the-moodle-architecture
Cagiltay, N. E., Cagiltay, K., & Celik, B. (2020). An analysis of course characteristics, learner characteristics, and certification rates in MITx MOOCs. The International Review of Research in Open and Distributed Learning, 21(3), 121–139. https://doi.org/10.19173/irrodl.v21i3.4698
Capuano, N., & Toti, D. (2019). Experimentation of a smart learning system for law based on knowledge discovery and cognitive computing. Computers in Human Behavior, 92, 459–467. https://doi.org/10.1016/j.chb.2018.03.034
Colace, F., & De Santo, M. (2010). Ontology for e-learning: A Bayesian approach. IEEE Transactions on Education, 53(2), 223–233. https://doi.org/10.1109/TE.2009.2012537
Colace, F., De Santo, M., Lombardi, M., Mosca, R., & Santaniello, D. (2020). A multilayer approach for recommending contextual learning paths. Journal of Internet Services and Information Security (JISIS), 10(2), 91–102. https://doi.org/10.22667/JISIS.2020.05.31.091
Correa, M. I. (2017). Visual maps to navigate non-linear learning environments. https://static1.squarespace.com/static/58856d643a0411d1d209ba6f/t/59a075c9f14aa16f879c3b86/1503688141364/Visual+Maps+to+navigate+Non-Linear+Learning+Environments+.pdf
Coursera. (2021). Course Certificates. Learner Help Center. https://www.coursera.support/s/article/208280196-Course-Certificates?language=en_US
da Silva, F. L., Slodkowski, B. K., da Silva, K. K. A., & Cazella, S. C. (2023). A systematic literature review on educational recommender systems for teaching and learning: research trends, limitations and opportunities. Education and Information Technologies, 28(3), 3289–3328. https://doi.org/10.1007/s10639-022-11341-9
Deschênes, M. (2020). Recommender systems to support learners’ agency in a learning context: a systematic review. International Journal of Educational Technology in Higher Education, 17(1). https://doi.org/10.1186/s41239-020-00219-w
El-Bouhdidi, J., Ghailani, M., & Fennan, A. (2013). An intelligent architecture for generating evolutionary personalized learning paths based on learner profiles. Journal of Theoretical and Applied Information Technology, 57(2), 294–304.
Erdt, M., Fernandez, A., & Rensing, C. (2015). Evaluating recommender systems for technology enhanced learning: A quantitative survey. IEEE Transactions on Learning Technologies, 8(4), 326–344. https://doi.org/10.1109/TLT.2015.2438867
Fernando, W. (2020). Moodle quizzes and their usability for formative assessment of academic writing. Assessing Writing, 46(July), 100485. https://doi.org/10.1016/j.asw.2020.100485
Field, A., & Hole, G. (2002). How to design and report experiments. SAGE Publications Ltd.
George, G., & Lal, A. M. (2019). Review of ontology-based recommender systems in e-learning. Computers & Education, 142, 1–18. https://doi.org/10.1016/j.compedu.2019.103642
Gong, J., Liu, T. X., & Tang, J. (2021). How monetary incentives improve outcomes in MOOCs: Evidence from a field experiment. Journal of Economic Behavior & Organization, 190, 905–921. https://doi.org/10.1016/j.jebo.2021.06.029
Harley, J. M., Taub, M., Azevedo, R., & Bouchet, F. (2018). “Let’s set up some subgoals”: Understanding human-pedagogical agent collaborations and their implications for learning and prompt and feedback compliance. IEEE Transactions on Learning Technologies, 11(1), 54–66. https://doi.org/10.1109/TLT.2017.2756629
Hase, S. (2016). Self-determined Learning (heutagogy): Where Have We Come Since 2000?
Hasso Plattner Institute. (2023). Records on openHPI. https://open.hpi.de/pages/document_types
He, J., Liu, Z., & Kong, X. (2023). A novel link prediction approach for MOOC forum thread recommendation using personalized PageRank and machine learning. 2023 3rd International Conference on Educational Technology (ICET), 37–41. https://doi.org/10.1109/ICET59358.2023.10424086
Hnida, M., Idrissi, M. K., & Bennani, S. (2014). A formalism of the competency-based approach in adaptive learning systems. WSEAS Transactions on Information Science and Applications, 11, 83–93.
Huang, R., Spector, J. M., & Yang, J. (2019). Lecture notes in educational technology. Springer Nature Singapore. https://doi.org/https://doi.org/10.1007/978-981-13-6643-7
Ishaq, M., Abid, A., Farooq, M. S., Manzoor, M. F., Farooq, U., Abid, K., & Helou, M. A. (2023). Advances in database systems education: Methods, tools, curricula, and way forward. Education and Information Technologies, 28(3), 2681–2725. https://doi.org/https://doi.org/10.1007/s10639-022-11293-0
Isnawan, M. G. (2020). Kuasi-Eksperimen (Sudirman, Ed.). Nashir Al-Kutub Indonesia.
Jeng, Y., & Huang, Y.-M. (2019). Dynamic learning paths framework based on collective intelligence from learners. Computers in Human Behavior, 100, 242–251. https://doi.org/10.1016/j.chb.2018.09.012
Jerz, R. (2018, January 19). Amount of time spent in course. Moodle Forum. https://moodle.org/mod/forum/discuss.php?d=364261
Jevremovic, A., Shimic, G., Veinovic, M., & Ristic, N. (2017). IP addressing: Problem-based learning approach on computer networks. IEEE Transactions on Learning Technologies, 10(3), 367–378. https://doi.org/10.1109/TLT.2016.2583432
Jing, X., Patel, V. L., Cimino, J. J., Shubrook, J. H., Zhou, Y., Liu, C., & De Lacalle, S. (2022). The roles of a secondary data analytics tool and experience in scientific hypothesis generation in clinical research: Protocol for a mixed methods study. JMIR Research Protocols, 11(7), e39414. https://doi.org/10.2196/39414
Kaplan, H., Bar-Tov, I., Glassner, A., & Back, S. (2021). Promoting agentic engagement and heutagogy in Tomer elementary school in Beer Sheva, Israel. In S. Hase & L. M. Blaschke (Eds.), Unleashing the Power of Learner Agency (pp. 112–123). EdTech Books. https://edtechbooks.org/up
Kelly, E., Colella, J., & Sottosanti-Kusnir, A. (2024). Student perceptions of effective educators in online learning. Online Learning, 28(3). https://doi.org/10.24059/olj.v28i3.3534
Kintu, M. J., Zhu, C., & Kagambe, E. (2017). Blended learning effectiveness: the relationship between student characteristics, design features and outcomes. International Journal of Educational Technology in Higher Education, 14(1), 7. https://doi.org/10.1186/s41239-017-0043-4
Klašnja-Milićević, A., Ivanović, M., & Nanopoulos, A. (2015). Recommender systems in e-learning environments: a survey of the state-of-the-art and possible extensions. Artificial Intelligence Review, 44(4), 571–604. https://doi.org/10.1007/s10462-015-9440-z
Leroux, J. S., de Lannoy, L., & Baillie, C. P. T. (2024). Disorientation in community service-learning: A phenomenological inquiry. Pedagogy in Health Promotion, 10(1), 16–24. https://doi.org/10.1177/23733799231204023
Li, J., Tang, Y., Cao, M., & Hu, X. (2018). The moderating effects of discipline on the relationship between asynchronous discussion and satisfaction with MOOCs. Journal of Computers in Education, 5(3), 279–296. https://doi.org/10.1007/s40692-018-0112-2
Lowe, S. R., Dillon, C. O., Rhodes, J. E., & Zwiebach, L. (2013). Defining adult experiences. Journal of Adolescent Research, 28(1), 31–68. https://doi.org/10.1177/0743558411435854
Lowenthal, P., Bauer, C., & Chen, K.-Z. (2015). Student perceptions of online learning: An analysis of online course evaluations. American Journal of Distance Education, 29(2), 85–97. https://doi.org/10.1080/08923647.2015.1023621
Mamun, M. A. Al, Lawrie, G., & Wright, T. (2020). Instructional design of scaffolded online learning modules for self-directed and inquiry-based learning environments. Computers & Education, 144, 103695. https://doi.org/10.1016/j.compedu.2019.103695
Meşe, E., & Sevilen, Ç. (2021). Factors influencing EFL students’ motivation in online learning: A qualitative case study. Journal of Educational Technology and Online Learning, 4(1), 11–22. https://dergipark.org.tr/en/pub/jetol/issue/60134/817680
Moodle. (2024, March 4). Plugins FAQ. Installation. https://docs.moodle.org/403/en/Plugins_FAQ
Moore, R. L. (2020). Developing lifelong learning with heutagogy contexts critiques and challenges. Distance Education, 41(3), 381–401. https://doi.org/https://doi.org/10.1080/01587919.2020.1766949
Murad, D. F., Heryadi, Y., Isa, S. M., & Budiharto, W. (2020). Personalization of study material based on predicted final grades using multi-criteria user-collaborative filtering recommender system. Education and Information Technologies, 25(6), 5655–5668. https://doi.org/10.1007/s10639-020-10238-9
O’Brien, E., & Reale, J. (2021). Supporting learner agency using the pedagogy of choice. In S. Hase & L. M. Blaschke (Eds.), Unleashing the Power of Learner Agency (pp. 73–82). EdTech Books. https://edtechbooks.org/up
Oliveira, R. J. M. (2016). Recommender system for an e-learning platform [Master thesis]. University of Porto. https://sigarra.up.pt/fcup/pt/pub_geral.show_file?pi_doc_id=90364b
Peters, O. (2002). Distance education in transition: New trends and challenges. Bibliotheks- und Informationssystem der Carl von Ossietzky Universität Oldenburg (BIS). http://oops.uni-oldenburg.de/550/2/petdis02.pdf
Premlatha, K. R., & Geetha, T. V. (2015). Learning content design and learner adaptation for adaptive e-learning environment: A survey. Artificial Intelligence Review, 44(4), 443–465. https://doi.org/10.1007/s10462-015-9432-z
Rahayu, N. W., Adi, A. N., Ferdiana, R., & Kusumawardani, S. S. (2024). Nonlinear learning path: A systematic review. Turkish Online Journal of Distance Education, 25(4), 143–161. https://doi.org/10.17718/tojde.1349356
Rahayu, N. W., Ferdiana, R., & Kusumawardani, S. S. (2022). A systematic review of ontology use in E-Learning recommender system. Computers and Education: Artificial Intelligence, 3(6), 100047. https://doi.org/https://dx.doi.org/10.1016/j.caeai.2022.100047
Rahayu, N. W., Ferdiana, R., & Kusumawardani, S. S. (2023). A systematic review of learning path recommender systems. Education and Information Technologies, 28(6), 7437–7460. https://doi.org/10.1007/s10639-022-11460-3
Reichardt, C. S. (2019). Quasi-experimentation: a guide to design and analysis. The Guilford Press.
Robberecht, R. (2007). Interactive nonlinear learning environments. The Electronic Journal of E-Learning, 5(1), 59–68. https://academic-publishing.org/index.php/ejel/article/view/1503/1466
Rogers, J., & Révész, A. (2019). Experimental and quasi-experimental designs. In J. McKinley & H. Rose (Eds.), The Routledge Handbook of Research Methods in Applied Linguistics. Routledge. https://doi.org/10.4324/9780367824471
Rohman, A., & Muhtamiroh, S. (2022). Shaping the santri’s inclusive attitudes through learning in pesantren: A case study of pesantren Al-Anwar Sarang Rembang Indonesia. Journal of Educational and Social Research, 12(2), 367. https://doi.org/10.36941/jesr-2022-0058
Romero, L., Saucedo, C., Caliusco, Ma. L., & Gutiérrez, M. (2019). Supporting self-regulated learning and personalization using ePortfolios: a semantic approach based on learning paths. International Journal of Educational Technology in Higher Education, 16(1), 16. https://doi.org/10.1186/s41239-019-0146-1
Rootzén, H. (2015). Individualized learning through non-linear use of learning objects: With examples from math and stat. Proceedings of the 14th European Conference on E-Learning (ECEL 2015), 500–506. https://core.ac.uk/download/pdf/43254856.pdf
Roy, S., Beer, C., & Lawson, C. (2020). The importance of clarity in written assessment instructions. Journal of Further and Higher Education, 44(2), 143–155. https://doi.org/10.1080/0309877X.2018.1526259
Salehi, M., Kamalabadi, I. N., & Ghoushchi, M. B. G. (2013). An effective recommendation framework for personal learning environments using a learner preference tree and a GA. IEEE Transactions on Learning Technologies, 6(4), 350–363. https://doi.org/10.1109/TLT.2013.28
Sartor, L. B., Rosa, L. L. da, Madeira, K., Uggioni, M. L. R., Ferreira Filho, O. F., & Rosa, M. I. da. (2020). Undergraduate medical student’s perception about the progress testing. Revista Brasileira de Educação Médica, 44(2). https://doi.org/10.1590/1981-5271v44.2-20190286.ing
Shen, S., Chen, E., Liu, Q., Huang, Z., Huang, W., Yin, Y., Su, Y., & Wang, S. (2022). Monitoring student progress for learning process-consistent knowledge tracing. IEEE Transactions on Knowledge and Data Engineering, 1–15. https://doi.org/10.1109/TKDE.2022.3221985
Shih, Y.-C., Huang, P.-R., Hsu, Y.-C., & Chen, S. Y. (2012). A complete understanding of disorientation problems in web-based learning. Turkish Online Journal of Educational Technology-TOJET, 11(3), 1–13.
Sibanda, M., & Donnelly, S. (2014). The impact of e-learning on student performance: A case study of an entry-level module at a South African University. Mediterranean Journal of Social Sciences. https://doi.org/10.5901/mjss.2014.v5n9p478
Smith, M. R. (2018). Student perspectives on satisfaction with learning in Bachelor of Nursing programs in Australia: A constructivist grounded theory study [Doctoral dissertation, Griffith University]. http://hdl.handle.net/10072/381377
Strmečki, D., Bernik, A., & Radošević, D. (2015). Gamification in e-learning: Introducing gamified design elements into e-learning systems. Journal of Computer Science, 11(12), 1108–1117. https://doi.org/10.3844/jcssp.2015.1108.1117
Sudijono, A. (2015). Pengantar Evaluasi Pendidikan (Ed. 1). Rajawali Pers.
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. Springer Science+Business Media. https://doi.org/10.1007/978-1-4419-8126-4
Tarus, J. K., Niu, Z., & Mustafa, G. (2018). Knowledge-based recommendation: A review of ontology-based recommender systems for e-learning. Artificial Intelligence Review, 50(1), 21–48. https://doi.org/10.1007/s10462-017-9539-5
Tarus, J. K., Niu, Z., & Yousif, A. (2017). A hybrid knowledge-based recommender system for e-learning based on ontology and sequential pattern mining. Future Generation Computer Systems, 72, 37–48. https://doi.org/https://doi.org/10.1016/j.future.2017.02.049
Tazi, K., Azzouzi, S., & Charaf, M. E. H. (2023). An ontology-based recommender system for identifying learners’ confusion in MOOCs. 2023 7th IEEE Congress on Information Science and Technology (CiSt), 492–496. https://doi.org/10.1109/CiSt56084.2023.10409939
Tedre, M., & Moisseinen, N. (2014). Experiments in computing: A survey. The Scientific World Journal, 2014, 1–11. https://doi.org/10.1155/2014/549398
Tondello, G. F., Orji, R., & Nacke, L. E. (2017). Recommender systems for personalized gamification. Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization, 425–430. https://doi.org/10.1145/3099023.3099114
Vesin, B., Klasnja-Milicevic, A., Mirjana, I., & Budimac, Z. (2013). Applying recommender systems and adaptive hypermedia for e-learning personalization. Computing and Informatics, 32, 629–659. https://www.cai.sk/ojs/index.php/cai/article/view/1736
Wu, X., Jiang, H., Zhang, J., Wu, Z., Cheng, X., Yang, Q., & Zhou, Y. (2024). KAMC: Knowledge-aware meta-concept recommendation [Preprint]. Research Square. https://doi.org/10.21203/rs.3.rs-4521552/v1
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Nur W. Rahayu, Kurniawan Irwan N., Hidayah Indriana, Ferdiana Ridi, Kusumawardani Sri S.
This work is licensed under a Creative Commons Attribution 4.0 International License.
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
