Effects of Accountability Interventions on Environmental Educators' Engagement in MOOCs
DOI:
https://doi.org/10.24059/olj.v29i4.5215Keywords:
online learning, accountability, field experiment, cultural differencesAbstract
Many participants in online courses struggle to remain engaged and complete the course, often due to a lack of perceived accountability. This study experimentally tested the effects of accountability interventions designed to encourage learners to discuss their progress with someone either inside or outside the course who could help hold them accountable. In a Massive Open Online Course (MOOC) on Environmental Education Outcomes, developed for a global audience of environmental educators, 1,259 participants were randomly assigned to one of three conditions: accountability-inside, accountability-outside, or a business-as-usual control. Overall, the interventions did not significantly affect the average number of discussion posts or project completion rates. However, participants in the accountability-inside condition made significantly fewer comments across all countries. Notably, among U.S. participants, those in the accountability-outside made more posts and comments and had higher project completion rates than those in the accountability-inside condition. These findings offer insights into culturally contingent responses to accountability structures and point to a scalable approach for enhancing engagement in online learning environments. They also raise important theoretical questions about the psychological mechanisms through which accountability influences learner behavior.
References
2U. (2024). 2023 Transparency & outcomes report. https://press.edx.org/2023-transparency-outcomes-report
Adefila, A., Arrobbio, O., Brown, G., Robinson, Z., Spolander, G., Soliev, I., Willers, B., Morini, L., Padovan, D., & Wimpenny, K. (2021). Ecologized collaborative online international learning: Tackling wicked sustainability problems through education for sustainable development. Journal of Teacher Education for Sustainability, 23(1), 41–57. https://doi.org/10.2478/jtes-2021-0004
Ammeter, A. P., Douglas, C., Ferris, G. R., & Goka, H. (2004). A social relationship conceptualization of trust and accountability in organizations. Human Resource Management Review, 14(1), 47–65. https://doi.org/10.1016/j.hrmr.2004.02.003
Anderson, A., Huttenlocher, D., Kleinberg, J., & Leskovec, J. (2014). Engaging with massive online courses. Proceedings of the 23rd International Conference on World Wide Web, 687–698. https://doi.org/10.1145/2566486.2568042
Anthony H. Normore. (2004). The edge of chaos: School administrators and accountability. Journal of Educational Administration. https://doi.org/10.1108/09578230410517477
Arnesen, K. T., Leary, H., Graham, C. R., & McMurry, A. I. (2025). Encouraging self-regulation in an online university course for preservice teachers: An intervention study. Online Learning, 29(1), Article 1. https://doi.org/10.24059/olj.v29i1.4318
Askeroth, J. H., & Richardson, J. C. (2019). Instructor perceptions of quality learning in MOOCs they teach. Online Learning, 23(4), Article 4. https://doi.org/10.24059/olj.v23i4.2043
Bayeck, R. Y. (2016). Exploratory study of MOOC learners’ demographics and motivation: The case of students involved in groups. Open Praxis, 8(3). https://doi.org/10.5944/openpraxis.8.3.282
Bell, B., & Gilbert, J. (1996). Teacher development: A model from science education. Falmer Press, Taylor and Francis, Inc.
Bipp, T., Kleingeld, A., & Schelp, L. (2021). Achievement goals and goal progress as drivers of work engagement. Psychological Reports, 124(5), 2180–2202.
Blair, G, Cooper, J, Coppock, A, Humphreys, M, & Sonnet, L. (2025). Estimatr: Fast estimators for design-based inference. R package (Version 1.0.4) [Computer software]. https://github.com/declaredesign/estimatr
Borrella, I., Caballero-Caballero, S., & Ponce-Cueto, E. (2022). Taking action to reduce dropout in MOOCs: Tested interventions. Computers & Education, 179, 104412. https://doi.org/10.1016/j.compedu.2021.104412
Bovens, M. (2005). Public accountability. In Ferlie, E., Lynn, L. E., & Pollitt, C (Eds.), The Oxford Handbook of Public Management. Oxford University Press.
Canals, A., Cobarsí-Morales, J., & Ortoll, E. (2025). Social capital in online environments: Effects of social structure on academic performance in an online university. Online Learning, 29(1), Article 1. https://doi.org/10.24059/olj.v29i1.4388
Carnoy, M., & Loeb, S. (2002). Does external accountability affect student outcomes? A cross-state analysis. Educational Evaluation and Policy Analysis, 24(4), 305–331. https://doi.org/10.3102/01623737024004305
Castellanos-Reyes, D. (2021). The dynamics of a MOOC’s learner-learner interaction over time: A longitudinal network analysis. Computers in Human Behavior, 123, 106880. https://doi.org/10.1016/j.chb.2021.106880
Chen, K.-Z., & Yeh, H.-H. (2021). Acting in secret: Interaction, knowledge construction and sequential discussion patterns of partial role-assignment in a MOOC. Australasian Journal of Educational Technology, 37(6), Article 6. https://doi.org/10.14742/ajet.6979
Choi, S., & Chun, Y. H. (2021). Accountability and organizational performance in the public sector: Analysis of higher education institutions in Korea. Public Administration, 99(2), 353–370. https://doi.org/10.1111/padm.12683
Chu, R. J. (2010). How family support and Internet self-efficacy influence the effects of e-learning among higher aged adults – Analyses of gender and age differences. Computers & Education, 55(1), 255–264. https://doi.org/10.1016/j.compedu.2010.01.011
Cisel, M. T. (2018). Interactions in MOOCs: The hidden part of the iceberg. International Review of Research in Open and Distributed Learning, 19(5). https://doi.org/10.19173/irrodl.v19i5.3459
Coetzee, D., Fox, A., Hearst, M. A., & Hartmann, B. (2014). Should your MOOC forum use a reputation system? Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing, 1176–1187. https://doi.org/10.1145/2531602.2531657
Cogliano, M., Bernacki, M. L., Hilpert, J. C., & Strong, C. L. (2022). A self-regulated learning analytics prediction-and-intervention design: Detecting and supporting struggling biology students. Journal of Educational Psychology, 114(8), 1801–1816. https://doi.org/10.1037/edu0000745
Cummings, L. L., & Anton, R. J. (1990). The logical and appreciative dimensions of accountability. In Appreciative management and leadership: The power of positive thought and action in organizations (pp. 257–286). Jossey-Bass/Wiley.
Darling-Hammond, L. (2004). Standards, accountability, and school reform. Teachers College Record, 106(6), 1047–1085. https://doi.org/10.1111/j.1467-9620.2004.00372.x
Davis, D., Jivet, I., Kizilcec, R. F., Chen, G., Hauff, C., & Houben, G.-J. (2017). Follow the successful crowd: Raising MOOC completion rates through social comparison at scale. Proceedings of the Seventh International Learning Analytics & Knowledge Conference, 454–463. https://doi.org/10.1145/3027385.3027411
Dewi, R. C., & Riantoputra, C. D. (2019). Felt accountability: The role of personality and organizational factors. Journal of Management Development, 38(4), 312–322. https://doi.org/10.1108/JMD-12-2018-0375
Đurišić, M., & Bunijevac, M. (2017). Parental involvement as a important factor for successful education. Center for Educational Policy Studies Journal, 7(3), Article 3. https://doi.org/10.26529/cepsj.291
Frink, D. D., Hall, A. T., Perryman, A. A., Ranft, A. L., Hochwarter, W. A., Ferris, G. R., & Royle, M. T. (2008). Meso-level theory of accountability in organizations. In Research in Personnel and Human Resources Management (world; Vol. 27, pp. 177–245). Emerald Group Publishing Limited. https://doi.org/10.1016/S0742-7301(08)27005-2
Frink, D. D., & Klimoski, R. J. (1998). Toward a theory of accountability in organizations and human resource management. In Research in personnel and human resources management, Vol. 16 (pp. 1–51). Elsevier Science/JAI Press.
Gao, H., Ou, Y., Zhang, Z., Ni, M., Zhou, X., & Liao, L. (2021). The relationship between family support and e-learning engagement in college students: The mediating role of e-learning normative consciousness and behaviors and self-efficacy. Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.573779
Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. The Internet and Higher Education, 13(1), 5–9. https://doi.org/10.1016/j.iheduc.2009.10.003
Grothaus, C. M. (2023). Engagement in online learning among Thai and German students: The role of classmates, instructors, and technology across country contexts. Online Learning, 27(3), 188–208. https://doi.org/10.24059/olj.v27i3.3413
Guntur, M., & Purnomo, Y. W. (2024). A meta-analysis of self-regulated learning interventions: Studies on learning outcomes in online and blended environments. Online Learning, 28(3), Article 3. https://doi.org/10.24059/olj.v28i3.4025
Guo, D., Ramos, R., Zhao, Y., & Sun, Q. (2025). Design of interactive spaces for promoting parental involvement: Strategies used by EFL teachers. Online Learning, 29(2), Article 2. https://doi.org/10.24059/olj.v29i2.4683
Hall, A. T., Bowen, M. G., Ferris, G. R., Royle, M. T., & Fitzgibbons, D. E. (2007). The accountability lens: A new way to view management issues. Business Horizons, 50(5), 405–413. https://doi.org/10.1016/j.bushor.2007.04.005
Hall, A. T., & Ferris, G. R. (2011). Accountability and extra-role behavior. Employee Responsibilities and Rights Journal, 23(2), 131–144. https://doi.org/10.1007/s10672-010-9148-9
Hall, A. T., Frink, D. D., & Buckley, M. R. (2017). An accountability account: A review and synthesis of the theoretical and empirical research on felt accountability. Journal of Organizational Behavior, 38(2), 204–224. https://doi.org/10.1002/job.2052
Handoko, E., Gronseth, S. L., McNeil, S. G., Bonk, C. J., & Robin, B. R. (2019). Goal setting and MOOC completion: A study on the role of self-regulated learning in student performance in Massive Open Online Courses. The International Review of Research in Open and Distributed Learning, 20(3). https://doi.org/10.19173/irrodl.v20i4.4270
Hansen, J. D., & Reich, J. (2015). Democratizing education? Examining access and usage patterns in massive open online courses. Science, 350(6265), 1245–1248. https://doi.org/10.1126/science.aab3782
Henderikx, M. A., Kreijns, K., & Kalz, M. (2017). Refining success and dropout in massive open online courses based on the intention–behavior gap. Distance Education, 38(3), 353–368. https://doi.org/10.1080/01587919.2017.1369006
Henderikx, M. A., Kreijns, K., & Kalz, M. (2018). A classification of barriers that influence intention achievement in MOOCs. In V. Pammer-Schindler, M. Pérez-Sanagustín, H. Drachsler, R. Elferink, & M. Scheffel (Eds.), Lifelong Technology-Enhanced Learning (pp. 3–15). Springer International Publishing. https://doi.org/10.1007/978-3-319-98572-5_1
Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45–58. https://doi.org/10.1016/j.edurev.2014.05.001
Hoch, E., Scheiter, K., & Schüler, A. (2020). Implementation intentions related to self-regulatory processes do not enhance learning in a multimedia environment. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.00046
Hofstede, G. (2001). Culture’s consequences: Comparing values, behaviors, institutions and organizations across nations. SAGE Publications Inc. https://us.sagepub.com/en-us/nam/cultures-consequences/book9710
Hone, K. S., & El Said, G. R. (2016). Exploring the factors affecting MOOC retention: A survey study. Computers & Education, 98, 157–168. https://doi.org/10.1016/j.compedu.2016.03.016
Huang, L., Wang, K., Li, S., & Guo, J. (2023). Using WeChat as an educational tool in MOOC-based flipped classroom: What can we learn from students’ learning experience? Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.1098585
Huber, S. G., & Skedsmo, G. (2016). Teacher evaluation—Accountability and improving teaching practices. Educational Assessment, Evaluation and Accountability, 28(2), 105–109. https://doi.org/10.1007/s11092-016-9241-1
Jekabsone, I., & Gudele, I. (2023). Online adult education for sustainable development: The analysis of the consequences of the Covid-19 pandemic in Latvia. Journal of Teacher Education for Sustainability, 25(1), 155–167. https://doi.org/10.2478/jtes-2023-0010
Jitpaisarnwattana, N., Reinders, H., & Darasawang, P. (2021). Understanding the roles of personalization and social learning in a language MOOC through learning analytics. Online Learning, 25(4), Article 4. https://doi.org/10.24059/olj.v25i4.2509
Kellogg, S., Booth, S., & Oliver, K. (2014). A social network perspective on peer supported learning in MOOCs for educators. The International Review of Research in Open and Distributed Learning, 15(5). https://doi.org/10.19173/irrodl.v15i5.1852
Kucuk, S., & Richardson, J. C. (2019). A structural equation model of predictors of online learners’ engagement and satisfaction. Online Learning, 23(2), Article 2. https://doi.org/10.24059/olj.v23i2.1455
Lerís, D., Sein-Echaluce, M. L., Hernández, M., & Bueno, C. (2017). Validation of indicators for implementing an adaptive platform for MOOCs. Computers in Human Behavior, 72, 783–795. https://doi.org/10.1016/j.chb.2016.07.054
Levin, H. M. (1974). A conceptual framework for accountability in education. The School Review, 82(3), 363–391.
Levitt, R., Janta, B., & Wegrich, K. (2008). Accountability of teachers: Literature review. RAND Corporation. https://www.rand.org/pubs/technical_reports/TR606.html
Li, S., Du, J., & Yu, X. (2025). Dynamics of cMOOC learner interactions in different social media: A longitudinal network analysis. Interactive Learning Environments, 33(3), 2460–2478. https://doi.org/10.1080/10494820.2024.2412058
Limbu, Y. (2025). Factors associated with student engagement in online learning during COVID-19 pandemic: A systematic review. Online Learning, 29(1), Article 1. https://doi.org/10.24059/olj.v29i1.4221
Liu, Z., Zhang, N., Peng, X., Liu, S., Yang, Z., Peng, J., Su, Z., & Chen, J. (2022). Exploring the relationship between social interaction, cognitive processing and learning achievements in a MOOC discussion forum. Journal of Educational Computing Research, 60(1), 132–169. https://doi.org/10.1177/07356331211027300
Loh, H. S., Martins van Jaarsveld, G., Mesutoglu, C., & Baars, M. (2024). Supporting social interactions to improve MOOC participants’ learning outcomes: A literature review. Frontiers in Education, 9. https://doi.org/10.3389/feduc.2024.1345205
Moore, R. L., & Wang, C. (2021). Influence of learner motivational dispositions on MOOC completion. Journal of Computing in Higher Education, 33(1), 121–134. https://doi.org/10.1007/s12528-020-09258-8
Moore, R. L., Wang, C., Liu, L., & Lee, S. S. (2025). The knowledge quest: Navigating the Learner-Intention Continuum from curiosity to achievement. Education and Information Technologies, 30(9), 12991–13009. https://doi.org/10.1007/s10639-025-13327-9
Normore, A. H. (2004). The edge of chaos: School administrators and accountability. Journal of Educational Administration, 42(1), 55–77. https://doi.org/10.1108/09578230410517477
Parrish, P., & Linder-VanBerschot, J. A. (2010). Cultural dimensions of learning: Addressing the challenges of multicultural instruction. International Review of Research in Open and Distance Learning, 11(2), 1–19.
Rabin, E., Kalman, Y. M., & Kalz, M. (2019). An empirical investigation of the antecedents of learner-centered outcome measures in MOOCs. International Journal of Educational Technology in Higher Education, 16(1), 14. https://doi.org/10.1186/s41239-019-0144-3
Rahimi, A. R. (2024). A tri-phenomenon perspective to mitigate MOOCs’ high dropout rates: The role of technical, pedagogical, and contextual factors on language learners’ L2 motivational selves, and learning approaches to MOOC. Smart Learning Environments, 11(1), 11. https://doi.org/10.1186/s40561-024-00297-7
Rahmatollahi, M., & Mohamadi Zenouzagh, Z. (2021). Designing and validating an evaluation inventory for assessing teachers’ professional accountability. Language Testing in Asia, 11(1), 11. https://doi.org/10.1186/s40468-021-00128-3
Redmond, P., Alexsen, M., Maloney, S., Turner, J., Brown, A., Basson, M., Galligan, L., Lawrence, J., & Henderson, R. (2023). Student perceptions of online engagement. Online Learning, 27(1), Article 1. https://doi.org/10.24059/olj.v27i1.3320
Ruipérez-Valiente, J. A., Jenner, M., Staubitz, T., Li, X., Rohloff, T., Halawa, S., Turro, C., Cheng, Y., Zhang, J., Despujol, I., & Reich, J. (2020). Macro MOOC learning analytics: Exploring trends across global and regional providers. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 518–523. https://doi.org/10.1145/3375462.3375482
Sanz-Martínez, L., Martínez-Monés, A., Bote-Lorenzo, M. L., Muñoz-Cristóbal, J. A., & Dimitriadis, Y. (2017). Automatic group formation in a MOOC based on students’ activity criteria. In É. Lavoué, H. Drachsler, K. Verbert, J. Broisin, & M. Pérez-Sanagustín (Eds.), Data Driven Approaches in Digital Education (pp. 179–193). Springer International Publishing. https://doi.org/10.1007/978-3-319-66610-5_14
Schlenker, B. R., Britt, T. W., Pennington, J., Murphy, R., & Doherty, K. (1994). The triangle model of responsibility. Psychological Review, 101(4), 632–652. https://doi.org/10.1037/0033-295X.101.4.632
Schlenker, B. R., Weigold, M. F., & Doherty, K. (1991). Coping with accountability: Self-identification and evaluative reckonings. In Handbook of social and clinical psychology: The health perspective (pp. 96–115). Pergamon Press.
Schwartz, S. (2012). An overview of the Schwartz theory of basic values. Online Readings in Psychology and Culture, 2(1). https://doi.org/10.9707/2307-0919.1116
Semenova, T. (2022). Not only the intention to complete: The role of action-oriented intentions in MOOC completion. Technology, Knowledge and Learning, 27(3), 707–719. https://doi.org/10.1007/s10758-021-09534-1
Singh, A., & Ishrat, A. (2025). The role of social support in enhancing self-efficacy and learning satisfaction in online education among secondary school students. On the Horizon: The International Journal of Learning Futures, ahead-of-print(ahead-of-print). https://doi.org/10.1108/OTH-03-2025-0033
Sitzmann, T., & Ely, K. (2010). Sometimes you need a reminder: The effects of prompting self-regulation on regulatory processes, learning, and attrition. Journal of Applied Psychology, 95(1), 132–144. https://doi.org/10.1037/a0018080
Sitzmann, T., & Johnson, S. K. (2012). The best laid plans: Examining the conditions under which a planning intervention improves learning and reduces attrition. Journal of Applied Psychology, 97(5), 967–981. https://doi.org/10.1037/a0027977
Sonnenberg, C., & Bannert, M. (2019). Using Process Mining to examine the sustainability of instructional support: How stable are the effects of metacognitive prompting on self-regulatory behavior? Computers in Human Behavior, 96, 259–272. https://doi.org/10.1016/j.chb.2018.06.003
Sparke, M. (2017). Situated cyborg knowledge in not so borderless online global education: Mapping the geosocial landscape of a MOOC. Geopolitics, 22(1), 51–72. https://doi.org/10.1080/14650045.2016.1204601
Stein, R. E., Colyer, C. J., & Manning, J. (2016). Student accountability in team-based learning classes. Teaching Sociology, 44(1), 28–38. https://doi.org/10.1177/0092055X15603429
Tang, Y., Tseng, H., Kuo, Y.-C., Yeh, H.-T., & Liu, J. (2025). Setting the stage for success: Readiness and connectedness as antecedents of learning continuance intention in online learning. Online Learning, 29(1), Article 1. https://doi.org/10.24059/olj.v29i1.4402
Tetlock, P. E. (1992). The impact of accountability on judgment and choice: Toward a social contingency model. In M. P. Zanna (Ed.), Advances in Experimental Social Psychology (Vol. 25, pp. 331–376). Academic Press. https://doi.org/10.1016/S0065-2601(08)60287-7
von Hippel, P. (2017). When can you fit a linear probability model? More often than you think. Statistical Horizons. https://statisticalhorizons.com/when-can-you-fit/
Wei, W., Liu, J., Xu, X., Kolletar-Zhu, K., & Zhang, Y. (2023). Effective interactive engagement strategies for MOOC forum discussion: A self-efficacy perspective. PLOS ONE, 18(11), e0293668. https://doi.org/10.1371/journal.pone.0293668
Winrotte, S. (Director). (2022). Student accountability in online environments [Video recording]. https://onlineteachinghub.education.purdue.edu/student-accountability-in-online-environments/
Wong, J., Baars, M., He, M., de Koning, B. B., & Paas, F. (2021). Facilitating goal setting and planning to enhance online self-regulation of learning. Computers in Human Behavior, 124, 106913. https://doi.org/10.1016/j.chb.2021.106913
Xu, Q., & Richardson, J. (2024). The impact of social media and gamification of a mobile vocabulary learning app: Self-regulation and learning persistence. Online Learning, 28(4), Article 4. https://doi.org/10.24059/olj.v28i4.4592
Yang, B., & He, H. (2022). Learning alone yet together: Enhancing between-learner docial connectivity at scale. Proceedings of the Ninth ACM Conference on Learning @ Scale, 374–378. https://doi.org/10.1145/3491140.3528294
Zellars, K. L., Hochwarter, W. A., Lanivich, S. E., Perrewé, P. L., & Ferris, G. R. (2011). Accountability for others, perceived resources, and well being: Convergent restricted non-linear results in two samples. Journal of Occupational and Organizational Psychology, 84(1), 95–115. https://doi.org/10.1348/2044-8325.002004
Zhang, J., Skryabin ,Maxim, & and Song, X. (2016). Understanding the dynamics of MOOC discussion forums with simulation investigation for empirical network analysis (SIENA). Distance Education, 37(3), 270–286. https://doi.org/10.1080/01587919.2016.1226230
Zheng, S., Han, K., Rosson, M. B., & Carroll, J. M. (2016). The role of social media in MOOCs: How to use social media to enhance student retention. Proceedings of the Third (2016) ACM Conference on Learning @ Scale, 419–428. https://doi.org/10.1145/2876034.2876047
Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102_2
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Yue Li, Rene Kizilcec, Ji Yong Cho, Marianne Krasny
This work is licensed under a Creative Commons Attribution 4.0 International License.
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
