A Pandemic of Busywork: Increased Online Coursework Following the Transition to Remote Instruction is Associated with Reduced Academic Achievement

Benjamin A Motz, Joshua D Quick, Julie A Wernert, Tonya A Miles

Abstract


Under normal circumstances, when students invest more effort in their schoolwork, they generally show evidence of improved academic achievement.  But when universities abruptly transitioned to remote instruction in Spring 2020, instructors assigned rapidly-prepared online learning activities, disrupting the normal relationship between effort and outcomes.  In this study, we examine this relationship using data observed from a large-scale survey of undergraduate students, from logs of student activity in the online learning management system, and from students’ estimated cumulative performance in their courses (n = 4,636).  We find that there was a general increase in the number of assignments that students were expected to complete following the transition to remote instruction, and that students who spent more time and reported more effort carrying out this coursework generally had lower course performance and reported feeling less successful.  We infer that instructors, under pressure to rapidly put their course materials online, modified their courses to include online busywork that did not constitute meaningful learning activities, which had a detrimental effect on student outcomes at scale.  These findings are discussed in contrast with other situations when increased engagement does not necessarily lead to improved learning outcomes, and in comparison with the broader relationship between effort and academic achievement.

Keywords


LMS, survey, COVID-19, student effort, self-regulated learning, busywork, student effort, student engagement, higher education

Full Text:

PDF

References


Beck, J. E., & Gong, Y. (2013). Wheel-Spinning: Students Who Fail to Master a Skill. Artificial Intelligence in Education, 431–440. https://doi.org/10.1007/978-3-642-39112-5_44

Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.) Metacognition: Knowing about Knowing, (pp. 185-205). Cambridge, MA: MIT Press.

Broadbent, J., & Poon, W. L. (2015). Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. The Internet and Higher Education, 27, 1–13. https://doi.org/10.1016/j.iheduc.2015.04.007

Bürkner, P.-C. (2017). brms: An R Package for Bayesian Multilevel Models Using Stan. In Journal of Statistical Software (Vol. 80, Issue 1). https://doi.org/10.18637/jss.v080.i01

Carey, K. (2020, March 13). Everybody Ready for the Big Migration to Online College? Actually, No. The New York Times. https://www.nytimes.com/2020/03/13/upshot/coronavirus-online-college-classes-unprepared.html

Cerezo, R., Sánchez-Santillán, M., Paule-Ruiz, M. P., & Núñez, J. C. (2016). Students’ LMS interaction patterns and their relationship with achievement: A case study in higher education. Computers & Education, 96, 42–54. https://doi.org/10.1016/j.compedu.2016.02.006

Conijn, R., Snijders, C., Kleingeld, A., & Matzat, U. (2017). Predicting Student Performance from LMS Data: A Comparison of 17 Blended Courses Using Moodle LMS. IEEE Transactions on Learning Technologies, 10(1), 17–29. https://doi.org/10.1109/TLT.2016.2616312

Cooper, H. (1989). Homework. Longman.

Day, T. (2015). Academic Continuity: Staying True to Teaching Values and Objectives in the Face of Course Interruptions. Teaching & Learning Inquiry: The ISSOTL Journal, 3(1), 75–89. https://doi.org/10.2979/teachlearninqu.3.1.75

Denwood, M. (2016). runjags: An R Package Providing Interface Utilities, Model Templates, Parallel Computing Methods and Additional Distributions for MCMC Models in JAGS. Journal of Statistical Software, Articles, 71(9), 1–25. https://doi.org/10.18637/jss.v071.i09

Dettmers, S., Trautwein, U., Lüdtke, O., Kunter, M., & Baumert, J. (2010). Homework works if homework quality is high: Using multilevel modeling to predict the development of achievement in mathematics. Journal of Educational Psychology, 102(2), 467–482. https://doi.org/10.1037/a0018453

Fang, Y., Nye, B., Pavlik, P., Xu, Y. J., Graesser, A., & Hu, X. (2017). Online Learning Persistence and Academic Achievement. International Educational Data Mining Society. https://eric.ed.gov/?id=ED596608

Gelman, A., & Rubin, D. B. (1992). Inference from Iterative Simulation Using Multiple Sequences. Statistical Science: A Review Journal of the Institute of Mathematical Statistics, 7(4), 457–472. https://doi.org/10.1214/ss/1177011136

Hoffman, M. D., & Gelman, A. (2014). The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research: JMLR, 15(1), 1593–1623. http://www.jmlr.org/papers/volume15/hoffman14a/hoffman14a.pdf

Joksimović, S., Gašević, D., Loughin, T. M., Kovanović, V., & Hatala, M. (2015). Learning at distance: Effects of interaction traces on academic achievement. Computers & Education, 87, 204–217. https://doi.org/10.1016/j.compedu.2015.07.002

Keith, T. Z. (1988). Using Path Analysis to Test the Importance of Manipulate Influences on School Learning. School Psychology Review, 17(4), 637–643. https://doi.org/10.1080/02796015.1988.12085381

Kirk-Johnson, A., Galla, B. M., & Fraundorf, S. H. (2019). Perceiving effort as poor learning: The misinterpreted-effort hypothesis of how experienced effort and perceived learning relate to study strategy choice. Cognitive Psychology, 115, 101237. https://doi.org/10.1016/j.cogpsych.2019.101237

Kruschke, J. (2014). Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan. Academic Press.

Kruschke, J. K., & Liddell, T. M. (2018a). Bayesian data analysis for newcomers. Psychonomic Bulletin & Review, 25(1), 155–177. https://doi.org/10.3758/s13423-017-1272-1

Kruschke, J. K., & Liddell, T. M. (2018b). The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychonomic Bulletin & Review, 25(1), 178–206. https://doi.org/10.3758/s13423-016-1221-4

Lang, C., Siemens, G., Wise, A., & Gasevic, D. (Eds.). (2017). Handbook of Learning Analytics. Society for Learning Analytics Research (SoLAR). https://doi.org/10.18608/hla17

Liddell, T. M., & Kruschke, J. K. (2018). Analyzing ordinal data with metric models: What could possibly go wrong? Journal of Experimental Social Psychology, 79, 328–348. https://doi.org/10.1016/j.jesp.2018.08.009

McDaniel, M. A., & Butler, A. C. (2011). A contextual framework for understanding when difficulties are desirable. Successful Remembering and Successful Forgetting: A Festschrift in Honor of Robert A. Bjork, 175–198.

Mervosh, S., & Swales, V. (2020, March 12). Colleges and Universities Cancel Classes and Move Online Amid Coronavirus Fears. The New York Times. https://www.nytimes.com/article/coronavirus-closings.html

Motz, B., Quick, J., Schroeder, N., Zook, J., & Gunkel, M. (2019). The validity and utility of activity logs as a measure of student engagement. Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 300–309. https://doi.org/10.1145/3303772.3303789

Ortagus, J. C., & Tyler Derreth, R. (2020). “Like Having a Tiger by the Tail”: A Qualitative Analysis of the Provision of Online Education in Higher Education. Teachers College Record, 122(2). https://www.tcrecord.org/Content.asp?contentid=23025

Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.13.3406

Roll, I., & Winne, P. H. (2015). Understanding, evaluating, and supporting self-regulated learning using learning analytics. Journal of Learning Analytics. https://epress.lib.uts.edu.au/index.php/JLA/article/view/4491

Rosário, P., Carlos Núñez, J., Vallejo, G., Nunes, T., Cunha, J., Fuentes, S., & Valle, A. (2018). Homework purposes, homework behaviors, and academic achievement. Examining the mediating role of students’ perceived homework quality. Contemporary Educational Psychology, 53, 168–180. https://doi.org/10.1016/j.cedpsych.2018.04.001

SchWeber, C. (2013). Survival Lessons: Academic Continuity, Business Continuity, and Technology. In P. Van den Bossche, W. H. Gijselaers, & R. G. Milter (Eds.), Facilitating Learning in the 21st Century: Leading through Technology, Diversity and Authenticity (pp. 151–163). Springer Netherlands. https://doi.org/10.1007/978-94-007-6137-7_9

Selwyn, N. (2020). Telling Tales on Technology: Qualitative Studies of Technology and Education. Routledge.

Stewart, B. L., Goodson, C., & Miertschin, S. L. (2010). Off-site faculty: Perspectives on online experiences. Quarterly Review of Distance Education, 11(3), 187.

Sun, M., Du, J., & Xu, J. (2019). Are homework purposes and student achievement reciprocally related? A longitudinal study. Current Psychology . https://doi.org/10.1007/s12144-019-00447-y

Tobin, T. J. (2020, March 25). Now Is Not the Time to Assess Online Learning. The Chronicle of Higher Education. https://www.chronicle.com/article/now-is-not-the-time-to-assess-online-learning/

Trautwein, U. (2007). The homework–achievement relation reconsidered: Differentiating homework time, homework frequency, and homework effort. Learning and Instruction, 17(3), 372–388. https://doi.org/10.1016/j.learninstruc.2007.02.009

Trautwein, U., & Köller, O. (2003). The relationship between homework and achievement—still much of a mystery. Educational Psychology Review, 15(2), 115–145.

Tze, V. M. C., Daniels, L. M., & Klassen, R. M. (2015). Evaluating the Relationship Between Boredom and Academic Outcomes: A Meta-Analysis. Educational Psychology Review, 28(1), 119–144. https://doi.org/10.1007/s10648-015-9301-y

Unizin Consortium. (2020). http://unizin.org/

Winne, P. H. (2017). Learning analytics for self-regulated learning. Handbook of Learning Analytics, 241–249. https://solaresearch.org/wp-content/uploads/2017/05/chapter21.pdf

You, J. W. (2016). Identifying significant indicators using LMS data to predict course achievement in online learning. The Internet and Higher Education, 29, 23–30. https://doi.org/10.1016/j.iheduc.2015.11.003

Yu, T., & Jo, I.-H. (2014). Educational technology approach toward learning analytics: relationship between student online behavior and learning performance in higher education. Proceedings of the Fourth International Conference on Learning Analytics And Knowledge, 269–270. https://doi.org/10.1145/2567574.2567594




DOI: http://dx.doi.org/10.24059/olj.v25i1.2475



Copyright (c) 2021 Benjamin A Motz, Joshua D Quick, Julie A Wernert, Tonya A Miles

License URL: https://creativecommons.org/licenses/by/4.0/