A Pandemic of Busywork: Increased Online Coursework Following the Transition to Remote Instruction is Associated with Reduced Academic Achievement
DOI:
https://doi.org/10.24059/olj.v25i1.2475Keywords:
LMS, survey, COVID-19, student effort, self-regulated learning, busywork, student engagement, higher educationAbstract
Under normal circumstances, when students invest more effort in their schoolwork, they generally show evidence of improved academic achievement. But when universities abruptly transitioned to remote instruction in Spring 2020, instructors assigned rapidly-prepared online learning activities, disrupting the normal relationship between effort and outcomes. In this study, we examine this relationship using data observed from a large-scale survey of undergraduate students, from logs of student activity in the online learning management system, and from students’ estimated cumulative performance in their courses (n = 4,636). We find that there was a general increase in the number of assignments that students were expected to complete following the transition to remote instruction, and that students who spent more time and reported more effort carrying out this coursework generally had lower course performance and reported feeling less successful. We infer that instructors, under pressure to rapidly put their course materials online, modified their courses to include online busywork that did not constitute meaningful learning activities, which had a detrimental effect on student outcomes at scale. These findings are discussed in contrast with other situations when increased engagement does not necessarily lead to improved learning outcomes, and in comparison with the broader relationship between effort and academic achievement.References
Beck, J. E., & Gong, Y. (2013). Wheel-Spinning: Students Who Fail to Master a Skill. Artificial Intelligence in Education, 431–440. https://doi.org/10.1007/978-3-642-39112-5_44
Bjork, R. A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe and A. Shimamura (Eds.) Metacognition: Knowing about Knowing, (pp. 185-205). Cambridge, MA: MIT Press.
Broadbent, J., & Poon, W. L. (2015). Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review. The Internet and Higher Education, 27, 1–13. https://doi.org/10.1016/j.iheduc.2015.04.007
Bürkner, P.-C. (2017). brms: An R Package for Bayesian Multilevel Models Using Stan. In Journal of Statistical Software (Vol. 80, Issue 1). https://doi.org/10.18637/jss.v080.i01
Carey, K. (2020, March 13). Everybody Ready for the Big Migration to Online College? Actually, No. The New York Times. https://www.nytimes.com/2020/03/13/upshot/coronavirus-online-college-classes-unprepared.html
Cerezo, R., Sánchez-Santillán, M., Paule-Ruiz, M. P., & Núñez, J. C. (2016). Students’ LMS interaction patterns and their relationship with achievement: A case study in higher education. Computers & Education, 96, 42–54. https://doi.org/10.1016/j.compedu.2016.02.006
Conijn, R., Snijders, C., Kleingeld, A., & Matzat, U. (2017). Predicting Student Performance from LMS Data: A Comparison of 17 Blended Courses Using Moodle LMS. IEEE Transactions on Learning Technologies, 10(1), 17–29. https://doi.org/10.1109/TLT.2016.2616312
Cooper, H. (1989). Homework. Longman.
Day, T. (2015). Academic Continuity: Staying True to Teaching Values and Objectives in the Face of Course Interruptions. Teaching & Learning Inquiry: The ISSOTL Journal, 3(1), 75–89. https://doi.org/10.2979/teachlearninqu.3.1.75
Denwood, M. (2016). runjags: An R Package Providing Interface Utilities, Model Templates, Parallel Computing Methods and Additional Distributions for MCMC Models in JAGS. Journal of Statistical Software, Articles, 71(9), 1–25. https://doi.org/10.18637/jss.v071.i09
Dettmers, S., Trautwein, U., Lüdtke, O., Kunter, M., & Baumert, J. (2010). Homework works if homework quality is high: Using multilevel modeling to predict the development of achievement in mathematics. Journal of Educational Psychology, 102(2), 467–482. https://doi.org/10.1037/a0018453
Fang, Y., Nye, B., Pavlik, P., Xu, Y. J., Graesser, A., & Hu, X. (2017). Online Learning Persistence and Academic Achievement. International Educational Data Mining Society. https://eric.ed.gov/?id=ED596608
Gelman, A., & Rubin, D. B. (1992). Inference from Iterative Simulation Using Multiple Sequences. Statistical Science: A Review Journal of the Institute of Mathematical Statistics, 7(4), 457–472. https://doi.org/10.1214/ss/1177011136
Hoffman, M. D., & Gelman, A. (2014). The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research: JMLR, 15(1), 1593–1623. http://www.jmlr.org/papers/volume15/hoffman14a/hoffman14a.pdf
Joksimović, S., Gašević, D., Loughin, T. M., Kovanović, V., & Hatala, M. (2015). Learning at distance: Effects of interaction traces on academic achievement. Computers & Education, 87, 204–217. https://doi.org/10.1016/j.compedu.2015.07.002
Keith, T. Z. (1988). Using Path Analysis to Test the Importance of Manipulate Influences on School Learning. School Psychology Review, 17(4), 637–643. https://doi.org/10.1080/02796015.1988.12085381
Kirk-Johnson, A., Galla, B. M., & Fraundorf, S. H. (2019). Perceiving effort as poor learning: The misinterpreted-effort hypothesis of how experienced effort and perceived learning relate to study strategy choice. Cognitive Psychology, 115, 101237. https://doi.org/10.1016/j.cogpsych.2019.101237
Kruschke, J. (2014). Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan. Academic Press.
Kruschke, J. K., & Liddell, T. M. (2018a). Bayesian data analysis for newcomers. Psychonomic Bulletin & Review, 25(1), 155–177. https://doi.org/10.3758/s13423-017-1272-1
Kruschke, J. K., & Liddell, T. M. (2018b). The Bayesian New Statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a Bayesian perspective. Psychonomic Bulletin & Review, 25(1), 178–206. https://doi.org/10.3758/s13423-016-1221-4
Lang, C., Siemens, G., Wise, A., & Gasevic, D. (Eds.). (2017). Handbook of Learning Analytics. Society for Learning Analytics Research (SoLAR). https://doi.org/10.18608/hla17
Liddell, T. M., & Kruschke, J. K. (2018). Analyzing ordinal data with metric models: What could possibly go wrong? Journal of Experimental Social Psychology, 79, 328–348. https://doi.org/10.1016/j.jesp.2018.08.009
McDaniel, M. A., & Butler, A. C. (2011). A contextual framework for understanding when difficulties are desirable. Successful Remembering and Successful Forgetting: A Festschrift in Honor of Robert A. Bjork, 175–198.
Mervosh, S., & Swales, V. (2020, March 12). Colleges and Universities Cancel Classes and Move Online Amid Coronavirus Fears. The New York Times. https://www.nytimes.com/article/coronavirus-closings.html
Motz, B., Quick, J., Schroeder, N., Zook, J., & Gunkel, M. (2019). The validity and utility of activity logs as a measure of student engagement. Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 300–309. https://doi.org/10.1145/3303772.3303789
Ortagus, J. C., & Tyler Derreth, R. (2020). “Like Having a Tiger by the Tail”: A Qualitative Analysis of the Provision of Online Education in Higher Education. Teachers College Record, 122(2). https://www.tcrecord.org/Content.asp?contentid=23025
Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.13.3406
Roll, I., & Winne, P. H. (2015). Understanding, evaluating, and supporting self-regulated learning using learning analytics. Journal of Learning Analytics. https://epress.lib.uts.edu.au/index.php/JLA/article/view/4491
Rosário, P., Carlos Núñez, J., Vallejo, G., Nunes, T., Cunha, J., Fuentes, S., & Valle, A. (2018). Homework purposes, homework behaviors, and academic achievement. Examining the mediating role of students’ perceived homework quality. Contemporary Educational Psychology, 53, 168–180. https://doi.org/10.1016/j.cedpsych.2018.04.001
SchWeber, C. (2013). Survival Lessons: Academic Continuity, Business Continuity, and Technology. In P. Van den Bossche, W. H. Gijselaers, & R. G. Milter (Eds.), Facilitating Learning in the 21st Century: Leading through Technology, Diversity and Authenticity (pp. 151–163). Springer Netherlands. https://doi.org/10.1007/978-94-007-6137-7_9
Selwyn, N. (2020). Telling Tales on Technology: Qualitative Studies of Technology and Education. Routledge.
Stewart, B. L., Goodson, C., & Miertschin, S. L. (2010). Off-site faculty: Perspectives on online experiences. Quarterly Review of Distance Education, 11(3), 187.
Sun, M., Du, J., & Xu, J. (2019). Are homework purposes and student achievement reciprocally related? A longitudinal study. Current Psychology . https://doi.org/10.1007/s12144-019-00447-y
Tobin, T. J. (2020, March 25). Now Is Not the Time to Assess Online Learning. The Chronicle of Higher Education. https://www.chronicle.com/article/now-is-not-the-time-to-assess-online-learning/
Trautwein, U. (2007). The homework–achievement relation reconsidered: Differentiating homework time, homework frequency, and homework effort. Learning and Instruction, 17(3), 372–388. https://doi.org/10.1016/j.learninstruc.2007.02.009
Trautwein, U., & Köller, O. (2003). The relationship between homework and achievement—still much of a mystery. Educational Psychology Review, 15(2), 115–145.
Tze, V. M. C., Daniels, L. M., & Klassen, R. M. (2015). Evaluating the Relationship Between Boredom and Academic Outcomes: A Meta-Analysis. Educational Psychology Review, 28(1), 119–144. https://doi.org/10.1007/s10648-015-9301-y
Unizin Consortium. (2020). http://unizin.org/
Winne, P. H. (2017). Learning analytics for self-regulated learning. Handbook of Learning Analytics, 241–249. https://solaresearch.org/wp-content/uploads/2017/05/chapter21.pdf
You, J. W. (2016). Identifying significant indicators using LMS data to predict course achievement in online learning. The Internet and Higher Education, 29, 23–30. https://doi.org/10.1016/j.iheduc.2015.11.003
Yu, T., & Jo, I.-H. (2014). Educational technology approach toward learning analytics: relationship between student online behavior and learning performance in higher education. Proceedings of the Fourth International Conference on Learning Analytics And Knowledge, 269–270. https://doi.org/10.1145/2567574.2567594
Downloads
Published
Issue
Section
License
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
