Enhancing Online STEM Education:  Impacts of Pre-Released Exam Materials  and Remote Proctoring

Meta Landys

doi:10.24059/olj.v29i3.4743

Authors

Meta Landys Oregon State University

DOI:

https://doi.org/10.24059/olj.v29i3.4743

Keywords:

achievement gap, attitudes, grades, motivation, online education, exams, remote proctors, socioeconomic status, STEM, test anxiety

Abstract

Undergraduate STEM courses commonly evaluate students using timed, high-stakes “traditional” exams. However, the use of such exams may harm cognitive development, attitudes of students toward their discipline, and various categories relating to motivation. Also, implementation of such traditional exams may perpetuate demographic achievement gaps. I hypothesized that the pre-release of exam materials could address such concerns. To this end, I examined the effect of pre-released exam materials on broad student outcomes in a distance education biology course, paying attention to exam performance, attitudes towards the discipline, and various motivational components related to learning. I found that in comparison to a traditional exam system, the pre-release of exam materials did not improve exam performance, attitudes, or motivation of online biology students. Also, pre-released exams did not rescue the achievement gap associated with lower socioeconomic status (SES). Nevertheless, qualitative responses of students to an open-ended survey indicated that pre-released exams offered a unique experience – with reduced anxiety as a major benefit. These data suggest that a pre-released exam system could be utilized to decrease test-taking anxiety in online STEM courses, without exacerbating SES-related achievement gaps. In fact, the pre-release of exam information may increase the authenticity of exams, as it mirrors work in real-life STEM careers. In parallel to the above, I found that remote online proctoring requirements imposed penalties on student exam performance, but maintained favorable attitudes toward the discipline. This suggests that the impact of remote proctors for online STEM students should be more carefully considered.

References

Artino, A. R., Jr., & Stephens, J. M. (2009). Beyond grades in online learning: Adaptive profiles of academic self-regulation among naval academy undergraduates. Journal of Advanced Academics, 20(4), 568-601. https://doi.org/10.1177/1932202X09020004

Ballen, C. J., Salehi, S., & Cotner, S. (2017). Exams disadvantage women in introductory biology. PLOS ONE, 12(10), e0186419. https://doi.org/10.1371/journal.pone.0186419

Barber, P. H., Hayes, T. B., Johnson, T. L., & Márquez-Magaña, L.; 10,234 signatories. (2020). Systemic racism in higher education. Science, 369(6510), 1440–1441. https://doi.org/10.1126/science.abd7140

Barber, P. H., Shapiro, C., Jacobs, M. S., Avilez, L., Brenner, K. I., Cabral, C., … & Levis-Fitzgerald, M. (2021). Disparities in remote learning faced by first-generation and underrepresented minority students during COVID-19: Insights and opportunities from a remote research experience. Journal of Microbiology & Biology Education, 22(1):22.1.54. https://doi.org/10.1128/jmbe.v22i1.2457

Bawa, P. (2016). Retention in online courses: Exploring issues and solutions - A literature review. SAGE Open, 6(1). https://doi.org/10.1177/2158244015621777

Beasley, M. A., & Fischer, M. J. (2012). Why they leave: The impact of stereotype threat on the attrition of women and minorities from science, math and engineering majors. Social Psychology of Education, 15(4), 427–448. https://doi.org/10.1007/s11218-012-9185-3

Black, P., & Wiliam, D. (2010). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 92(1), 81-90. https://doi.org/10.1177/003172171009200119

Brazeal, K. R., & Couch, B. A. (2017). Student buy-in toward formative assessments: The influence of student factors and importance for course success. Journal of Microbiology & Biology Education, 18(1), 18.1.6. https://doi.org/10.1128/jmbe.v18i1.1235

Burtner, J. (2005). The use of discriminant analysis to investigate the influence of non-cognitive factors on engineering school persistence. Journal of Engineering Education, 94(3), 335–338. https://doi.org/10.1002/j.2168-9830.2005.tb00858.x

Campos-Sánchez, A., López-Núñez, J. A., Carriel, V., Martín-Piedra, M.-Á., Sola, T., & Alaminos, M. (2014). Motivational component profiles in university students learning histology: A comparative study between genders and different health science curricula. BMC Medical Education, 14, 46. https://doi.org/10.1186/1472-6920-14-46

Cho, M. H., & Shen, D. M. (2013). Self-regulation in online learning. Distance Education, 34(3), 290-301. https://doi.org/10.1080/01587919.2013.835770

Chung, J., McKenzie, S., Schweinsberg, A., & Mundy, M. E. (2022). Correlates of academic performance in online higher education: A Systematic Review. Frontiers in Education, 7, 820567. https://doi.org/10.3389/feduc.2022.820567

Cleveland, L. M., Olimpo, J. T., & DeChenne-Peters, S. E. (2017). Investigating the relationship between instructors’ use of active-learning strategies and students’ conceptual understanding and affective changes in introductory biology: A comparison of two active-learning environments. CBE—Life Sciences Education, 16(2), ar19. https://doi.org/10.1187/cbe.16-06-0181

Cohen, G. L., Garcia, J., Apfel, N., & Master, A. (2006). Reducing the racial achievement gap: A social-psychological intervention. Science, 313(5791), 1307–1310. https://doi.org/10.1126/science.1128317

Cotner, S., & Ballen, C. J. (2017). Can mixed assessment methods make biology classes more equitable? PLOS ONE, 12(12), e0189610. https://doi.org/10.1371/journal.pone.0189610

Crisp, G., Nora, A., & Taggart, A. (2009). Student characteristics, pre-college, college, and environmental factors as predictors of majoring in and earning a STEM degree: An analysis of students attending a Hispanic serving institution. American Educational Research Journal, 46(4), 924-942. https://doi.org/10.3102/0002831209349460

Daffin, L. W., Jr., & Jones, A. A. (2018). Comparing student performance on proctored and non-proctored exams in online psychology courses. Online Learning, 22(1), 131–145. https://doi.org/10.24059/olj.v22i1.1079

Delahunty, J., Verenikina, I. & Jones, P. (2014). Socio-emotional connections: Identity, belonging and learning in online interactions. A literature review. Technology, Pedagogy and Education, 23(2), 243-265. https://doi.org/10.1080/1475939X.2013.813405

Eddy, S. L., Brownell, S. E., & Wenderoth, M. P. (2014). Gender gaps in achievement and participation in multiple introductory biology classrooms. CBE—Life Sciences Education, 13(3), 478–492. https://doi.org/10.1187/cbe.13-10-0204

Estrada, M., Burnett, M., Campbell, A. G., Campbell, P. B., Denetclaw, W. F., Gutiérrez, … & Zavala, M. (2017). Improving underrepresented minority student persistence in STEM. CBE—Life Sciences Education, 15(3), es5. https://doi.org/10.1187/cbe.16-01-0038

Glynn, S. M., Brickman, P., Armstrong, N., & Taasoobshirazi, G. (2011). Science Motivation Questionnaire II: Validation with science majors and nonscience majors. Journal of Research in Science Teaching, 48(10), 1159–1176. https://doi.org/10.1002/tea.20442

Haak, D. C., HilleRisLambers, J., Pitre, E., & Freeman, S. (2011). Increased structure and active learning reduce the achievement gap in introductory biology. Science, 332(6034), 1213–1216. https://doi.org/10.1126/science.1204820

Harmon, O. R., Lambrinos, J., & Kennedy, P. (2008). Are online exams an invitation to cheat? Journal of Economic Education, 39(2), 116–125. https://doi.org/10.3200/JECE.39.2.116-125

Hattie, J., Biggs, J., & Purdie, N. (1996). Effects of learning skills interventions on student learning: A meta-analysis. Review of Educational Research, 66(2), 99-136. https://doi.org/10.3102/00346543066002099

Hofstra, B., Kulkarni, V. V., Munoz-Najar Galvez, S., He, B., Jurafsky, D., & McFarland, D. A. (2020). The diversity–innovation paradox in science. Proceedings of the National Academy of Sciences, 117(17), 9284–9291. https://doi.org/10.1073/pnas.1915378117

Jegede, O. J., & Kirkwood, J. (1994). Students’ anxiety in learning through distance education. Distance Education, 15(2), 279–290. https://doi.org/10.1080/0158791940150207

Jensen, J. L., McDaniel, M. A., Woodard, S. M., & Kummer, T. A. (2014). Teaching to the test…or testing to teach: Exams requiring higher order thinking skills encourage greater conceptual understanding. Educational Psychology Review, 26(2), 307–329. https://doi.org/10.1007/s10648-013-9248-9

Johnson, M., & Crisp, V. (2009). An exploration of the effect of pre-release examination materials on classroom practice in the UK. Research in Education, 82(1), 47–59. https://doi.org/10.7227/RIE.82.4

Liu, A., Shapiro, C., Gregg, J., Levis-Fitzgerald, M., Sanders O’Leary, E., & Kennison, R. L. (2022). Scaling up a life sciences college career exploration course to foster STEM confidence and career self-efficacy. Research in Science & Technological Education, 42(2), 378–394. https://doi.org/10.1080/02635143.2022.2083599

Muljana, P. S., & Luo, T. (2019). Factors contributing to student retention in online learning andrecommended strategies for improvement: A systematic literature review. Journal of Information Technology Education: Research, 18, 19-57. https://doi.org/10.28945/4182

Oseguera, L., Park, H. J., De Los Rios, M. J., Aparicio, E. M., & Johnson, R. (2019). Examining the role of scientific identity in Black student retention in a STEM scholar program. The Journal of Negro Education, 88(3), 229–248. https://doi.org/10.7709/jnegroeducation.88.3.0229

Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: The National Academies Press. https://doi.org/10.17226/10019

Perkins, K. K., Adams, W. K., Pollock, S. J., Finkelstein, N. D., & Wieman, C. E. (2005). Correlating student beliefs with student learning using the Colorado Learning Attitudes about Science Survey. In: Marx, J., Heron, P., & Franklin, S. (Eds.), 2004 Physics Education Research Conference (pp. 61–64). Melville, NY. American Institute of Physics. https://doi.org/10.1063/1.2084701

Portela-Parra, E. T., & Leung, C. W. (2019). Food insecurity is associated with lower cognitive functioning in a national sample of older adults. The Journal of Nutrition, 149(10), 1812–1817. https://doi.org/10.1093/jn/nxz120

Putwain, D. (2008). Test anxiety and GCSE performance: The effect of gender and socio-economic background. Educational Psychology in Practice, 24(4), 319–334. https://doi.org/10.1080/02667360802488765

Redish, E. F., Saul, J. M., & Steinberg, R. N. (1998). Student expectations in introductory physics. American Journal of Physics, 66(3), 212–224. https://doi.org/10.1119/1.18847

Richardson, R. C., & Skinner, E. F. (1992). Helping first-generation minority students achieve degrees. New Directions for Community Colleges, 80, 29–43. https://doi.org/10.1002/cc.36819928005

Salehi, S., Cotner, S., Azarin, S. M., Carlson, E. E., Driessen, M., Ferry, V. E., … & Ballen, C. J. (2019). Gender performance gaps across different assessment methods and the underlying mechanisms: The case of incoming preparation and test anxiety. Frontiers in Education, 4, 107. https://doi.org/10.3389/feduc.2019.00107

Salehi, S., Cotner, S., & Ballen, C. J. (2020). Variation in incoming academic preparation: Consequences for minority and first-generation students. Frontiers in Education, 5, 552364. https://doi.org/10.3389/feduc.2020.552364

Seipp, B. (1991). Anxiety and academic performance: A meta-analysis of findings. Anxiety Research, 4(1), 27–41. https://doi.org/10.1080/08917779108248762

Selco, J. I., & Habbak, M. (2021). STEM students' perceptions on emergency online learning during the COVID-19 pandemic: Challenges and successes. Education Sciences, 11(12), 799. https://doi.org/10.3390/educsci11120799

Semsar, K., Knight, J. K., Birol, G., & Smith, M. K. (2011). The Colorado Learning Attitudes about Science Survey (CLASS) for use in biology. CBE—Life Sciences Education, 10(3), 268–278. https://doi.org/10.1187/cbe.10-10-0133

Shaikh, U. U., & Asif, Z. (2022). Persistence and dropout in higher online education: Review and categorization of factors. Frontiers in Psychology, 13, 902070. https://doi.org/10.3389/fpsyg.2022.902070

Shay, J., & Reese, M., (2004). Understanding why students select online courses and criteria they use in making that selection. International Journal of Instructional Technology and Distance Learning, 1(5). Retrieved from http://www.itdl.org/Journal/May_04/article03.htm

Smith, M. K., Wood, W. B., & Knight, J. K. (2008). The Genetics Concept Assessment: A new concept inventory for gauging student understanding of genetics. CBE—Life Sciences Education, 7(4), 422–430. https://doi.org/10.1187/cbe.08-08-0045

Stanger-Hall, K. F. (2012). Multiple-choice exams: An obstacle for higher-level thinking in introductory science classes. CBE—Life Sciences Education, 11(3), 294–306. https://doi.org/10.1187/cbe.11-11-0100

Stark, E. (2019). Examining the role of motivation and learning strategies in student success in online versus face-to-face courses. Online Learning Journal, 23(3), 234-251. https://doi.org/10.24059/olj.v23i3.1556

Theobald, E. J., Hill, M. J., Tran, E., Agrawal, S., Arroyo, E. N., Behling, S., … & Freeman, S. (2020). Active learning narrows achievement gaps for underrepresented students in undergraduate science, technology, engineering, and math. Proceedings of the National Academy of Sciences, 117(12), 6476–6483. https://doi.org/10.1073/pnas.1916903117

Tuckman, B. W., & Kennedy, G. J. (2011). Teaching learning strategies to increase success of first-term college students. The Journal of Experimental Education, 79, 478-504. https://doi.org/10.1080/00220973.2010.512318

Vogel, S., & Schwabe, L. (2016). Learning and memory under stress: Implications for the classroom. npj Science of Learning, 1, 16011. https://doi.org/10.1038/npjscilearn.2016.11

Walker, A., Aguiar, N. R., Soicher, R. N., Kuo, Y.-C., Resig, J. (2024). Exploring the relationship between motivation and academic performance among online and blended learners: A meta-analytic review. Online Learning Journal, 28(4), 76-116. https://doi.org/10.24059/olj.v28i4.4602

Wiggins, G. (1998). Educative assessment. Designing assessments to inform and improve student performance. San Francisco, CA: Jossey-Bass Publishers.

Wiggins, B. L., Lily, L. S., Busch, C. A., Landys, M. M., Shlichta, J. G., Shi, T., & Ngwenyama, T. R. (2023). Public exams may decrease anxiety and facilitate deeper conceptual thinking. Journal of STEM Education: Innovations and Research, 24(2). Retrieved from https://www.jstem.org/jstem/index.php/JSTEM/article/view/2624/2327

Wladis, C., Hachey, A. C., Conway, K. (2014). An investigation of course-level factors as predictors of online STEM course outcomes. Computers & Education, 77, 145-150. https://doi.org/10.1016/j.compedu.2014.04.015

Woldeab, D., & Brothen, T. (2019). 21st Century assessment: Online proctoring, test anxiety, and student performance. International Journal of E-Learning & Distance Education Revue Internationale Du E-Learning Et La Formation à Distance, 34(1). Retrieved from https://www.ijede.ca/index.php/jde/article/view/1106

Wright, C. D., Eddy, S. L., Wenderoth, M. P., Abshire, E., Blankenbiller, M., & Brownell, S. E. (2016). Cognitive difficulty and format of exams predicts gender and socioeconomic gaps in exam performance of students in introductory biology courses. CBE—Life Sciences Education, 15(2), ar23. https://doi.org/10.1187/cbe.15-12-0246

Wuthisatian, R. (2020). Student exam performance in different proctored environments: Evidence from an online economics course. International Review of Economics Education, 35, 100196. https://doi.org/10.1016/j.iree.2020.100196

Xavier, M., & Menses, J. (2020). Dropout in online higher education: A scoping review from 2014 to 2018. Barcelona: eLearn Center, Universitat Oberta de Catalunya. https://doi.org/10.7238/uoc.dropout.factors.2020