Student Performance in Online Classes at a Hispanic-Serving Institution: A Study of the Impact of Student Characteristics in Online Learning
DOI:
https://doi.org/10.24059/olj.v25i3.2853Keywords:
Distance education, Online education, Higher education, Success rates, Educational indicators, Propensity score analysisAbstract
As online enrollment increases in the United States, it is important to understand the impact of course modality on student outcomes. In particular, there has been limited research on the effect of course enrollment at Hispanic-serving institutions (HSI). The current study evaluated the effect of online course enrollment on student grades and on student withdrawal rates by comparing outcomes in online and face-to-face classes. The main innovation of this study is to use propensity score analysis to control for 15 different student characteristics as a way to control for the selection bias introduced when students self-select into different course modalities. The study used data from a large, public, HSI in the mountain west during the 2017-2018 and 2018-2019 academic years. Baseline results on a two-sample t-test indicated that online students have significantly higher course grades, and non-significantly different withdrawal rates before controlling for student characteristics. The study used a propensity score analysis (PSA) to control for 15 confounding covariates after testing three different PSA models: near-neighbor matching, Mahalanobis’ metric, and optimal matching. After evaluating each model for validity and sensitivity, a near-neighbor 1:2 matching PSA shows a non-significant difference in student grades, and higher withdrawal rates in online classes than face-to-face classes. Given these results, institutions should ensure that they are providing adequate academic support for online students to improve retention and success rates for online students.
References
Allen, I., & Seaman, J. (2014). Grade change: Tracking online education in the United States. Pearson. http://www.onlinelearningsurvey.com/reports/gradechange.pdf
Amro, H. J., Mundy, M., & Kupczynski, L. (2015). The effects of age and gender on student achievement in face-to-face and online college algebra classes. Research in Higher Education Journal, 27, 1–22.
Arias, J. J., Swinton, J., & Anderson, K. (2018). Online vs. face-to-face: A comparison of student outcomes with random assignment. E-Journal of Business Education and Scholarship of Teaching, 12(2), 1–23.
Ashby, J., Sadera, W. A., & McNary, S. W. (2011). Comparing student success between developmental math courses offered online, blended, and face-to-face. Journal of Interactive Online Learning, 10(3), 128–140.
Atchley, W., Wingenbach, G., & Akers, C. (2013). Comparison of course completion and student performance through online and traditional courses. International Review of Research in Open and Distance Learning, 14(4), 104–116.
Bouchey, B., Gratz, E., & Kurland, S. (2021). Remote student support during COVID-19: Perspectives of chief online officers in higher education. Online Learning, 25(1), 28–40. https://doi.org/10.24059/olj.v25i1.2481
Bunn, E., Fischer, M., & Marsh, T. (2014). Does the classroom delivery method make a difference? American Journal of Business Education, 7(2), 143–150.
Camara, I. (2016). A comparison of academic performance in face-to-face, hybrid and online anatomy and physiology I courses at a college in south Texas. ProQuest Dissertations Publishing.
Cavanaugh, J. K., & Jacquemin, S. J. (2015). A large sample comparison of grade based student learning outcomes in online vs. face-to-face courses. Online Learning, 19(2), 1–8.
Coates, D., Humphreys, B., R., Kane, J., & Vachris, M., A. (2004). "No significant distance" between face-to-face and online instruction: Evidence from principles of economics. Economics of Education Review, 23(5), 533–546. doi://dx.doi.org.du.idm.oclc.org/10.1016/j.econedurev.2004.02.002
Cooper, L. W. (2001). A comparison of online and traditional computer applications classes. T.H.E.Journal, 28(8), 52.
Cottrell, R. (2020). Student outcomes in online and face-to-face classes at a Hispanic-serving institution (HSI). ProQuest Dissertations Publishing.
Crisp, G., & Nora, A. (2010). Hispanic student success: Factors influencing the persistence and transfer decisions of Latino community college students enrolled in developmental education. Research in Higher Education, 51(2), 175–194.
Dellana, S. A., Collins, W. H., & West, D. (2000). On-line education in a management science course--effectiveness and performance factors. Journal of Education for Business, 76(1), 43–47.
DiSanto, J. M., & Guevara, C. (2019). The online learning initiative: Training the early adopters. In K. Wolfe, K. Lyons, C. Guevara (Eds.), Developing educational technology at an urban community college (pp. 79–93). SpringerLink, & LINK.
Dotterweich, D. P., & Rochelle, C. F. (2012). Online, instructional television and traditional delivery: Student characteristics and success factors in business statistics. American Journal of Business Education, 5(2), 129–138.
Espinoza, Penelope P., & Espinoza, Crystal C. (2012). Supporting the 7th-year undergraduate: Responsive leadership at a Hispanic-serving institution. Journal of Cases in Educational Leadership, 15(1), 32–50.
Euzent, P., Martin, T., Moskal, P., & Moskal, P. (2011). Assessing student performance and perceptions in lecture capture vs. face-to-face course delivery. Journal of Information Technology Education, 10, 295–307.
Excelencia in Education. (2019). Latinos in higher education: Compilation of fast facts. https://www.edexcelencia.org/research/publications/latinos-higher-education-compilation-fast-facts
Figueira, R. (2015). The applicability of Tinto's model of student retention in online learning: A faculty perspective. ProQuest Dissertations Publishing.
Flores, S. M., & Park, T. J. (2015). The effect of enrolling in a minority-serving institution for Black and Hispanic students in Texas. Research in Higher Education, 56(3), 247–276.
Garcia, G. A., & Ramirez, J. J. (2018). Institutional agents at a Hispanic serving institution: Using social capital to empower students. Urban Education, 53(3), 355–381. https://doi.org/10.1177/0042085915623341
Green, E. (2020, March 10). Rules eased on colleges seeking to close their campuses amid outbreak. The New York Times. https://www.nytimes.com/2020/03/10/us/politics/coronavirus-colleges.html?searchResultPosition=5
Gregory, C. B., & Lampley, J. H. (2016). Community college student success in online versus equivalent face-to-face courses. Journal of Learning in Higher Education, 12(2), 63–72.
Gulacar, O., Damkaci, F., & Bowman, C. R. (2013). A comparative study of an online and a face-to-face chemistry course. Journal of Interactive Online Learning, 12(1), 27–40.
Guo, S., & Fraser, M. (2015). Propensity score analysis. Sage.
Gutierrez, D., & Russo, S. (2005). Comparing student performance, attitudes and preferences in an introduction to business course: Online, hybrid and traditional delivery methods—who makes the "A" grade? College Teaching Methods & Styles Journal, 1(3), 83–90.
Hurlbut, A. R. (2018). Online vs. traditional learning in teacher education: A comparison of student progress. American Journal of Distance Education, 32(4), 248–266. doi://dx.doi.org.du.idm.oclc.org/10.1080/08923647.2018.1509265
Hurtado, S., & Ponjuan, L. (2005). Latino educational outcomes and the campus climate. Journal of Hispanic Higher Education, 4(3), 235–251.
Johnson, D., & Palmer, C. C. (2015). Comparing student assessments and perceptions of online and face-to-face versions of an introductory linguistics course. Online Learning, 19(2), 1–18.
Jorczak, R. L., & Dupuis, D. N. (2014). Differences in classroom versus online exam performance due to asynchronous discussion. Journal of Asynchronous Learning Networks, 18(2), 1–9.
Kaupp, R. (2012). Online penalty: The impact of online instruction on the Latino-white achievement gap. Journal of Applied Research in the Community College, 19(2), 8–16.
Kelly, A. P., Schneider, M., & Carey, K. (2010). Rising to the challenge: Hispanic college graduation rates as a national priority. American Enterprise Institute for Public Policy Research. https://files.eric.ed.gov/fulltext/ED508846.pdf
Kim, J. (2020, April 1). Teaching and learning after COVID-19:Three post-pandemic predictions. Inside Higher Ed. https://www.insidehighered.com/digital-learning/blogs/learning-innovation/teaching-and-learning-after-covid-19och, J. (2005a). Does distance learning work? A large sample, control group study of student success in distance learning. E-Journal of Instructional Science and Technology, 8(1), 1–21.
Koch, J. (2005b). The influence of ethnic background, gender and age on student performance in distance learning programs. Journal of Educational Technology, 2(3), 38–49.
LaMeres, B. J., & Plumb, C. (2014). Comparing online to face-to-face delivery of undergraduate digital circuits content. IEEE Transactions on Education, 57(2), 99–106. doi://dx.doi.org.du.idm.oclc.org/10.1109/TE.2013.2277031
Lapsley, R., Kulik, B., Moody, R., & Arbaugh, J. B. (2008). Is identical really identical? An investigation of equivalency theory and online learning. Journal of Educators Online, 5(1), 1–19.
Larson, D. K., & Sung, C. (2009). Comparing student performance: Online versus blended versus face-to-face. Journal of Asynchronous Learning Networks, 13(1), 31–42.
Leasure, A. R., Davis, L., & Thievon, S. L. (2000). Comparison of student outcomes and preferences in a traditional vs. world wide web-based baccalaureate nursing research course. Journal of Nursing Education, 39(4), 149–54.
Leite, W. (2017). Practical propensity score methods using R. Sage.
Martin, N. K., & Meyer, K. (2010). Efforts to improve undergraduate student retention rates at a Hispanic serving institution: Building collaborative relationships for the common good. College and University, 85(3), 40–49.
McCauley, A. (2020, April 10). How COVID-19 could shift the college business model: ‘It’s hard to go back.’ Forbes. https://www.forbes.com/sites/alisonmccauley/2020/04/09/how-covid-19-could-shift-the-college-business-model/#406361a9564f
McDonough, C., Roberts, R. P., & Hummel, J. (2014). Online learning: Outcomes and satisfaction among underprepared students in an upper-level psychology course. Online Journal of Distance Learning Administration, 17(3).
Meling, V. (2012). The role of supplemental instruction in academic success and retention at a Hispanic-serving institution, ProQuest Dissertations and Theses.
Miller, C. (2021, March 14). Online education statistics. Educationdata.org. https://educationdata.org/online-education-statistics
Museus, S. D., & Ravello, J. N. (2010). Characteristics of academic advising that contribute to racial and ethnic minority student success at predominantly White institutions. NACADA Journal, 30(1), 47–58.National Center for Education Statistics, U.S. Department of Education. (2019). Digest of Education Statistics 2019, Table 311.15. https://nces.ed.gov/programs/digest/d19/tables/dt19_311.15.asp
National Center for Education Statistics, U. S. Department of Education. (2012). Higher education: Gaps in access and persistence study. https://nces.ed.gov/pubs2012/2012046.pdf
Reuter, R. (2009). Online versus in the classroom: Student success in a hands-on lab class. American Journal of Distance Education, 23(3), 151–162.
Rivera, J. C., McAlister, M. K., & Rice, M. L. (2002). A comparison of student outcomes & satisfaction between traditional & web based course offerings. Online Journal of Distance Learning Administration, 5(3).
Rosenbaum, P. R. (2002). Observational studies (2nd ed). Springer.
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55.
Rovai, A. P. (2003). In search of higher persistence rates in distance education online programs. Internet and Higher Education, 6(1), 1–16. doi:10.1016/S1096-7516(02)00158-6
Russell, T. (1999). The no significant difference phenomenon: As reported in 355 research reports, summaries and papers. North Carolina State University Press.
Seaman, J., Allen, I., & Seaman, J. (2018). Grade increase: Tracking distance education in the United States. Pearson. http://onlinelearningsurvey.com/reports/gradeincrease.pdf
Smith, N. (2017). Examining the effects of online enrollment on course outcomes using weighting procedures after multiple imputation on a state-wide university system. ProQuest Dissertations Publishing.
Sorrells, D., & Wittmer, C. (2020). Reflections on transitioning to remote learning assistance during COVID-19 and possible implications for the future. In M. Frizell (Ed.), The learning assistance review: Special issue. Rising to the challenge: Navigating COVID-19 as higher education learning center leaders. (pp. 41–48). National College Learning Center Association.
Summers, J. J., Waigandt, A., & Whittaker, T. A. (2005). A comparison of student achievement and satisfaction in an online versus a traditional face-to-face statistics class. Innovative Higher Education, 29(3), 233–250. doi://dx.doi.org.du.idm.oclc.org/10.1007/s10755-005-1938-x
Tinto, V. (1993). Leaving college: rethinking the causes and cures of student attrition.
University of Chicago Press.
Tinto, V. (1975). Dropout from higher education: A theoretical synthesis of recent research. Review of Educational Research, 45(1), 89–125. doi:10.3102/00346543045001089
Tseng, H., & Walsh, E. J., Jr. (2016). Blended versus traditional course delivery: Comparing students' motivation, learning outcomes, and preferences. Quarterly Review of Distance Education, 17(1), 43–52.
Verhoeven, P., & Wakeling, V. (2011). Student performance in a quantitative methods course under online and face-to-face delivery. American Journal of Business Education, 4(11), 61–66.
Waschull, S. B. (2001). The online delivery of psychology courses: Attrition, performance, and evaluation. Teaching of Psychology, 28(2), 143–47.
Werhner, M. J. (2010). A comparison of the performance of online versus traditional on-campus earth science students on identical exams. Journal of Geoscience Education, 58(5), 310–312.
Wladis, C., Conway, K. M., & Hachey, A. C. (2015). The online STEM classroom—Who succeeds? An exploration of the impact of ethnicity, gender, and non-traditional student characteristics in the community college context. Community College Review, 43(2), 142–164. doi:10.1177/0091552115571729
Wolfe, K., Lyons, K., & Guevara, C., (Eds.). (2019). Developing educational technology at an urban community college. SpringerLink, & LINK.
Xu, D., & Jaggars, S. S. (2011a). The effectiveness of distance education across Virginia's community colleges: Evidence from introductory college-level math and English courses. Educational Evaluation and Policy Analysis, 33(3), 360–377. doi:10.3102/0162373711413814
Xu, D., & Jaggars, S. S. (2011b). Online and hybrid course enrollment and performance in Washington state community and technical colleges. Community College Research Center Working Paper No. 31. Community College Research Center, Columbia University.
Xu, D., & Jaggars, S. S. (2013). The impact of online learning on students’ course outcomes: Evidence from a large community and technical college system. Economics of Education Review, 37, 46–57. doi:10.1016/j.econedurev.2013.08.001
Xu, D., & Jaggars, S. S. (2014). Performance gaps between online and face-to-face courses: Differences across types of students and academic subject areas. The Journal of Higher Education, 85(5), 633–659. doi:10.1080/00221546.2014.11777343
Downloads
Published
Issue
Section
License
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
