Student Satisfaction and the Future of Online Learning in Higher Education: Lessons from a Natural Experiment

Although there is substantial research on the effectiveness of online learning at the individual class level, there is little reliable data on how a shift to a mostly or fully virtual campus would impact undergraduates’ satisfaction, engagement, and academic achievement. Until the COVID-19 pandemic, the limited adoption of widespread online learning at selective schools and challenges of selection bias hindered a reliable assessment of such a shift in selective institutions. After the initial period of “emergency remote learning” in 2020, many selective institutions continued widespread online learning in the second year of the pandemic. Treating the expanded deployment of online learning as a natural experiment, the present study assesses the impact of frequent online learning during the spring semester of 2021 on representative samples of undergraduate students at three selective, four-year universities. The study finds that students who participated in classes that met in person at least once a week had higher evaluations of faculty engagement and higher overall levels of academic satisfaction, compared to those who never or rarely participated in an in-person class. This relationship appears less pronounced for Black and Asian students than for White students but does not vary significantly by gender. Although online learning has great potential, these results suggest a need to better understand the conditions that will support an expansion of online learning that can maintain student satisfaction.

Long before the COVID-19 pandemic made online learning an essential part of college education, the internet and educational technology were disrupting the monopoly of the "brickand-mortar" classroom (Anglin, 2012;Nguyen, 2015).Scholars and administrators recognized that online teaching had the potential to address the rising costs of postsecondary education and make it more widely available to traditionally underrepresented populations (James, Swan, & Daston, 2016;Jung & Rha, 2000).Some argued that technological developments were undermining the idea of "location-dependent" teaching altogether, pointing the way towards a fully "deconstructed campus," unconstrained by the requirement that members of the campus community be physically present in the same location (Mazoué, 2012;cf. Shrock, 2012).Although there is a substantial body of research on the effectiveness of online learning at the level of individual classes (e.g., Bernard et al., 2004;Lack, 2013;Means, Toyama, Murphy, Bakia, & Jones, 2009;Spencer & Temple, 2021;Xu & Jaggars, 2014), fewer studies have been undertaken to understand how taking many of their classes online (versus in-person) affects undergraduates' overall college experience.This research has been hampered both by the limited adoption of widespread online learning, especially among selective, four-year institutions, and concomitant selection bias issues related to the types of students who choose to take online classes.
The onset of the COVID-19 pandemic in 2020 has had profound implications for online learning in higher education.Whether they were prepared or not, higher education institutions had little choice but to embrace online education modalities to a degree that had not been previously considered.For many, the idea of a "virtual campus," where students learn without ever being physically present in the same location, moved from hypothetical to real (Means & Neisler, 2020).The sudden and often haphazard deployment of "emergency remote teaching" during the early phase of the pandemic illustrated the potential of remote learning, as well as its challenges.The emergency use of online modalities did not, however, provide much insight into the efficacy of systematically designed online courses that require longer periods of preparation (Hodges, Moore, Lockee, Trust, & Bond, 2020).Following the initial "emergency," with the threat of the pandemic still open-ended, academic institutions were forced to make more methodical decisions in preparation for the 2020-21 academic year.Based on their experiences during the spring of 2020, institutions had to consider whether and how to use online teaching to mitigate the health threat to students and faculty in the upcoming academic year.
In the spring of 2021, a year after the onset of the pandemic, many students at institutions that had previously made limited use of online learning were still taking most or all their classes online.Because students had little or no choice with respect to the modality of classes or the proportion of online versus in-person classes in their course load for the semester, this situation can be treated as a "natural experiment," mitigating the effects of selection bias with respect to student preferences for online or in personal modalities that have challenged previous research.The present paper leverages this "natural experiment" to study the effects of a shift from a "brick-and-mortar" campus to a largely or entirely virtual campus at selective institutions that had not previously relied on online learning.
The present study compares educational experiences during the spring semester of 2021 among representative samples of undergraduate students at three selective, four-year private universities that made use of online learning to different degrees.Based on the results of previous research (Dumford & Miller, 2018;Paulsen & McCormick, 2020) we hypothesize that participating in a greater number of classes in person (as opposed to online) will be associated with higher student evaluations of the pedagogical quality at their university.Our findings indicate strong support for this hypothesis: students who participated at least once a week in classes that met in person had higher evaluations of faculty engagement at their school and higher overall levels of academic satisfaction, compared to similar students who never or rarely participated in an in-person class.This relationship appears less pronounced for Black and Asian students than for White students but does not vary significantly by gender.These results suggest that the expansion of online learning at selective institutions should be implemented with care to ensure the preservation of existing student satisfaction.

Literature Review
Debates over the effectiveness of online teaching in higher education have been contentious.Much of the empirical literature that reports differences between online and inperson learning has focused on academic outcomes, such as final grades.However, student satisfaction and perceptions of faculty engagement are also seen as critical measures of success for online teaching (Alqurashi, 2019;Moore, 2005) and can be predictors of higher academic performance (Jaggars & Xu, 2016).Some of the studies focusing on academic outcomes have concluded that there are "no significant differences" between online learning and in-person teaching (Bernard et al., 2004;Pei & Wu, 2019;Russell, 1999), while others suggest that online learning is associated with lower academic performance (Lack, 2013;Nguyen, 2015;Spencer & Temple, 2021;Xu & Jaggars, 2014;Zhao, Lei, & Yan, 2005).Research on the relationship between online learning and student satisfaction or engagement is similarly equivocal.Some studies argue that there are few differences between online and in-person modalities in terms of student satisfaction (Dinh & Nguyen, 2020;Yen, Lo, Lee, & Enriquez, 2018), but other studies find that online learners lagged behind in-person students in terms of collaborative learning and interaction with faculty (Dumford & Miller, 2018;Paulsen & McCormick, 2020) The motivating question for most extant studies is whether holding a particular class online, as opposed to in person, impacts students' perceptions of, or performance in, that specific course.However, translating and integrating knowledge across multiple classes, and building connections with study-partners and mentors outside the classroom is also an important component of a student's overall pedagogical experience (Kerr, Tweedy, Edwards, & Kimmel, 2017).Thus, a student's overall educational experience is more than the sum of individual class experiences.Taking a class online is likely to affect students' social relationships with peers, interactions with faculty, and campus support staff, as well as students' performance in and experience of other classes on related topics, regardless of whether those other classes are online or not.In the debate over the viability of a "virtual campus" model (Anderson, 2021;Mazoué, 2012;Shrock, 2012), or the widespread adoption of online learning, the increasingly important question is not whether and how taking a particular class online affects a student's experience or performance in that class; rather, it is whether or not taking many (or all) of their classes online impacts a student's overall educational experience at their institution.
Several studies have explored the impact of taking multiple online courses on student engagement using the National Survey of Student Engagement (NSSE).The questions ask about students' overall evaluation of several measures of engagement, including interactions with faculty and other students, as well as the deployment of key learning strategies.Some of these studies found that students who took all of their classes online had more interactions with faculty than in-person learners, but fewer interactions with peers (Chen, Gonyea, & Kuh, 2008).Other studies suggest that exclusive online learners interact less with faculty and peers (Paulsen & McCormick, 2020).Dumford and Miller (2018) also found that taking a larger proportion of online courses was associated with lower student engagement across a number of measures, including student-faculty interaction and collaborative learning, although only a small percentage of respondents (7.2%) took all of their courses online.
The generalizability of these studies is, however, limited.Many of these studies, and investigations of online learning in general (e.g.Xu & Jaggars, 2014), have been limited to the less selective institutions that have made the most intense use of online teaching (Bettinger, Fox, Loeb, & Taylor, 2017).Other work (James et al., 2016) also suggests that the effects of online learning at community colleges may not be generalizable to other types of institutions.The more limited adoption of widespread online learning at selective four-year institutions (in particular, highly selective private colleges and universities) has meant that there have been few opportunities to study what would happen if these institutions were to expand dramatically the proportion of classes that were taught exclusively online.
At the same time, the frequency of online or in-person classes is unlikely to have a uniform effect, even on students at the same school.Theory and existing evidence strongly suggest that the effectiveness of online learning at the level of individual classes varies by student characteristics including gender, race, and ethnicity (Nguyen, 2015).A number of studies that found a negative association between online learning and academic performance at the level of individual classes also argued that this relationship was more acute for Black and Hispanic students (Figlio, Rush, & Yin, 2013;Xu & Jaggars, 2014), who may face additional barriers to academic success at traditionally White educational institutions (Hurtado, Milem, Clayton-Pedersen, & Allen, 1998).At the same time, similar studies also suggested that online learning may have different effects on the academic performance of male and female students (Brown & Liedholm, 2002;Figlio et al., 2013;Xu & Jaggars, 2014).Spencer and Temple (2021), by contrast, found that the negative relationship between online learning and academic success they identified did not significantly vary by race or gender.While these studies examined the impact of demographic characteristics on the relationship between online learning and academic outcomes, few, if any, studies have analyzed whether race or gender impacted the relationship between online learning and student satisfaction or engagement.
In summary, there continues to be disagreement among researchers about how a shift to a mostly or fully online campus would impact students' perceptions of satisfaction or engagement.Furthermore, there is little or no research exploring whether the positive or negative effects of such a shift would disproportionately impact students of different demographic backgrounds.Explicit investigation of these questions would seem to be a prerequisite to any dramatic expansion of the use of online learning in higher education.

Selection Bias
Attempts to understand the effect of online learning on students' overall educational experience also face persistent challenges in accounting for selection bias about the types of students who choose to take courses online or in person, and the types of courses that are offered in each modality.Although many studies of online learning do not control for selection bias (Nguyen, 2015), it clearly has the potential to influence results, since students are likely to selfselect into the course modality that best fits their current situation and learning style.Insofar as this is true, the effectiveness of online learning among students who willingly sought it out may not be a reliable predictor of its effectiveness among students who would have preferred to learn in person (see Morgan & Winship, 2007).
Other work has directly investigated this phenomenon and has tended to find that failing to account for selection bias leads to overestimation of the positive effects of online learning.Anstine and Skidmore (2005) and Coates, Humphreys, Kane, and Vachris (2004) show that failing to account for these selection effects can lead to a substantial overestimation of the effectiveness of online learning relative to in-person learning in terms of academic achievement.Paulsen and McCormick (2020) likewise show that when propensity score matching is used to account for selection effects, many of the positive effects of online learning identified by earlier work with respect to student engagement (e.g.Chen et al., 2008;Rabe-Hemp & Woollen, 2009) diminished in magnitude or disappeared entirely, while negative effects on collaborative learning remained.
Although randomized assignment may be an effective mechanism to control for selection bias at the level of individual classes (see, e.g.Bowen, Chingos, Lack, & Nygren, 2013;Figlio et al., 2013), it is infeasible for studies analyzing the overall effect of taking multiple classes online.Furthermore, even more sophisticated analytic methods such as regression analysis (Dumford & Miller, 2018) or even propensity score-matching (Paulsen & McCormick, 2020) cannot adjust for bias on variables that are unobserved in the data, such as the student's own preference for online or in-person learning styles (Fan & Nowell, 2011;Reiffel, 2020).Thus, the issue of selection bias remains a key challenge for investigating the overall effectiveness of online learning, including dropout, academic achievement, satisfaction, and engagement.
Regardless of whether they were learning online or in person, student experiences during the pandemic were clearly unique.Nevertheless, the deliberate expansion of online modalities by a wide variety of institutions during the second year of the COVID-19 pandemic provided an opportunity to address some of the major limitations in existing research, including reducing the threat of selection bias.The present study thus examines how taking all or most of their classes online during the spring of 2021 impacted students' overall satisfaction with their pedagogical experiences and faculty interactions at their institution.It also explores whether and how this impact varied across student demographic characteristics.

Method Research Design
In response to the sudden onset of the COVID-19 pandemic during the spring semester of 2020, administrators and faculty were called upon to rapidly deploy "emergency remote teaching" to safeguard the health of community members (Hodges et al., 2020).Many faculty members with little or no prior experience in online teaching were forced to move their classes online, with scant preparation.Notably, this shift was not limited to community colleges or forprofit institutions that had traditionally embraced online learning, but also occurred among highly selective, private, four-year colleges and universities (Vigdor & Zaveri, 2020).Over the summer of 2020, with the pandemic still raging, institutions had to decide proactively how to balance in-person and online modalities for the coming academic year.Unsurprisingly, differences in administrator views about COVID-19 safety, as well as political and budgetary considerations, had a major impact on whether the classes available to students at particular institutions were held online or in person during the 2021-21 academic year (Felson & Adamczyk, 2021).The spring semester of 2021 thus represents a unique opportunity.Unlike the situation in the spring of 2020, where instructors were not given sufficient time to prepare for online instruction (Tsang, So, Chong, Lam, & Chu, 2021), by spring 2021, most faculty and students had nearly a year of practice in adapting to online teaching and learning.At the same time, the continued threat of the pandemic meant that many students who would previously not have chosen to take online courses were still forced to do so, and many selective institutions that previously made limited use of online learning embraced it, even as they allowed for some classes to take place in person.
This situation provides an imperfect but valuable "natural experiment" that enables us to study the impact of a dramatic expansion in online learning at selective institutions, while reducing the threat of selection bias.In particular, student preferences for online or in-person classes, which have contributed to selection bias in earlier work (Anstine & Skidmore, 2005;Coates et al., 2004), were far less likely to be correlated with students' propensity to take online classes during the 2021 spring semester.Even at schools where students had some discretion about attending classes in person, these decisions were likely influenced more by concerns related to COVID-19, which can be more easily controlled for, than by prior preference for online or in-person learning styles, which are more likely to be strongly correlated with potential outcome variables.By studying multiple schools with different COVID-19 mitigation policies, and by controlling for COVID-19 related concerns, it is possible to use the pandemic to study how a shift to a mostly "virtual" campus affected student perceptions of pedagogical quality at selective institutions that had never previously made widespread use of online learning.
Our focus is on students at three private, selective universities in the northeast United States.Our outcome of interest is students' perceptions of their pedagogical experience.To develop a measure of this construct, we conducted exploratory factor analysis on a set of six questions adapted from the UCLA HERI first-year and senior surveys (Higher Education Research Institute, 2021a, 2021b).Following James et al. (2016), our key independent variable distinguishes between three types of students: those who never had in-person classes in spring 2021, those who had in-person classes occasionally in spring 2021, and those who had in-person classes once a week or more in spring 2021.The frequency with which students in spring 2021 had in-person classes was unlikely to have been strongly influenced by traditional confounders related to personal preference and was more likely to be driven by university policy, faculty discretion, and concerns related to COVID-19.Including university fixed effects and controls for concerns about COVID-19, financial stress, and demographic characteristics helps to minimize selection effects associated with student preferences.
Existing research generally analyzes the impact of taking classes online as opposed to in person.However, during the spring of 2021, online learning remained the "default" mode of instruction at many schools, with in-person learning allowed only in particular contexts.As such, we consider in-person learning the "treatment" condition and analyze the impact of taking classes in person as opposed to online.Earlier work that attempts to account for selection bias has generally found that more frequent online learning is associated with lower levels of student engagement across a number of measures (Dumford & Miller, 2018;Paulsen & McCormick, 2020).Considering these results, we hypothesize that taking more classes in person (as opposed to online) will be associated with higher student evaluations of the pedagogical quality at their university.These hypotheses are tested using OLS models on the scales developed from the HERI instructional satisfaction questions.We also specify additional models with interaction terms to explore whether, as suggested by earlier research, the effect of in-person learning on student evaluations varies significantly by gender or race/ethnicity (Brown & Liedholm, 2002;Figlio et al., 2013;Xu & Jaggars, 2014).

Data Collection
Data for this study come from online surveys of undergraduate students at three private, selective research universities in the greater Boston area: Brandeis University (Brandeis), Boston College (BC), and Northeastern University (Northeastern).1At each school the unique links to the survey were sent to the official ".edu" email addresses of a simple random sample drawn from the population of full-time undergraduate students ages 18 or older enrolled in the 2021 spring semester.The Brandeis and BC surveys were conducted while classes were in session during the 2021 spring semester, while the Northeastern survey was conducted shortly after the end of the semester.Approval for this research was granted by the researchers' home institution's Human Research Protection Program and accepted by the other two institutions. 2tudents at Brandeis and BC were informed that upon completion of the survey they could choose to receive a $10 Amazon.comgift card or have $10 donated on their behalf to the Greater Boston Food Bank.Due to university policy, respondents at Northeastern were not offered a gift card or a donation option.The response rate (including complete and partial respondents) was 40.2% at Brandeis (achieved N = 817), 20.9% at BC (achieved N = 836), and 8.5% at Northeastern (achieved N = 715).For each school, weights were calculated to adjust for differences between the characteristics of respondents and known characteristics of the undergraduate student body with respect to gender (at all three schools), international student status (Brandeis and Northeastern), and class year (Brandeis only).
All three schools deployed both in-person and online learning modalities during the spring 2021 semester but to different degrees due to differences in university "re-opening" plans announced at or before the beginning of the 2020 fall semester.Brandeis policy required that the "majority" of undergraduate classes be taught online, "with the opportunity for in-person elements of those courses for students who live on or near campus," but also noted that "[s]elect courses will occur primarily in person" (Office of the President, 2020).Northeastern emphasized a more flexible approach whereby "some students will be present in the classroom and others will be participating remotely," with the aid of "a dynamic scheduling tool….[to]allow students to indicate and inform faculty whether they will be participating face-to-face in-class in a given week, or whether they'll attend remotely" (Madigan & Henderson, 2020).At BC, classes were taught "in a mix of in-person, online, and hybrid modes, using Zoom or other technologies," with classrooms "reorganized to ensure social distancing, and…limited to 50 percent capacity" (University Communications, 2020).

Measures
To develop a measure of students' perceptions of the quality of instruction at their university we asked six questions, drawn from the 2021 UCLA HERI senior and first-year surveys (Table 1).We used exploratory factor analysis to create measures of the underlying latent variable or variables driving student responses to these six items.The "satisfaction" item was measured on a four-point scale ranging from "not at all satisfied" to "very satisfied." 3The remaining items were measured on a five-point scale, ranging from "never" to "all the time."Because of the difference in measurement scales, all variables were z-scored prior to being entered into exploratory factor analysis.To measure students' experiences with in-person/online teaching we asked students: "Thinking about all of your classes THIS SEMESTER, how often did you participate in a class in person?" with options of "never," "1-2 times during the semester," "once a month," "2-3 times a month," and "at least once a week." 4,5The effects of the different policies related to online and in-person instruction discussed above can be clearly seen in the different frequencies of in-person learning reported by students at different schools (Тable 2).At none of the schools was weekly in-person classes or completely online learning universal, limiting the danger of multicollinearity between modality and school.Following James et al. (2016), responses to the question regarding frequency of online instruction were collapsed into three categories: never, more than once a semester but less than once a week, and at least once a week.This is a fairly coarse measure of the amount of in-person/online learning students experienced during the semester and does not, for example, distinguish between students who had all of their courses taught fully in-person at every class and those who only had a single course that met in person each week.Nor does it distinguish between different class types, subject matters, or different types of online instruction (e.g., asynchronous versus synchronous).As such, the measure is likely to produce a conservative estimate of differences between online and in-person learning during this semester.All models also control for student characteristics that could confound the relationship of interest.This includes the student's maximum class size, class year, housing situation, four-point ordinal questions measuring level of concern about becoming hospitalized with COVID-19 or spreading COVID-19 to others, 6 race/ethnicity (White, Hispanic, Black, Asian, other), 7 gender, and an index of financial stress. 8All models also include dummy variables for schools, which control for any unobserved differences between schools that could bias results.

Factor Analysis
To develop a measure of student perceptions of the pedagogical quality of their academic experience exploratory factor analyses were conducted on the z-scored versions of the six items adapted from the UCLA HERI survey (Table 1).A scree plot of unrotated eigenvalues recommended a two-factor solution.After orthogonal varimax rotation, the first factor explained 26% of the common variance, while the second factor explained 17%. 9 Table 3 presents loadings and variances for the rotated solution.Based on the rotated factor loadings, Factor 1 is interpreted as perceived faculty engagement, since it is primarily measured by questions concerning the faculty's responsiveness and encouragement.Factor 2 is interpreted as a measure of overall academic satisfaction, since it is primarily measured by items related to the overall quality of courses at the students' institutions, including satisfaction with the quality of instruction, whether students were often bored, and whether their classes inspired them to think in new ways.These results were used to calculate regression-based factor scores to serve as measures of both latent constructs.Table 4 presents descriptive statistics of the two resulting measures. 7The "other ethnicity" category includes American Indian, Middle Eastern, Pacific Islander and "other" ethnicities.Respondents were also offered the opportunity to select multiple racial/ethnic identities.Respondents who selected multiple identities were sorted into mutually exclusive categories using the following hierarchy: Black, Hispanic, Asian, other, White.This means that only those respondents who identified exclusively as "White" were coded as white, while all respondents who identified as "Black" were coded as Black, regardless of whether they also identified as another racial/ethnic group. 8The financial stress index represents the average of two question that relate to the financial situation of respondents during the semester: "To what extent were your financial obligations a source of stress?" and "To what extent was the financial situation of your immediate family a source of stress?"Response options for both questions were along a four-point ordinal scale ranging from "not at all" to "very much." 9 Proportion of common variance calculations use the trace of the reduced correlation matrix as the denominator.

OLS Models
For each hypothesis, separate OLS models were run for the two latent constructs identified by the factor analysis-perceived faculty engagement and academic satisfaction.Table 5 presents models testing the overall relationship between the frequency of in-person learning and each construct.The models show that students who participated in an in-person class at least once a week had significantly more positive evaluations of faculty engagement and significantly higher overall academic satisfaction, compared to those who only occasionally participated in an inperson class (p < .001).For both outcomes, the estimated positive impact of participating in an in-person class at least once a week was approximately 0.4 standard deviations on the underlying scale.Students who never participated in an in-person class did not have significantly different scores on either outcome, compared to those who only occasionally participated in an in-person class.
In both models, concerns about becoming hospitalized with COVID-19 or spreading it to others were non-significant, as were school fixed effects, housing situation, and gender identity.Having a maximum class size of 30 students (as opposed to 50+) was associated with higher scores on both outcome measures, while financial stress was negatively associated with both measures.Black students had significantly higher evaluations of faculty engagement compared to White students.Asian students had significantly higher levels of satisfaction compared to White students, although the magnitude of this effect was small.Sophomore and junior students reported lower levels of satisfaction compared to first-year students, and juniors reported significantly lower perceptions of faculty engagement compared to first-year students.Additional models of each outcome variable that include interactions between frequency of inperson classes and either race/ethnicity (Table 6) or gender identity (Table 7) are also presented. 10The results of the interactions with race/ethnicity suggest that the effects of participating in frequent in-person classes were significantly moderated by race.In the model of faculty engagement, the interaction terms for participating in an in-person class at least once a week (as opposed to occasionally) and identifying as Black, Asian, or other ethnicity (as opposed to White) were negative and significant, although the interaction term for Hispanic students was not significant.This indicates that the positive relationship between frequency of in-person classes and perceptions of faculty engagement was significantly smaller for non-White, non-Hispanic students, than for White students.Notably, the absolute value of the interaction term coefficients for these students (.60 for Black students, .47 for Asian students, and .89for other ethnicities) were of comparable magnitude to that of the main effect for White students (.51), suggesting that for Black, Asian, and other ethnicity students, the association between frequent in-person classes and perceptions of faculty engagement was negligible.A similar dynamic for Asian students is evident in the model of academic satisfaction, suggesting that these students realized a significantly smaller benefit from frequent in-person classes compared to White students.The coefficients for the interaction terms between frequency of in-person classes and gender, by contrast, were not statistically significant in either model (Table 7), indicating that the positive relationship between taking in-person classes once a week (as opposed to occasionally) and both outcomes was not significantly different for male students, or those who identified as neither male nor female, compared to female students.

Discussion
In line with our hypotheses, students across the three universities who had in-person classes at least once a week reported higher levels of faculty engagement and academic satisfaction, compared to those who only had in-person classes occasionally.After controlling for student backgrounds and school-level differences, there was an approximately 0.4 standard deviation increase on both measures for those who participated at least once a week in an in-person class.The size and significance of these effects are notable given the limitations of our measure of online/in-person learning experiences, which merely indicates whether students participated in an in-person class at least once per week as opposed to only a few times per semester.
We also find no significant difference between those who never participated in an inperson class and those who did so only occasionally, suggesting that holding in-person classes only a few times a semester does not confer the same benefits to students as holding them weekly.As discussed below, both online and in-person learning environments during the second year of the COVID-19 pandemic were clearly distinctive.Yet, the fact that the disruptions of the pandemic took many of the decisions about whether to take a class online or in person out of the hands of students suggests that these results are less likely to be biased by the selection effects impacting earlier research.Perhaps surprisingly, the dummy variables for schools were not significant in any models, indicating that, after accounting for other factors-including differences in the proportion of students who had frequent in-person classes and differences in the demographic make-up of the student population at each school-students at these three schools did not significantly differ in their average levels of academic satisfaction or perceptions of faculty engagement.
This study is one of very few efforts to explore the effect on students of taking many online classes at highly selective, private, four-year institutions.Nevertheless, our findings are in concordance with earlier results indicating that online learning is associated with lower levels of student engagement at public and less selective four-year institutions (Dumford & Miller, 2018;Paulsen & McCormick, 2020).We also found that differences in student satisfaction and engagement associated with online learning were significantly moderated by race/ethnicity, as suggested by earlier work on the relationship between online learning and academic performance outcomes (but contrary to the findings of Spencer and Temple (2021)).However, while Xu and Jaggars (2014) and Figlio et al. (2013) found that differences in academic outcomes between online and in-person learning were larger for non-White students compared to White students, we find that differences in student satisfaction and engagement were significantly smaller for Black and Asian students (and those of other ethnicities) compared to White students.This suggests that the positive relationship between in-person learning, and student perceptions of academic satisfaction and faculty engagement was concentrated among White and Hispanic students.Regarding gender identity, we found no significant interaction for either outcome, suggesting that the positive effect of frequent in-person learning was of similar magnitude for male and gender non-binary students as for female students.
In addition to differences in our outcome variable of interest, there are other reasons why care should be taken in comparing our results regarding the moderating impact of race/ethnicity on student perceptions with earlier studies of the differential impact of online learning across demographic characteristics.Most notably, the Black and Asian students who attend selective four-year institutions may come from different socioeconomic backgrounds and face different challenges, compared to Black and Asian students who attend the community colleges studied by Xu and Jaggars or the large, public, land-grant university studied by Spencer and Temple.It should also be noted that Black and Asian students at elite schools, especially those who are first-generation college students, may have dramatically different expectations about faculty engagement compared to White students.At the same time, creating a healthy and welcoming campus climate for students from marginalized racial or ethnic backgrounds remains a challenge for traditionally White higher education institutions (Hurtado et al., 1998;Park, 2009).Even if online learning can provide additional flexibility and opportunities to these students, more research is needed to ensure that a shift to a largely or fully virtual campus does not raise further barriers to the formation of robust social connections within and across racial and ethnic identities on campus.
Although the spring semester of 2021 provided a "natural experiment" that helped reduce the threat of selection bias, the pandemic still potentially limits the external validity of the study.While faculty in the schools included in our study had over a year to prepare for teaching their courses online, it seems inarguable that more robust preparation and training could mitigate some of the disparities identified by our analyses.More broadly, the pandemic limited the extent to which online teaching during this semester could be effectively integrated into a broader educational ecosystem that could support students effectively (Hodges et al., 2020).
At the same time, the disruptions of the pandemic were not limited to the expansion of online teaching modalities; they also affected teaching and learning at classes that were held in person.Restrictions related to testing, masks, and social distancing inarguably hampered the enjoyment and quality of in-person learning relative to a "typical" semester, in which students and faculty could congregate freely in classrooms, offices, or other physical locations on campus.The fact that, even with such restrictions, attending a single in-person class as rarely as once a week was associated with a substantial increase in student evaluations of the overall academic quality of their entire institution suggests that, even in a "typical" year, in-person teaching has substantial value at selective institutions.

Implications for Online Education
In innumerable ways, the pandemic demonstrated the important role online learning has come to play in higher education, including at selective schools that had previously eschewed widespread adoption.Regardless of its efficacy in comparison to in-person learning, it seems likely that, in some form, online learning will be an increasingly important part of the future of higher education, especially because of its potential to facilitate greater access and flexibility for marginalized student populations.Existing research strongly indicates that the effectiveness of online learning varies for different types of classes, different types of students, and different types of schools, and whether online classes are taken exclusively or in combination with inperson classes.Thus, the question facing administrators, as well as faculty, is not whether to make use of online learning, but how to use it most effectively and promote students' educational development.
This paper treats the continued disruptions of the pandemic as an opportunity to study the impact of a particular way that online learning might be deployed in higher education: a dramatic shift away from "brick-and-mortar" classrooms in favor of online teaching modalities at selective, four-year institutions.Our results suggest that more selective institutions should carefully examine the conditions that could support such a shift while continuing to maintain student satisfaction.Future work should continue to explore how the effects of online learning depend on context and implementation, while appropriately accounting for selection bias and including data from students at different types of institutions.This work can provide a more nuanced understanding of some of the situations under which online learning can be deployed with the most benefits and fewest costs.

Table 1
Pedagogical Quality QuestionsPlease rate your satisfaction with your school's overall quality of instruction.This semester how often have you felt that your courses inspired you to think in new ways?This semester how often have you felt that faculty provided you with feedback that helped you assess your progress in class?This semester how often have you felt that your contributions were valued in class?This semester how often have you felt that faculty encouraged you to ask questions and participate in discussions?This semester how often have you felt that you were bored in class?(reverse coded)

Table 2
Frequency of In-person Instruction by School

Table 5
OLS Models of Faculty Engagement and Academic Satisfaction

Table 6
OLS Models of Faculty Engagement and Academic Satisfaction with Interactions Between Inperson Classes and Race/EthnicityCoefficients for concerns about COVID, maximum class size, class year, housing, financial stress, university fixed effects, and gender omitted.** p < 0.001, * p < 0.05.

Table 7
OLS Models of Faculty Engagement and Academic Satisfaction with Interactions Between Inperson Classes and Gender