Examining Students’ Online Course Perceptions and Comparing Student Performance Outcomes in Online and Face-to-Face Classrooms

Through the use of existing grade and student survey data, this study investigated online courses offered at a public four-year university. Specifically, the study explored differences in student success rates between online and face-to-face courses for traditional undergraduate students as well as the climate of student perceptions towards online courses. Our general results suggest that students performed better in, and had higher levels of preference toward, traditional face-to-face formats. However, overall perceptions of online courses were positive, with students viewing instructional technologies as reliable and easy to use, as well as reporting that online technologies facilitated prompt feedback, enhanced their problem-solving skills, and met their learning needs. Alongside this, students exhibited positive views towards their instructors’ skill level and use of technology to support academic success. Logistic regression analyses of differences in student success across instructional formats revealed interaction effects with variables of age (nontraditional/traditional), aid status and whether or not courses were taken to fulfill general education or major requirements, suggesting a more complex effect of instructional format across student subpopulations. The variability in the results observed in the current study warrant further exploration before definitive conclusions on the impact of instructional format on student outcomes and perceptions can be made. format variables of student financial aid status general (GEP) status. to a lack of prior research that has included interactions between aid and format, our findings provide an initial insight into the impact of instructional format for differing financial aid groups. Results suggest that receiving financial aid impacts observed differences in student success between online and face-to-face formats. The model indicated that for those not receiving aid, students were less likely to receive an ABC in online courses compared to face-to-face ( OR = .73). However, this was not replicated for those with aid. When comparing students who received financial aid to those who did not, findings indicated that those receiving aid were less likely to receive ABC in face-to-face settings. This finding contradicts the general on financial aid that has suggested receiving aid leads to improved academic performance Stater, However, these results should be interpreted with some caution due to the effect sizes being relatively small ( OR = .77). Alongside this, no differences were observed based on financial aid status in online courses. In the context of the current model/interaction, it could be argued that the smaller odds ratios observed between aid groups in online settings compared to face-to-face settings is a result of those without aid showing decreased performance online, rather than a positive effect of aid in online settings.


Examining Students' Online Course Perceptions and Comparing Student Performance Outcomes in Online and Face-to-Face Classrooms
The rise to prominence of distance education (DE) has been primarily driven by the advent of online instructional technologies and the desire to provide access to education to those who would otherwise not be able to participate (Bates, 2005;Beldarrain, 2006;Kentnor, 2015). In recent years, the need for DE programs has increased in order to address the growing population of college-aged students and increased enrollment numbers (Howell, Williams, & Lindsay, 2003). Since 2012, DE enrollments have grown steadily (Allen & Seaman, 2017;Seaman, Allen, & Seaman, 2018). As of fall 2016, nearly one third of students (31.6%) were taking at least one DE course, with 86.1% of these students studying at the undergraduate level (Seaman, Allen, & Seaman, 2018). Of these students, a significant portion received their education solely via online courses (47.2%; Seaman, Allen, & Seaman, 2018).
With the rapid increase in DE course offerings, policymakers and stakeholders in higher education have been concerned with the quality of education provided to students. Commonly, comparisons have been made between online DE courses and traditional face-to-face classrooms (Means, Toyama, Murphy, Bakia & Jones, 2009), with the most prominent measures of quality being student success (in the form of completion rate and student grade performance). Conclusions drawn from this literature have been mixed, with studies varying significantly in their findings. Several studies have shown reduced completion rates in online courses compared to traditional face-to-face courses (e.g., Atchley, Wingenbach, & Akers, 2013). However, results for student success have ranged from lowered success rates and student outcomes in online courses compared to traditional face-to-face courses (e.g., Brown & Liedholm, 2002;Murphy & Stewart, 2017), to no differences (e.g., Buckley, 2003;Neuhauser, 2002), and even heightened success for those online (e.g., Means et al., 2009).
It should be noted that "success" has had a wide range of meanings within the literature comparing the efficacy of instructional formats. Often studies have compared academic performance, in the form of grades or grade point average (GPA), and course completion or retention rates. Yet, these measures have not always been standardized, with researchers noting differences in the calculation of grades and course completion/retention across studies (Howell, Laws, & Lindsay, 2004). Also, contemporary researchers have argued for the need to expand research methodology to include the measurement of variables that may predict success in the classroom, such as student perspectives, attitudes, or preferences towards the instructional format itself (e.g., Buchanan & Palmer, 2017;Ganesh, Paswan, & Sun, 2015;Gundlach, Richards, Nelson, & Levesque-Bristol, 2015;Keramidas, 2012). Therefore, in the current review of the available literature, findings for student preferences and attitudes, as well as variables (at the individual and course level) that may impact success in both online and face-to-face formats, are considered alongside research investigating traditional success measurement areas (e.g., grades or GPA).

Review of Relevant Literature
In its simplest form, course delivery in higher education can be categorized into one of three instructional formats: traditional face-to-face learning, online learning, and blended learning. Each instructional format can use technology. However, in comparison to a traditional face-to-face classroom where student-teacher interactions are conducted in a physical classroom setting, online learning involves the learners (for the most part) being physically separated from their teacher(s), with interactions carried out via synchronous or asynchronous two-way communication (Bernard, Borokhovski, Schmid, Tamin, & Abrami, 2014). Blended learning, on the other hand, transitions between traditional and online learning, involving student-teacher interaction in both face-to-face and online formats, and integrates learning technologies that are most suitable for achieving learning outcomes (Alexander et al., 2019). A course is often considered blended when the amount of online time replaces 30% to 79% of the total course time (Allen, Seaman, & Garrett, 2007).
The current study focuses on the comparison of traditional (face-to-face) learning and online learning, omitting blended learning from its investigation. The rationale for the omission of blended learning from the study was based on (a) the large variation in what can constitute a blended course in the research literature (Allen, Seaman, & Garrett, 2007;Brown, 2016), and (b) the inability to accurately document the extent to which a course was blended due to use of secondary (institutional) data sources in the main analyses.
Studies investigating differences between online and traditional face-to-face courses have produced incredibly mixed findings, ranging from positive gains to potential negative impacts of online course format (Bernard et al., 2004;Means et al., 2009). Studies that have found a positive impact of online learning suggest that online formats increase students' performance on course assignments and examinations (Barak et al., 2016;Faulconer et al., 2018;Greenway & Makus, 2014;Means et al., 2009;Soffer & Nachmias, 2018), increase levels of satisfaction and/or engagement in the course (Cummings, Chaffin, & Cockerham, 2015;Soffer & Nachmias, 2018), and help engage students in higher levels of discussion (Chen et al., 2015), as well as in deeper forms of learning (Manning-Ouellette & Black, 2017).
The mixed findings across multiple studies and academic settings suggest that there may be complex underlying factors influencing student outcomes, perceptions, and retention in online and traditional face-to-face instructional formats. The following sections separate the available literature based on discrete variables (at both the individual and course level) that have been shown to impact student success.

Factors Influencing Student Success
Factors explored by researchers attempting to understand differences between online and face-to-face instructional formats can be grouped into one of two main categories: individual or course related. Individual factors commonly relate to student characteristics such as their gender, race, age, performance level, and prior experience, whereas course-related factors commonly refer to the design, structure, or implementation of the course itself.
Having said this, some researchers have suggested that gender may not influence student success and outcomes as much as the structure of the course itself (Maceli & Foligasso, 2011), with some also noting that the effect of gender is reduced for older students (Daymont & Blau, 2008;Vella et al., 2016). Further, although general findings indicate females to be at an advantage compared to males, research has evidenced females to be at an increased risk of course dropout (Stoessel et al., 2015), as well as showing lower performance in online courses compared to faceto-face formats (e.g., Flanagan, 2012;Wladis et al., 2015).
Comparisons of student perceptions of online and face-to-face environments have revealed gender differences. In online environments, female students report a stronger sense of community and are less likely to view the course as being performance focused compared to males; whereas, in face-to-face classes this is reversed, with males viewing the course as more communal and less performance focused/centered (Yang, Cho, & Watson, 2015). Alongside this, gender differences have been observed in the level of effort students report giving during a course. Specifically, male students report higher levels of effort compared to female students in online courses, whereas females report giving more effort in face-to-face courses compared to males (Yang, Cho, Mathew, & Worth, 2011).

Race.
In regard to race, minority students (e.g., Hispanic or African American/Black) are often found to have lower academic performance than White students in traditional face-to-face classrooms (Farruggia et al., 2016). This finding has also been replicated in online STEM courses, with Wladis et al. (2015) reporting that Black and Hispanic students performed worse than White and Asian students. Additionally, minority students have been found to be less likely to enroll online (O'Connell, Wostl, Crosslin, Berry, & Grover, 2018;Wladis et al., 2015).
Comparison studies of online and face-to-face courses suggest that instructional format has little effect on observed achievement differences between racial groups, with minority students performing at a lower level compared to their peers regardless of instructional format (e.g., Richardson). However, it should be noted that racial differences have not always been observed in prior research comparing online and face-to-face classrooms. For example, the findings of O'Connell et al. (2018) indicated that regardless of race, the rate at which students dropped or failed a course was much higher in online sections compared to face-to-face.
Age. In distinguishing differences in age, studies commonly compare traditional and nontraditional students, with nontraditional students being defined as those over the age of 24 (National Center for Education Statistics, n.d.). In face-to-face settings, nontraditional students have been found to report working and studying for longer than their traditional peers, however, this does not translate into differences in performance (Woods & Frogge, 2017). Similar to findings for face-to-face courses, nontraditional students tend to be more academically engaged as well as have more positive perceptions of teaching and course interactions online (Rabourn, BrckaLorenz & Shoup, 2018). Additionally, nontraditional students have been found to be more likely to take classes online (Rabourn, BrckaLorenz, & Shoup, 2018;Wladis et al., 2015) and (for students with scholarships or without loans) to be less likely to withdraw from online courses compared to younger students (Cochran, Campbell, Baker, & Leeds, 2014).
Surprisingly, although nontraditional students have been found to be more likely to enroll in online courses (Tanyel & Griffin, 2014), studies have failed to find a difference between traditional and nontraditional students in their preference towards online and face-to-face formats (Woods & Frogge, 2017). Nonetheless, differences have been found between formats in students' performance level. Nontraditional students perform at a higher level online compared to face-toface, whereas traditional students show no differences in performance between instructional formats (Slover & Mandernach, 2018).
Performance level. Several studies have investigated whether differences between online and traditional course formats hold when students are split into high and low performing groups (e.g., based on overall course performance or GPA). Findings provide support for the negative effect of online learning across both performance groups, with the largest differences observed in low performing student populations (Bacolod, Mehay, & Pema, 2018;Lu & Lemonde, 2012;Sanford, 2017). Students with low GPAs are found to perform even worse in online courses compared to face-to-face (Cavanaugh & Jacquemin, 2015;Cummings, Chaffin, & Cockerham, 2015;Fendler, Ruff, & Shrikhande, 2016;Johnson & Palmer, 2015). However, in spite of this, low performing students tend to opt for online courses at a higher rate (Johnson & Palmer, 2015).
Lower levels of success in even one prior online course negatively impact course completion compared to those with no prior online experience (Cochran et al., 2014;. Interestingly, although students with a history of poor academic performance exhibit higher rates of withdrawal in online courses compared to face-to-face, they often choose to enroll in online over traditional face-to-face courses (Poellhuber, Chomienne & Karsenti, 2008; as cited in Murphy & Stewart, 2017).
Perceptions of course format. Examination of students' perceptions has focused on three main areas: flexibility or control in the learning process, level of interaction (student-student and student-teacher), and knowledge gained from the course (Platt, Amber, & Yu, 2014). These areas have not only been linked to enrollment choice (e.g., interaction level; Crews, Bordonada, & Wilkinson, 2017) but also student outcomes such as satisfaction (e.g., flexibility of course; Soffer & Nachmias, 2018). Overall, studies have found students to perceive online courses to be more flexible, and face-to-face courses as offering higher levels of interaction. However, there appears to be less consensus regarding knowledge gained from a course (Platt, Amber, & Yu, 2014).
Further, efforts to understand which format students prefer has also evidenced mixed findings. Research has shown preferences toward both traditional (Barak, Hussein-Farraj, & Dori, 2016;Blau et al., 2018) and online courses (Brown, 2012;Galy et al., 2011), as well as no difference in preference (Chen, Jones, & Xu, 2018). This is further complicated by the suggestion that preferences may depend on individual characteristics. For example, Wood and Frogge (2017) noted that face-to-face instruction was the top choice for traditional students, while nontraditional students preferred online courses.

Course Factors
Participation. Studies investigating success in traditional face-to-face formats commonly assume that students are attending class, but this may not always be the case. Murphy & Stewart (2017) note that despite the relationship between lecture attendance and higher academic achievement, student absenteeism remains an issue. Students often decide not to attend class due to other coursework, low interest in the course topic, poor perception of teaching quality, and/or availability of course materials outside of class. Similarly, in online classrooms a student's level and quality of participation impacts their learning outcomes. Participation in higher-order online learning activities has been linked with increased grades. Low performers commonly exhibit lower levels of participation across learning activities (Banoor, Rennie, & Santally, 2018), as well as displaying lower levels of persistence (Faulconer, Griffith, Wood, Acharyya, & Roberts, 2018). Further, those who complete courses successfully exhibit higher levels of engagement in online learning activities than those who withdraw (Morris, Finnegan, & Wu, 2005).
Course structure. Students' perceptions of the online learning environment and course structure have been highlighted as fundamental to student satisfaction as well as learning outcomes (Beach, 2018;Blau et al., 2017;Eom and Ashill, 2016;Galy, Downey, & Johnson, 2011;Gering, Sheppard, Adams, Renes, & Morotti, 2018). Students have reported that one of the most important factors for their success online is the course configuration, including easy access to materials, clear due dates, regular announcements, and distributed assignments (Beach, Stefanick and VanOverbeke, 2018;as cited in Beach, 2018). Furthermore, student perceptions of e-learning tools (perceived usefulness, perceived ease of use, and ability to work independently) have been found to be related to final course grades (Galy et al., 2011), as well as satisfaction with the overall course, and retention rate (Park & Choi, 2009).
A well-organized course, although desirable, may not always impact student grades (Jaggars & Xu, 2016) or satisfaction (Simpson & Benson, 2013). However, the organization of a course does provide a framework to engage students with each other, the content, and the instructor, which have all been shown to be important for positive student outcomes (Bernard et al., 2004;Jokisimovic et al., 2015) and satisfaction (Kuo & Belland, 2016;Kuo, Walker, Belland, & Schroder, 2013;Kuo, Walker, Schroder, & Belland, 2014).
Research comparing the structure/components of online and face-to-face formats has identified some subtle differences between modalities. For example, in work by Brocato, Bonanmo, and Ulbig (2015), pedagogical aspects such as teaching fundamentals, developing a relationship with students, providing stimulating content, and timely feedback were found to predict students' perceptions of a course/instructor in both online and face-to-face formats. Whereas the pedagogical variable of challenging students in their learning, as well as the course complexity/level, were only found to be predictors of evaluations for traditional face-to-face formats. In addition, findings on student perceptions of course structure has indicated that although students in online courses reported a better understanding of the course structure itself, those in traditional courses rated reading materials and supplementary materials as contributing more to their understanding than students online (Soffer & Nachmias, 2018).
Instructor. Student's perceptions of the instructor are important. Having an instructor that is flexible, supportive, and communicates frequently and effectively has been cited by students as one of the most important components of a successful online course (Beach, Stefanick, & Van Overbeke, 2018, as cited in Beach, 2018. Nevertheless, online instructors have been shown to receive lower instructor performance ratings from students for their teaching (Brocato, Bonanno, & Ulbig, 2015).
Another important aspect to consider is the opinions held by instructors themselves. Large survey-based studies of higher education institutions have been conducted to understand viewpoints of academic leaders in regard to online education, finding a surprising level of skepticism amongst faculty (Allen & Seaman, 2013;2015;2016). Records dating from 2003 to 2015 indicate some decrease in negative perceptions of online education; however, in 2015 there were still over 25% of faculty who reported that they believe learning outcomes in online education to be inferior compared to face-to-face education (Allen & Seaman, 2016). Further, only a small portion of all academic leaders (approximately 60%) reported that their faculty "accept the value and legitimacy of online education" (p. 26). Most believe that online learning leads to increased access and flexibility for students, as well as providing an opportunity for innovative teaching methods; however, faculty cite challenges in finding adequate resources (including training, instructor resistance, and specialist support) for online learning, especially for smaller institutions (Bates et al., 2017).
When asked to compare online and face-to-face formats, faculty reported the development of an online course as more time intensive and providing less feeling of control when teaching (Chiasson et al., 2015). Of greater importance, they cited the retention of students as a greater challenge for online courses than for face-to-face courses, as well as that students need more selfdiscipline to succeed online (Allen & Seaman, 2017). Having said this, faculty also reported increased confidence and improvements with their instruction in their face-to-face courses as a result of teaching online (Chiasson et al., 2015), and overall faculty ratings of online instruction are comparable to, or better than, face-to-face instruction (Allen & Seaman, 2017).

Project Overview
Based on the available literature, it appears that the mixed findings when comparing student success (i.e., course completion, student perceptions, and satisfaction) in online and traditional learning environments are further complicated by multiple variables relating to students (gender, age, race, prior experience) and the courses they take (participation, course structure, instructor). The ambiguity of these findings is heightened by (a) studies often being in isolation both in setting (e.g., studies of single courses or courses within a department) and construct/factors examined (e.g., studies that focus on a single variable), as well as (b) researchers questioning study methodology (e.g., difficulty in interpretation of student outcomes due to large variations in grade calculations).
Based on these mixed findings and the anecdotal evidence that their institution's undergraduate students enrolled in on-campus course sections perform better than students in the DE sections, the researchers identified the need to focus on understanding differences in student success between face-to-face and online course formats at their university. In particular, the current study had three main goals: (a) Investigate potential differences in student success rates between online and face-to-face courses for campus-based undergraduate students.
(b) Understand whether differences in online and face-to-face courses (if observed) vary across specific student populations (e.g., race, gender, age, students receiving aid) or course types (e.g., general education program (GEP) versus major courses).
(c) Provide an overview of student perceptions of online education over the prior five academic years.
Alongside the goal to provide information that will aid institutional efforts focused on the quality of online/DE experiences, promotion of access to higher education, and improvement of educational attainment rates at the sampled university, our study aimed to expand upon the available prior literature on three fronts: (a) by examining all courses at the university administered in both online and face-to-face formats, rather than a single college, department, or academic area; (b) by using a binary grade outcome variable to reduce the systematic biases in grading across academic departments or colleges (Dziuban, Graham, Moskal, Norberg, & Sicilia, 2018); and (c) by controlling not only for multiple individual-related variables (race, gender, financial aid status, Pell grant status, transfer status, academic standing, GPA, prior academic experience) but also course-related variables (credit value, course level, and general education program status) in the same model. Further, we reduced any potential student population bias by looking only at fulltime on-campus students.

Methods
The current study uses two secondary data sources from a public, four-year, research one and land grant university in order to explore the research goals: (a) institution level data on courses taught in both online and face-to-face formats, and (b) existing survey response data for students who have taken online courses at the university.

Course Data
Grade data from fall 2012 to fall 2017 was collated for full-time, on-campus, undergraduate students enrolled in courses that had a minimum enrollment of 10 students and were administered both online and face-to-face (N = 471,205). To control for variation in course design that occurs in specialized course offerings and shortened formats, student data was removed for courses that were intended for seniors or graduate students (i.e., courses numbered above 300-level) (n = 45,677), counted for 5 or more credit hours (n = 1), labelled as a physical activity, studio format, student teaching, recital/ensemble, or Agricultural Institute (AGI) course (n = 22,541), or taught in a shortened format (e.g., during summer sessions) (n = 2,052). Additionally, due to restructuring/changes in two colleges (and some of their corresponding departments) between the 2012-13 and 2013-14 academic years, course data (n = 80,677) had to be removed from the analyses to avoid cross-contamination of subject level data across academic areas (defined by catalog and course number).
Following this student data was removed for those outside of undergraduate status (n = 3,106) as well as those who received a grade outside of ABCDFW (n = 12,204). The data set was then reviewed to remove any courses that (a) did not include both an online and face-to-face section, or (b) exhibited low student numbers in online sections (i.e., less than 100 cases coded as online) (n = 34,007). The decision was made to remove courses with low online student numbers as it indicated that the course had not been taught consistently in both formats across the time period investigated (fall 2012 to fall 2017). This left a final sample of 270,939 cases spanning across academic areas of agriculture and life sciences (ALS), division of academic and student affairs (DASA), humanities and social sciences (HASS), natural resources (NR), sciences, management (MGMT), and textiles (TEX). See Table 1 below for demographics.

Student Survey Data
Secondary data were collated from a biannual survey administered to students enrolled in DE courses (online and site-based). The purpose of the survey was to gauge the quality of DE students' experiences, and to measure the effectiveness of services and support toward helping them achieve their academic goals. Data were collated from surveys administered between fall 2013 and fall 2018 (N = 5,761). Each survey was sent out to a stratified sample of students taking DE courses (30,742 students total across fall 2013 to fall 2018). Response rates for academic years varied from 14% to 22%. The initial data set was cleaned to remove any cases involving students with an academic standing other than undergraduate (i.e., graduate or nondegree; n = 1,901), as well as responses relating to courses numbered above 300-level or labelled as AGI courses (n = 272). This resulted in a final sample of 3,588 cases spanning across academic areas of ALS, design, engineering, HASS, NR, sciences, MGMT, and DASA. Available demographic information for students completing the survey was limited to age (nontraditional/traditional), academic standing and prior online experience. See Table 2 below for full demographics. A large majority of students were traditional age (88.8%) upperclassmen (73.76% junior or senior) and had not completed an online course before (30.34%; 18.86% had prior experience online, 50.79% did not respond to the question).

Course Data
For the analysis of course data, instructional format (online/face-to-face) was chosen as the independent variable and student success as the outcome variable. Online courses were defined as those courses designated in the university's student information system (SIS) as online with the instruction mode of internet (IN). Traditional face-to-face courses were defined as those courses designated in the SIS with an instruction mode, in-person (IP). A binary grade variable was created for student success by categorizing letter grades into two categories: pass (or ABC) and fail (or DFW). A binary grade was chosen for performance based on prior research practices that have suggested that this approach can mitigate the systematic biases in grading across academic departments or colleges (Dziuban, Graham, Moskal, Norberg, & Sicilia, 2018).
To isolate the effects of instructional format (online or face-to-face) on student success several variables relating to both students and the course were measured. These variables were selected based on the institutional data available and prior research that has noted factors relating to students, course/program design, and environment that impact student outcomes in online and face-to-face environments. Continuous variables were used to measure constructs of cumulative GPA and credits passed. Binary variables were created to measure student factors of age (traditional/nontraditional), race (white/nonwhite), gender (female/male), academic level (upperclassmen/underclassmen), transfer status (transfer student/not a transfer student), financial aid status (received aid/did not receive aid), and Pell grant status (received grant/did not receive grant), as well as course-related factors of course level (300 level/other), course credit (3-credit/other), and general education program status (GEP course/non-GEP course).

Student Survey Data
For the purposes of the current study, four general areas from the student survey were used to understand the climate surrounding online instruction at the university. These areas were selected based on their close fit with the current study aim to understand students' experiences and perceptions of the DE courses they were enrolled in. Other areas that asked students about general services provided at the university level were not included due to lack of alignment with study aims.
• Course format: Students were asked to indicate how their course was being taught, why they chose to enroll in their chosen course format, as well as specify their level of preference for both online and face-to-face formats. • Instructional technology: Student perspectives of the instructional technologies used in their course were sampled, including (a) if they felt prepared to use instructional technologies entering the class, (b) the level of ease and reliability of instructional technologies used, and (c) their overall level of satisfaction with instructional technologies. • DE Experience: Students were asked several questions relating to their experience in online courses, including (a) if technologies facilitated prompt feedback, enhanced problem solving, met their learning needs, and contributed to their learning, (b) if they felt more comfortable interacting with classmates and asking their professor questions online (compared to other course formats), as well as (c) if the course (and specifically the technologies used) helped prepare them for the workplace.
• Course instructor technology use: Students rated the extent to which their instructor effectively used technology to support academic success and facilitate group work/collaboration, as well as if they felt the instructor had the technical skill to carry out online instruction.

Course Data
To compare the occurrence of passing (ABC) grades across traditional face-to-face and online courses, binary logistic regression was used. The overall model comprised grade (ABC or DFW) as the outcome variable and class format (online or face-to-face) as the main predictor. Further, the analysis controlled for individual-related variables of race, gender, financial aid status, Pell grant status, transfer status, academic standing, GPA, credit passed, and course-related variables of credit value, course level, and GEP status. Interaction terms of age x format, race x format, gender x format, aid x format, and GEP x format were added to a second iteration of the model to understand if the effect of instructional format varied across different subpopulations.
When statistically significant interactions were found, secondary models were analyzed to tease apart the interaction effects. Interaction analyses investigated two aspects: (a) if there were differences across online and face-to-face courses for specific groups of students (e.g., for age x instructional format interactions, differences between online and face-to-face courses for (1) traditional and (2) nontraditional students) and (b) if there were differences between these student groups (e.g., for age x instructional format interactions, comparisons between traditional and nontraditional students) within online and face-to-face courses respectively. Odds ratios (OR) were used to understand the size of effects observed within interactions. In the context of the current study, an odds ratio below 1 indicates a group/person to be less likely to obtain an ABC, and an odds ratio above 1 indicates a group/person to be more likely to obtain an ABC. See Table 3 below for classification of effect sizes. For both models, data met the assumptions of logistic regression in that: (a) the dependent variable (ABC) was binary, (b) observations were independent of each other, (c) there was little multicollinearity among the independent variables, and 4) the analysis was conducted on a large sample size. In regard to model fit, following the acceptable ranges in Field (2009), models showed a good level of fit with the data. In particular, for each model it was observed that Cook's Distances was less than 1, DFBeta for the constant and predictors was less than 1, Leverage was close to expected value, and standardized residuals had values of less than ±2. All models showed a significant change in -2LL.

Student Survey Data
In order to enumerate overall perceptions of online courses, student survey data analyzed and reported in the current study was collapsed across academic years. As the biannual student survey (and its contents) has been modified over the selected sampling period, data for each measure were not available for every academic year. In the reporting of frequencies, missing data were removed from the calculation to avoid misrepresentation of the responses given by students.
Descriptive statistics were used to describe trends in students' perceptions of instructional technologies, their experience in online courses, as well as their perceptions of online instructors' proficiencies with instructional technologies. To compare reported levels of preference between online and face-to-face formats, Wilcoxon signed ranks analyses were used. For the analysis, response options were recoded into numerical values for preferences towards both face-to-face and online course format (strongly prefer = 5, slightly prefer = 4, neutral/it depends = 3, slightly do not prefer = 2, strongly do not prefer = 1). Data met the general assumptions for analysis using Wilcoxon signed ranks as (a) the dependent variable was ordinal, (b) the independent variable consisted of two related groups, and (c) the distribution of the differences between the two groups was closely related in their shape (Laerd Statistics, 2018). Further, to understand the size of effect observed, the r statistic was calculated. For the purpose of the analyses, 0.1 was considered a small effect, 0.3 a medium effect, and 0.5 a large effect (Field, 2009).

Course Data
Overall model statistics can be found in Table 4. The general model showed instructional format to be a significant negative predictor (β = -.24, p < .001) of success, with those online being 1.27 times less likely (OR = .79) to pass the course (receive an ABC) compared to those in a traditional face-to-face format. Additionally, the interaction model revealed significant interactions between course format and variables of age (p < .001), aid status (p < .001), and GEP course status (p < .001).

Age x format interaction.
Comparisons between online and face-to-face formats for nontraditional and traditional students showed differing results. For traditional students, those online were less likely to receive an ABC grade compared to those face-to-face (OR = .73). However, for nontraditional students this was reversed, with those online being more likely to receive an ABC grade compared to those face-to-face (OR = 1.29). When comparing nontraditional and traditional students within each format, no differences were observed for either face-to-face (OR = 1.10) or online formats (OR = .85). Aid x format interaction. Findings for comparisons between online and face-to-face formats differed based on aid status, with those not receiving aid being less likely to receive an ABC online compared to face-to-face formats (OR = 0.73). Those receiving aid showed no significant differences (OR = 0.95). When comparing students who received aid and those who did not within online and face-to-face formats, students receiving aid were found to be less likely to obtain an ABC in face-to-face courses compared to those not receiving aid (OR = 0.77). However, this difference was not replicated in online formats (OR = 0.95).

GEP x format interaction.
Comparisons between online and face-to-face formats for GEP and non-GEP courses revealed those online as less likely to receive an ABC compared to those in face-to-face formats in non-GEP courses (OR = 0.73), but not GEP courses (OR = 1.07). When comparing GEP and non-GEP courses, contrasting results were evidenced, with those in GEP courses being less likely online (OR = 0.59), and more likely in face-to-face formats (OR = 1.33) to receive an ABC compared to those in non-GEP courses.

Student Survey Data
Course format. The majority of DE students sampled during academic years (AY) 13-14, AY 14-15, and AY 15-16 reported that their course was being delivered via the internet (average percentage 86.8%). Students provided a range of reasons why they enrolled in their chosen online course. The most frequently cited reasons for enrolling online were (a) to make progress toward their degree/certificate (66.5%), (b) to make progress towards their professional development/career goals (54.8%), and (c) due to course schedule conflicts (57.7%). Students also reported taking a course online as an open section was not available on campus (37.5%), for personal enrichment (38.6%), due to the course only being offered via DE (10.5%), as well as to enhance their job opportunities after losing a job (9.6%).
To compare student preferences between online and face-to-face formats, data from AY 14-15 through AY 16-17 were analyzed. Results from Wilcoxon signed rank tests indicated that individuals showed higher levels of preference towards face-to-face course formats compared to online (z = -11.88, p < .001, r = -.36).

Instructional technology.
When students were asked about their perspectives on the instructional technologies used in their course (data collected each year from AY 14-15 to AY 17-18), a large proportion (81.5%) agreed that they felt prepared to use instructional technologies entering the class (10.2% neutral, 8.3% disagree).
With regards to the reliability and ease of use of instructional technologies in their course, students surveyed in AY 13-14 felt course technologies were both reliable (94% rated technology good or excellent, 5.2% fair, 0.8% poor) and easy to use (91.2% good or excellent, 7% fair, 1.8% poor). Further, when reporting on the ease of use for specific technologies (during AY 14-15), student perceptions of course learning management system (LMS), video lectures, and online collaboration tools were positive. Although almost half of students reported not using online collaboration tools (42.5%), those who did use such a tool rated them as easy to use (82.1% of those who used the tool responded "easy"). Table 5 below lists full frequencies for ease of technology use.
In the most recent DE student survey (AY 17-18), students rated their satisfaction with specific technologies (LMS, video lectures, online communication tools, and online collaboration tools). Students reported the highest level of satisfaction with the LMS (89.4% satisfied, 8.8% neutral, 1.8% dissatisfied), followed by video lectures (78.1% satisfied, 12.5% neutral, 9.4% dissatisfied), online collaboration tools (76.8% satisfied, 5.4% neutral, 17.9% dissatisfied), and online communication tools (69% satisfied, 6.9% neutral, 24.1% dissatisfied). DE experience. During the survey in AY 13-14, students were asked to reflect on their experience in online courses. Student perceptions of their online experience were positive, with a large majority of students agreeing that the instructors' use of DE technologies facilitated prompt feedback (86.9% agree, 13.1% disagree), enhanced problem solving (75.3% agree, 24.7% disagree), and met their learning needs (77.8% agree, 22.2% disagree). Further, a majority of students surveyed between AY 14-15 and AY 17-18 felt the course (and specifically technologies) had prepared them for the workplace (57.3% agree, 32% neutral, 10.7% disagree). Smaller proportions of students reported that they felt more comfortable interacting with classmates (54.8% agree, 45.2% disagree) and asking their professor questions online (59.6% agree, 40.4% disagree).

Perceptions of use of instructional technology during course.
In both AY 14-15 and AY 15-16, students were asked whether they felt the instructor effectively used technology in their classrooms. A large proportion of students had positive perceptions of their instructors, with students agreeing that the instructor effectively used technology to support academic success (80.7% agree, 11.6% neutral, 7.7% disagree) as well as had the technical skill to carry out online instruction (80.9% agree, 11.2% neutral, 7.9% disagree). A smaller proportion of students agreed that the instructor used technology to facilitate group work and collaboration (59.1% agree, 17.2% neutral, 23.7% disagree).

Discussion
The current study investigated online courses offered at a public four-year university. In particular, through the use of existing grade and student survey data, the study sought to both explore potential differences in student success rates between online and face-to-face courses for full-time, on-campus students as well as understand current student perceptions of online education at the university.

Course Data
Our findings in relation to grade data support prior literature that has shown differences between online and face-to-face classrooms at the college level (Bernard et al., 2004;Means et al., 2010). In particular, they support the assertion that undergraduate students perform at a lower level in online classrooms (Amro, Mundy, & Kupczynski, 2015;Bettinger et al., 2017;Brown & Liedholm, 2002;Buchanan & Palmer, 2017;Emerson & MacKay, 2011;Flanagan, 2012;Ganesh, Paswan, & Sun, 2015;Gundlach et al., 2015;Helms, 2014;Murphy & Stewart, 2017). The overall model found that students are 1.27 times less likely to obtain an ABC in online courses compared to face-to-face courses.
The current study adds to the literature by investigating how the impact of instructional format is influenced by variables of age, race, gender, financial aid status and general education course status. Although not all interactions were significant, the results for interactions between instructional format and variables of age, race, and GEP status nevertheless revealed some interesting findings for student populations at the sampled university.
Contrasting findings were observed for traditional and nontraditional students when comparing instructional formats in the age x instructional format interaction. Findings for nontraditional students match prior literature (e.g., Slover & Mandernach, 2018), in that nontraditional students were found to be more likely to achieve an ABC online compared to F2F (OR = 1.29). However, the finding that traditional students were less likely to achieve an ABC online compared to face-to-face (OR = .73) contradicts prior work that has found no differences in performance between instructional formats (e.g., Slover & Mandernach, 2018). Although it should be noted that the observed effect sizes for the interaction between age and instructional format are small in the current study, the discrepancy in findings may be based on student population examined, with prior research cited in the literature review being conducted with specific subpopulations (e.g., business majors in Slover & Mandernach).
When comparing traditional and nontraditional students within a given delivery format, no differences were observed for face-to-face or online formats. The lack of an age effect for performance within online or face-to-face courses resembles prior literature that has shown differences in engagement, but not in overall performance levels for traditional and nontraditional students (e.g., Woods & Frogge, 2017).
The current study extended knowledge in the field through its exploration of interactions between instructional format and variables of student financial aid status and general education course (GEP) status. Due to a lack of prior research that has included interactions between financial aid and format, our findings provide an initial insight into the impact of instructional format for differing financial aid groups. Results suggest that receiving financial aid impacts observed differences in student success between online and face-to-face formats. The model indicated that for those not receiving aid, students were less likely to receive an ABC in online courses compared to face-to-face (OR = .73). However, this was not replicated for those with aid. When comparing students who received financial aid to those who did not, findings indicated that those receiving aid were less likely to receive ABC in face-to-face settings. This finding contradicts the general literature on financial aid that has suggested receiving aid leads to improved academic performance (e.g., Stater, 2009). However, these results should be interpreted with some caution due to the effect sizes being relatively small (OR = .77). Alongside this, no differences were observed based on financial aid status in online courses. In the context of the current model/interaction, it could be argued that the smaller odds ratios observed between aid groups in online settings compared to face-to-face settings is a result of those without aid showing decreased performance online, rather than a positive effect of aid in online settings.
Based on studies comparing general education program (GEP) status having not previously been reported, our findings from the GEP x format interaction provide an initial view into the impact of instructional format for differing GEP course status. Results suggest that the GEP status for a course impacts observed differences between online and face-to-face formats. Our model indicated that for non-GEP courses (OR = .73), students were less likely to receive ABC online compared to face-to-face. However, this was not replicated for GEP courses. When comparing GEP and non-GEP courses, contrasting results were evidenced, with those in GEP courses being less likely online (OR = 0.59), and more likely in face-to-face formats (OR = 1.33) to receive an ABC compared to those in non-GEP courses. As the model indicated that students in non-GEP courses were less likely to receive an ABC in online courses, and students in GEP courses show no differences in formats, it seems contradictory that those in non-GEP courses are found to be less likely online, and more likely face-to-face, to receive an ABC grade. Unfortunately, based on the limitations of the current data and analyses, it is unknown why this may have occurred and warrants further investigation in future research.
Limitations and future directions. Amongst literature investigating online education there is some debate over whether comparative studies between online and face-to-face instruction should be conducted. Bernard et al. (2004) outline in their meta-analyses that comparative studies are useful only when comparison conditions (i.e., traditional classrooms) are clearly outlined and similarities and differences are acknowledged. Due to the type of data available (and limits of variables measured), the current study could not control for all variables cited in prior work. It is hard, therefore, to determine the existence of potential confounds across treatments which makes the interpretation of our results more difficult. Further research is needed to control for variables relating to course design and structure. In particular, surveying instructors and examination of course syllabi may be beneficial in understanding and categorizing the type and quality of instruction in both course formats.
A second limitation of the analyses is that our overall model is influenced by large amounts of student data from academic areas of the sciences (38.77%) and humanities and social sciences (33.82%). While our model is representative of the institution sampled for the study, based on the diverse nature of online programming across institutions, our ability to generalize the findings to the wider higher education population is somewhat hindered. Further, the specified models were limited to five two-way interaction terms. Although our analyses control for the potential impact of a number of covariates, our models are not able to shed light on the complex potential relationship between age, race, gender, and financial aid status, and the additive effects of these on the interaction terms we studied in our specified models. It is recommended that future research not only expand data collection to academic areas missing in the current study, but also investigate narrower sub-populations of students through the use of three (or four) way interactions within its analysis methods.

Survey Data
Student survey data collected in the current study provided a window into the perceptions of students who take courses online and the climate surrounding online education at the current institution. Research shows that there is a strong link between student satisfaction, retention, and college completion rates (Ruffalo Noel Levitz, 2018), which underlines the importance and rationale for including student perception and satisfaction. Although all the students surveyed were currently enrolled in an online course, on average they were found to report a higher level of preference for the face-to-face compared to the online course format. The medium effect size observed in the current study (r = -.36) supports prior research that has indicated that students prefer face-to-face formats over online (e.g., Diebel & Gow, 2009). A possible explanation for this finding could be that students' preferences toward instructional format are influenced by their perceptions of student-student and student-teacher interactions (Crews, Bordonada, & Wilkinson, 2017). In the current study it was observed that a relatively small proportion of students felt more comfortable interacting with classmates and asking their professors questions in an online format. These negative views towards interactions in the online classroom may have led to lowered preference ratings towards online formats. Unfortunately, we cannot confirm this hypothesis in the current study due to the inability to compare student preferences toward online and face-to-face formats based on their view of online interaction. It is therefore suggested that future research should explore the influence on student perceptions of course elements (such as student-student and student-teacher interaction) on their overall preference toward instructional formats. Although students reported a higher level of preference for face-to-face formats, experiences online were largely positive. Students reported instructional technologies as reliable and easy to use, as well as that the technologies facilitated prompt feedback, enhanced problem solving, and met their learning needs. Alongside this, students had positive perceptions of their instructors' skill level and use of technology to support academic success. These results support literature that has shown high levels of satisfaction and engagement in online courses (Cummings, Chaffin, & Cockerham, 2015;Crews, Bordonada, & Wilkinson, 2017;Soffer & Nachmias, 2018), as well as those that suggest online learning can help engage students in higher levels of discussion (Chen et al., 2015) and deeper forms of learning (Manning-Ouellette & Black, 2017).
Limitations and future directions. From the current findings, it is unclear whether those in face-to-face classes had similar perspectives on quality standards for (a) their course structure, instructional technology use, and instructor (Crews, Bordonada, & Wilkinson, 2017), or (b) whether they would also exhibit differences between preference towards online and face-to-face courses. Further, due to the anonymity of the data collected, we were unable to connect study information to individual responses and examine the relation of students' performance to their perceptions of instructional format, as well as understand whether factors known to impact student success in online course sections (i.e., demographic variables, prior experience/performance, course structure) also impacted their perceptions and preferences. It is recommended that future studies that rely on institutional data also consider collecting supplementary information available to the institution (i.e., instructional design, online pedagogy, quality standards, self-selection, interaction, and feedback, and so on) (Crews, Bordonada, & Wilkinson, 2017) to allow for a more complex understanding of the impact of student perceptions with regards to online and face-toface instruction. Overall, findings from the literature and institutional level data in this study point to the need for future research to ask more specific questions, moving from the initial testing of whether there are differences in instructional formats, to (based on our findings) the how and why traditional undergraduate students enrolled in on-campus course sections perform better than students in online course sections.