Exploring Differences in Business Undergraduate Perceptions by Preferred Classroom Delivery Mode

The purpose of this study was to compare business undergraduate online/hybrid course perceptions across three different preferred classroom environment delivery modes: online, hybrid, or face-toface (F2F). Four different perceptions were measured: perceived favorability of online/hybrid courses (PFoOC); intent to recommend online/hybrid courses; perceived learning; and timely graduation. Undergraduates who were taking at least one online or hybrid class voluntarily completed an online survey. In the fall 2015 a complete-data sample (n = 264) of respondents was obtained and in the spring 2016 the complete-data sample (n = 272). Consistent results across both samples were found for three of four outcomes. Undergraduates who preferred either online or hybrid classroom delivery had significantly higher PFoOC and intent to recommend online/hybrid courses than students preferring F2F. There were no differences in perceived timely graduation across the three classroom delivery mode groups, and inconsistent results were found for perceived learning. The fall sample showed no differences on perceived learning but for the spring sample, undergraduates preferring either an online or hybrid delivery mode perceived higher learning than F2F preferred-mode students. A new, short four-item measure of PFoOC was found to be reliable. As universities increase their online and hybrid course offerings keeping course integrity or equivalence between F2F and online/hybrid course sections will be important. Study implications for studying future online student perceptions are suggested. Ways to increase the PFoOC for undergraduates who prefer F2F are suggested. Increased PFoOC should lead to higher intent to recommend online/hybrid courses.


Introduction
Many universities and colleges are increasingly viewing online education as a critical component of their enrollment strategic plan to accommodate undergraduate students' needs (Comer, Lenaghan & Sengupta, 2015).There has been strong growth over the past several years in colleges offering online courses and degree programs, which is expected to continue (Britt, 2015).However, as Raju and Schumacker (2014-15) noted, the percentage of college students graduating within five years in 2012 was 51.9%, which was a decrease from 54.4% in 1991 across all institutions.The purpose of this paper was to compare business undergraduate online/hybrid course perceptions across three different preferred classroom environment delivery modes: online, hybrid or face-to-face.Four different perceptions were measured: perceived favorability of online/hybrid courses; intent to recommend online/hybrid courses; perceived learning and timely graduation.After reviewing relevant literature, the research question to be addressed is posed.Then the methodology used, including samples, measures, and data analyses are presented.Next results-both descriptive statistics and tests of the research question-will be shown.After a discussion of the results, study limitations and implications of the results are offered.

Literature Review
Three different delivery modes are now used in undergraduate college courses: face-toface (F2F), online, and blended/hybrid.The blended or hybrid delivery mode is a combination of F2F and online, for example, alternating online and F2F meetings (Arbaugh, 2014).Prior research has compared these three delivery modes in various combinations across different disciplines, with varying results on outcomes.Common outcomes measured included grades (e.g., examination, final course), attitudes (e.g., engagement, satisfaction), and learning.For example, Johnson and Palmer (2015) found that students did better and were more engaged in a F2F versus online linguistics course.Using a student sample in an introductory business statistics course, Haughton and Kelly (2016) found that students in the hybrid environment performed better than students in the F2F environment on the common final exam.However, Helms (2014) found that online undergraduates performed more poorly in a required psychology course than F2F undergraduates.Using online and F2F undergraduate samples taking a common finance class, Fendler, Rubb and Shrkhande (2016) found that learning styles affected class performance.For example, balanced visual/verbal learners were more likely to be successful F2F than verbal learners, while verbal learners were more likely to succeed online than balanced learners.
Concerning other learning outcomes, e.g., learner satisfaction and learning motivation, results comparing online with F2F or hybrid with F2F have not shown consistent differences (Arbaugh, 2014).In perhaps the most comprehensive single study based on over 5,000 courses taught by over 100 faculty members over a period of 10 academic terms, Cavanaugh and Jacquemin (2015) found minimal differences in grade-based student performance between online and F2F student samples.Arbaugh (2014) also noted that typically studies do not compare all three delivery modes simultaneously.The present study used undergraduates taking at least one online or hybrid class and asked what classroom environment delivery mode they were most comfortable in, i.e., online, hybrid or F2F.Three of the four outcomes measured have not been as extensively studied in prior research, i.e., perceived favorability of online/hybrid courses (PFoOC), intent to recommend online/hybrid courses, and timely graduation.In addition, research controlling for variables that might also affect these outcomes is summarized, prior to the study's research question.

Measuring PFoOC
Prior research has asked about general learning comparisons between online versus F2F courses.For example, Eom, Wen and Ashill (2006, p. 233) asked about a general learning comparison between online and face-to-face courses in their three-item measure, (e.g., "I feel like I learn more in online courses than in face-to-face courses").One item from Sun, Tasi, Finger, Chen andYeh's (2008, p.1198) three-item "e-learning course quality" scale was "conducting the course via the Internet improved the quality of the course compared to other courses."Daymont, Blau and Campbell (2011, p.162) used one item to measure preference for online course delivery (versus face-to-face), "I would have registered for the online section if there had been space available in the online section of this course."However, prior studies have not compared specific features of online/hybrid versus F2F, for example, video lectures versus class lectures, written discussion board versus classroom participation, and live online discussion versus face-to-face classroom discussion.It is important to directly compare these specific features to measure the "integrity" of a course (Daymont et al., 2011), i.e., approximating the same content and process in an online or hybrid course as the F2F equivalent course.Using these specific item comparisons, Blau and Kapanjie (2016) found that the four-item PFoOC scale had a Cronbach alpha of .91 and .89 at two separate times.Cronbach alpha is a measure of internal consistency (or reliability).Ideally a measure should have an internal consistency of at least .70(Nunnally, 1978), with a still higher number (e.g., .80,.90)indicating greater reliability.

Less Prior Research Focusing on Intent to Recommend and Perceived Timely Graduation
Prior research (Endres, Hurtubis, Chowdhury & Frye, 2009) investigated the relationships of five different online student course satisfaction facets (faculty practices, course materials, learning practices, student-to-student interaction, and online tools) to their recommendation intentions.Endres et al. (2009) found that student satisfaction with faculty practices, learning practices and course materials were each positively related to intent to recommend the online course to other MBA students.Only MBA students taking online courses were sampled.The impact of preferred classroom environment delivery mode on perceived timely graduation has not yet been empirically tested.
Controlling for Background, Technological, and Course Variables Arbaugh (2005) presented a model for testing the increasing impact of three variable sets on student perceived online learning: (1) control variables (e.g., age, number of prior online courses taken); (2) technological variables (e.g., perceived ease of use); and (3) pedagogical/course variables (e.g., perceived course interaction).More recent studies, (e.g., Cavanaugh & Jaquemin (2015), Haughton & Kelly (2016), and Helms (2014)) have collectively controlled for background variables including: grade point average (GPA), number of prior online courses, age, and current course load.Two pedagogical variables-student motivation and instructor effectiveness-can also be important to control for (Sebastianelli, Swift & Tamimi, 2015).Student motivation was found to have a positive impact on user online course satisfaction (Eom et al., 2006).Such motivation can be measured by students' perceptions of hours spent preparing for a class.Instructor effectiveness (e.g., course organization, interaction with students) can impact online course student satisfaction (Johnson & Palmer, 2015).Controlling the impact of background, technological, and course variables when investigating the impact of preferred classroom delivery mode on outcomes allowed for stronger inference about the impact of classroom delivery mode.This study's design controls for seven variables: (1) age, (2) GPA, (3) number of prior online/hybrid courses taken, (4) current class load, (5) number of hours spent preparing for class, (6) perceived ease of use and (7) instructor effectiveness.Given the lack of prior empirical research, this study asked the following general research question (RQ): After controlling for seven variables, will there be differences between online versus hybrid versus F2F preferred classroom environment delivery modes on four perceived outcomes: PFoOC, recommend online/hybrid course, learning & timely graduation?

Method Sample and Procedure
In the fall of 2015, all business undergraduate students (n=3,292) enrolled in at least one online or hybrid course were contacted by school email address and asked to voluntarily fill out an online survey.Permission for data collection was granted by the University Institutional Review Board.The business school is part of a large urban state-supported University located in the mid-Atlantic region of the United States.As an incentive to voluntarily fill out the online survey, two prizes were offered, i.e., the choice of an Apple or Android watch, with the two winners to be chosen by random number lottery.Prior research has suggested that incentives can improve online survey response rates (Fan & Yan, 2010).A student could fill out a survey for each different online/hybrid course he or she was taking and the student's name was entered in the lottery for each completed survey.Only respondents who completed a survey were eligible to win.One general survey reminder was sent after several days.Six hundred and thirty-eight students (N = 638) filled out at least part of the online course survey.This represents a 19% response rate (638/3,292).Generally, participating students were juniors or seniors (69%), taking one (54%) or two (14%) online courses, along with traditional classes.As a "mixed delivery course format" sample, (i.e., taking face-to-face (F2F) as well as online/hybrid courses), this allowed respondents to directly compare these courses.Eighty-six percent of the sample consisted of full-time students (at least 12 credit hours/semester).Eight percent of the sample was taking one hybrid course.Fiftyeight percent of the respondents were white (42% non-white), and 48% were male.The survey was posted near the end of each course for one week.The same procedure was followed in the spring of 2016, with 546/2712 (20%) filling out at least part of the online course survey.A similar breakdown to the fall sample was found for the spring sample, e.g., 58% taking one and 15% taking two online courses; 90% of the sample were full-time students.Fifty-six percent of the respondents were white (44% non-white), and 48% were male.

Instrumentation
Survey content was developed primarily based on a literature review, and available measures were adapted when possible.Prior to survey administration, a small pilot test of item content for the measures below was conducted among two business school online program faculty and two interning business undergraduates.Several survey iterations were conducted to reduce the length of the survey.Prior research working with college student samples suggests that shorter duration (e.g., less than 13 minutes for online survey completion time) results in a higher response rate (Fan & Yan, 2010).All measures used a seven-point Likert response scale, (1 = strongly disagree, 2 = disagree, 3 = somewhat disagree, 4 = neither disagree nor agree, 5 = somewhat agree, 6 = agree, and 7 = strongly agree) unless otherwise noted.Cronbach alpha, which is a measure of internal consistency (or reliability), is reported below for each multi-item measure.

Measures
Control variables.Seven variables were measured: age; GPA; number of prior online/hybrid courses taken; current class load; number of hours a week spent preparing for class; ease of use; and instructor effectiveness.Age was measured in yearly response categories, from 1-18 years old or less to 34-51 or older.GPA (cumulative) was measured in incremental tenth response categories, e.g., 2.0, 2.1, where 1 = less than 2.0 to 22 = 4.0.Number of prior online/hybrid courses taken was measured from 0 to 9 or more.Current class load was measured from 1 to 6 or more.Number of hours a week spent preparing for class (including completing course assignments) was measured from 1 = less than one to 7 = 8 or more.Ease of use was measured using six items.Three technology tools used in online/hybrid courses were measured: (1) Blackboard/Canvas, (2) WebEx, and (3) Discussion Board (either Blackboard or Edmodo).Each of these three tools was assessed using two different lead-in phrases, adapted from Arbaugh (2005): (1) "each tool is/was easy for you to use, i.e., you are comfortable using it" and (2) "each tool was useful, i.e., that it helped you do well in your course."In addition to the 7-point response scale noted above, an eighth response option, i.e., "Not Applicable," was offered.If respondents selected this option, it was coded as missing data.Table 1 shows the survey items for multi-item measures.Using the two lead-in phrases and three technology tool items together, a six-item "perceived ease of use" scale was formed.Cronbach alpha for this scale was .85 for the fall sample and .89for the spring sample.Two items adapted from Sebastianelli et al. (2015) were used to measure instructor effectiveness.Cronbach alpha for this scale was .77for the fall sample and .81for the spring sample.
Preferred classroom environment delivery mode.Undergraduates were asked "in general, what is the most comfortable classroom learning environment for you (please select one choice below):" where 1 = online, 2 = hybrid (combination of online and face-to-face classes), and 3 = face-to-face (F2F).
Outcomes.Four variables were measured using multi-item measures: perceived favorability of online/hybrid courses (PFoOC); intent to recommend online/hybrid courses; perceived learning; and timely graduation.Items are shown in Table 1.PFoOC (versus F2F) was measured using 4 items and responses were made using the following scale: 1 = very inferior to 7 = very superior.Any "not applicable" response to an item was coded as missing data.Cronbach alpha for this scale was .85 for the fall sample and .90 for the spring sample.Intent to recommend online courses was measured using two study-specific items.Cronbach alpha was .76 for the fall sample and .81for the spring sample.Two items adapted from Alavi (1994) were used to measure perceived learning in online/hybrid course.Cronbach alpha for this scale was .76 for the fall sample and .87 for the spring sample.Timely graduation was measured using two study-specific items.Cronbach alpha was .76 for the fall sample and .80 for the spring sample.

______________________________________________________________________________
Ease of Use 1. Blackboard/Canvas is/was easy for you to use, i.e., it is not difficult, you are comfortable using it 2. WebEx is/was easy for you to use, i.e., it is not difficult, you are comfortable using it 3. Discussion Board (either Blackboard or Edmodo) is/was easy for you to use, i.e., it is not difficult, you are comfortable using it 4. Blackboard/Canvas was useful, it helped you do/perform well in the course 5. WebEx was useful, it helped you do/perform well in the course 6. Discussion Board (either Blackboard or Edmodo) was useful, it helped you do/perform well in the course Table 1.Survey Items for Multi-Item Measures delivery mode, and the split for the fall sample (n=264) was: online (n = 61); hybrid (n = 97) and F2F (n = 106).For the spring sample (n = 272) the split was: online (n = 58); hybrid (n = 92) and F2F (n = 122).
An overall multivariate test was performed.If that test was significant, this then allowed for testing the impact of delivery mode on each outcome.If this test for the impact of delivery mode on each outcome was significant, this then allowed for pairwise comparisons of delivery mode groups (Stevens, 1992).There are different post hoc pairwise group comparison test options, such as least significant difference (LSD) and Scheffe.The LSD test is generally regarded as too liberal, however, because it does not control for family-wise error rate (FWE), while Scheffe is generally regarded as too conservative because it uses a single range value for all possible comparisons, not just pairwise group comparisons (Stevens, 1992).An acceptable, common post hoc pairwise group procedure in SPSS-PC, controlling FWE when comparisons are independent in a research design (as here), is the Sidak test (Stevens, 1992), which was used.

Descriptive Statistics for Control Variables
Table 2 reports the background variables' descriptive statistics for the complete data, fall and spring samples.Across both samples, there were very similar self-reported cumulative GPAs and age, and a natural increase from fall to spring in prior online/hybrid courses.Instructor effectiveness was rated slightly higher in the fall, while current course load, number of hours per week spent preparing for class, and ease of use were slightly higher in the spring sample.a Number of hours/week spent preparing for class, 1 = less than one to 7-8 or more b Ease of use; Instructor Effectiveness, 1 = strongly disagree to 7 = strongly agree

Test of Research Question
The research question asked if there were differences between online versus hybrid versus F2F preferred classroom environment delivery modes on four perceived outcomes: PFoOC, recommend online/hybrid course, learning, and timely graduation.Table 3 reports the univariate tests on each outcome, and cell mean paired comparisons by preferred classroom environment delivery mode for the fall and spring samples.Prior to these results, all reported overall multivariate test results, testing the impact of delivery mode while controlling for the seven covariates (MANCOVA) were significant at p < .01.The eta-squared results in parentheses give an estimate of the percentage of variance accounted for in the dependent variables by delivery mode (Stevens, 1992).For the fall sample: Wilk's lambda, F = 4.91 (7%); Pillai's trace, F = 4.76 (7%); and Hotelling's trace, F = 5.06 (8%), and for the spring sample: Wilk's lambda, F = 7.86 (11%); Pillai's trace, F = 7.48 (11%); and Hotelling's trace, F = 8.24 (11%).With multivariate significance found, univariate results are then shown in Table 2.The fall and spring results are consistent for three of four outcomes.Significant univariate F tests (p < .01)were found for PFoOC, intent to recommend online/hybrid courses, and perceived timely graduation.The significant univariate F for PFoOC (Fall, F = 16.13;Spring, F = 20.26)allowed for pairwise cell mean comparisons.The cell means for the online (Fall,4.79;Spring,4.84) and hybrid (Fall,4.50;Spring,4.57)preferred classroom environments were both significantly higher than the F2F cell means (Fall,3.86;Spring,3.76).The significant univariate F for intent to recommend online/hybrid course (Fall, F = 5.40; Spring, F = 14.72) allowed for pairwise cell mean comparisons.The cell means for the online (Fall, 5.93; Spring, 6.09) and hybrid (Fall,5.84;Spring,5.76)preferred classroom environments were both significantly higher than the F2F cell means (Fall,5.45;Spring,5.21).The online cell means were slightly higher than the hybrid cell means on these two outcomes but there were no significant differences.There was not a significant univariate F test for perceived timely graduation for either the fall (F = 1.89) or spring (F = .95)samples.For perceived learning, inconsistent results were found.While the univariate F test was not significant for the fall sample (F = .20),it was significant for the spring sample (F = 9.53).Following up on this significant F test result for the spring sample showed that the cell means for the online (5.59) and hybrid (5.58) were higher on perceived learning than the F2F cell mean (5.02).Thus overall, there is partial support for the research question.

Discussion
There has been little online research thus far empirically testing undergraduate students' perceived favorability of online/hybrid courses (PFoOC), intent to recommend online/hybrid courses, or impact on perceived graduation.By controlling for seven covariates first, stronger inference can be made about the impact of preferred classroom environment delivery mode on each outcome.Results consistently showed that online and hybrid preferred classroom delivery modes each had higher PFoOC and intent to recommend online/hybrid courses than undergraduates in the F2F preferred delivery mode.However, there was no impact for delivery mode on perceived graduation.Inconsistent results were found for impact of preferred delivery mode on perceived learning.Overall, preferred delivery mode accounted for 7% of the variance in these four outcomes for the fall sample, and 11% of the variance in the spring sample.These results suggest that classroom delivery mode can have an impact on student perceived outcomes.
Perceived favorability of online/hybrid over face-to-face courses (PFoOC) is a new scale and showed a strong reliability (Nunnally, 1978).The items within this scale made more specific comparisons, i.e., video lectures versus class lectures, written discussion board versus classroom participation, and live online discussion versus face-to-face classroom discussion versus prior measures which are more general (Daymont et al., 2011;Eom et al., 2006;Sun et al., 2008).Future research using mixed course delivery format samples, i.e., students taking both face-to-face and online or hybrid classes, can utilize this perceived favorability measure for direct comparison.

Limitations and Directions for Future Research
There are several study limitations to acknowledge.Ideally, the way to study PFoOC would be to have the same instructor across each different course delivery of the same course.This would better control for instructor effectiveness.Instead, instructor effectiveness was measured as a covariate.There were lower cell size frequencies, especially for the online preferred classroom environment delivery mode.Both fall and spring samples were business undergraduates at a large urban state university.Testing this study's results using other samples, e.g., private, small college, non-business undergraduates, is important to see if there is generalizability.All data were selfreport.A one-factor test (Podsakoff, Mackenzie, Lee & Podsakoff, 2003) showed that for the fall sample, six factors had eigenvalues of at least one, and that the first factor accounted for 25% of variance.For the spring sample, six factors with eigenvalues of one were also found with the first factor accounting for 31% of variance.This indicates that method variance, while an issue, is not an overriding limitation.Future research collecting non-self-report data, e.g., examination or final course grades, would be helpful.Despite the use of incentives, there was a large loss in completedata sample size.One option to consider is requiring a respondent to complete a survey page before being allowed to continue to the next page.

Practical Implications and Conclusions
Beyond programs that are completely online, growing numbers of traditional F2F classroom undergraduates are taking online or hybrid courses as part of their education (Comer et al., 2015).Keeping course integrity or equivalence between F2F and online/hybrid course sections is important (Daymont et al., 2011).For example, if group projects are important in a F2F section of a course, such group projects should be kept in online/hybrid sections of that course.Classroom delivery mode had a significant impact on two student perceived outcomes, PFoOC and intent to recommend.However, there was no impact on perceived timely graduation and there was a mixed impact on perceived learning.It was also found that there were no differences on any perceived outcomes between the online versus hybrid preferred classroom delivery mode samples.Yet these were the two smallest samples (versus F2F), so additional research with larger samples sizes for both online and hybrid preferred classroom delivery modes is clearly needed in future comparisons.Although there was no significant difference on perceived timely graduation, the cell means were consistently highest for the online, then hybrid, and finally, F2F samples.As noted earlier, research has shown a decrease from 1991 to 2012 (Raju & Schumacker, 2014) in the percentage of students graduating within five years (from 54.4% to 51.9%).Therefore, the impact of offering increased online and hybrid course sections on not just perceived graduation but persistence towards graduation and actual graduation rates needs to be further studied.
Online courses offer students a way to conveniently take college classes without having to make the trip to campus.For working adults, veterans who are getting ready to take traditional classes, or those students that want to take a class during the summer while away at an internship, online classes allow them the flexibility they need in their busy schedules to persist towards graduation.Carefully constructed online classes, including consistent and organized syllabi, tests, and instruction have been recommended to increase military students' persistence (Mentzer, Lowrie Black & Spohn, 2015).
The mixed impact of classroom environment on perceived learning also suggests follow-up research.Including additional perceived learning items would be useful (Alavi, 1994), as well as recognizing that student learning styles can affect their performance across different classroom delivery environments (Fendler et al., 2016).Considering course-level factors, such as quantitative versus qualitative or introductory versus advanced, may also impact student perceptions of their online learning (Comer et al., 2015).
Undergraduates who preferred F2F classroom delivery had the lowest means on both PFoOC and intent to recommend online/hybrid course.One way to help increase PFoOC and intent to recommend for F2F undergraduates would be to perhaps bring specific features of an online/hybrid course into an F2F course.For example, making video lectures available to F2F students to help them review covered course materials as they prepare for exam.Another application could be to use a discussion board in a F2F class as an alternative counting towards class participation.Undergraduates may complain about a F2F classroom participation requirement, and a discussion board would benefit students in a F2F classroom who are either more introverted or need more time to reflect before they participate (Daymont et al., 2011).
How can a university or college encourage F2F-preference undergraduates to voluntarily "try" an online or hybrid class?Offering "ease of technology use" seminars at convenient times and locations for students is one way, for example, the application of social networking sites as a learning/teaching tool such as Facebook (Wang, Lit Woo, Lang Quek, Yang & Liu, 2012).The transition for an undergraduate used to taking only F2F courses may be easier if starting with a hybrid course, as opposed to an online course, since a hybrid has some F2F class meetings.If resources allow, put first-time online or hybrid undergraduates in smaller sections of a course and also try to put the "best" online/hybrid instructors in these "first timer" classes.The Student Online Learning Readiness (SOLR) scale (Yu & Richardson, 2015) can help to determine undergraduate readiness for online or hybrid class learning.Stronger F2F undergraduates' PFoOC should then increase their likelihood of recommending such classes to their peers.
Face-to-Face (n = 122) a Outcomes -PfoOC = Perceived Favorability of Online/Hybrid Course; Recommend = Intent to Recommend Online/Hybrid Courses; Learning = Perceived Learning; Graduate = Perceived Timely Graduation; all items measured on 7-point response scale b Within each outcome, cell means that do not share the same superscript c versus d are significantly different at the 0.05 level by the Sidak post hoc test ** p < .01;

Table 2 .
Descriptive Statistics for Background Variables -Fall 2015 and Spring 2016

Table 3 .
Univariate Tests on Outcomes and Cell Means by Preferred Classroom Environment for Fall 2015 & Spring 2016 Samples