No Significant Difference — Unless You Are a Jumper

Much of the e-education literature suggests that no significant difference exists in aggregate student learning outcomes between online and face-to-face instruction. In this study, an empirical model is developed to forecast the grade that individual students would have most likely earned in the alternate class setting. Students for whom the difference between the actual grade received in one class format (for example, online) and the forecasted grade in the other class setting (for example, face-to-face) is one full letter grade or higher are called “jumpers.” The findings reported in this study indicate that while about half of the students in the sample would have received essentially the same grade in either setting, as many as 42 percent are jumpers (meaning a positive or negative potential change of at least one full letter grade). This discovery has important implications for student choice and advisement in universities where students are free to choose between taking a particular course either online or face-to-face.


No Significant Difference-Unless You Are a Jumper
The rapid growth in online education over the past fifteen years (Allen & Seaman, 2014) has motivated substantial research concerning the efficacy of online relative to traditional, faceto-face learning.Since Russell (1999) coined the term "no significant difference," numerous empirical studies report that final course grades of students who take online courses are essentially the same as for students who take the same course in a face-to-face setting.An implication of the no significant difference hypothesis is that if learning outcomes are the same in both settings, then the choice of whether to take a class in a traditional format or online is, in terms of learning, unimportant.
Many of these studies, after controlling for individual factors, compare the average grade on a common evaluation instrument used in each class setting.An important limitation of these findings is potential differences in individual performance that may cancel out in the aggregate.Specifically, if half of all students perform worse in an online class than they would have if they had instead taken the same class in a traditional setting, and the other half perform better, overall statistics may show no significant difference.However, for those individual students, the difference is very significant.
The purpose of this study is to forecast the grade, based on individual characteristics and learning styles, that a student would have most likely received if they had taken a class in an alternative setting (i.e., online or face-to-face).Students for whom the difference between the actual grade received and the forecasted grade is one full letter grade or higher are called "jumpers."Identifying jumpers and their characteristics have important implications for student choice and advisement, especially in universities where students are free to choose between taking a particular course either online or face-to-face.Russell (1999), based on a bibliography of 355 studies conducted between 1928 and 1998, concludes that no significant difference exists in final course grades between traditional and technology-aided instruction.This expansive review shows that students, instructors, and course design, as opposed to course setting or format, are the major factors that determine success, or lack of success, in a class.Not surprisingly, Russell's inference of no significant difference has created a deep literature, both confirming and refuting the conclusion.

Confirming No Significant Difference
Several recent studies that support the no significant difference hypothesis (and also provide excellent reviews of the prior literature on this subject) include Iverson, Colky and Cyboran (2005), Larson and Sung (2009), Euzent, Martin, Moskal andMoskal (2011), Murcock, Williams, Bruce andYoung (2012), and Means, Toyama, Murphy, Bakia and Jones (2010).Iverson et al. (2005) examine the performance of graduate students in an online versus traditional introductory course in training and development.Although the authors find significant differences in how students evaluated the different class settings and the impact of the class on student intent to transfer learning, they report no significant difference in learning outcomes between delivery modes.Larson and Sung (2009) evaluate student success in an introductory management information systems course across three different course delivery systems: online, blended, and face-to-face.They compare results on the same final exam used in each course and on final course average.Their research shows no significant difference between any of the delivery modes for either measure.Euzent et al. (2011) consider student performance in two very large (N > 300) sections of an introductory economics course.One of the sections was taught in a traditional face-to-face manner and the other was conducted online using lecture capture videos.All other aspects of the two courses-in particular, course content, the instructor, assignments, and exams-were identical.The authors find no significant difference between the two courses in terms of either performance or student satisfaction.Murdock et al. (2012) compare the acquisition of basic counseling skills of students who completed an online introductory counseling course to those who took the same course taught by the same instructor in a traditional face-to-face setting.Their data indicate no significant difference in the counseling skills developed by students in each setting.This finding is particularly interesting because counseling is an activity normally considered to require human interaction and experiential activities, which are difficult to achieve online.
Finally, Means et al. (2010) present a US Department of Education (USDOE) metaanalysis of over one thousand empirical studies of online learning published between 1996 and July 2008.This report is an updated and more statistically rigorous extension of Russell (1999).Nonetheless, the overall conclusion of the report is essentially the same.The USDOE finds that when an online course is properly designed and conducted, learning outcomes are not significantly different from those produced in a traditional, face-to-face classroom setting.

Refuting No Significant Difference
Acceptance of no significant difference is not universal.Several studies report significant differences in student learning outcomes between online and face-to-face course formats.Bertus, Gropper and Hinkelmann (2006) examine student performance in face-to-face versus online classes for graduate finance majors.Controlling for student characteristics, including GPA, the authors report that online students perform significantly better than face-to-face students.Connolly, MacArthur, Stansfield, and McLellan (2007), in a three-year study comparing online to face-to-face delivery, investigate student performance in a graduate-level computing class.In an extensive study involving over 4600 students, the authors report that online students consistently outperformed face-to-face students.Dutton, Dutton, and Perry (2001) find that students who took an online computer science course purposely designed to be as similar as possible to a traditional lecture delivery course performed significantly better.However, they also report that online students who were struggling up to the university's drop date were more likely to drop the course than comparably performing students in the traditional course.
Whereas the aforementioned studies report that online learning is superior to face-to-face learning, other studies find the opposite effect.Controlling for a host of student characteristics including GPA, gender, age, grades on course prerequisites, math background, SAT scores, and outside distractions, Anstine and Skidmore (2005) conclude that learning outcomes for online students are significantly lower than for traditional students.Bennet, Padgham, McCarty and Carter (2007) find that students in a face-to-face setting earned higher grades in a microeconomics class, yet online students performed better in a macroeconomics class.The authors suggest that this difference may be due to the fact that microeconomics is a more quantitative (e.g., math equations and graphs) subject matter, whereas macroeconomics tends to be more qualitative, or conceptual, in nature.

The Importance of Choice
One particularly controversial aspect of the USDOE conclusion is an assumption that students have no choice over whether to take a particular course online or face-to-face.That is, the finding of no significant difference depends on students being directly placed in a given learning environment.In fact, much of the current growth in online course offerings occurs in large universities offering an online section of a course that is also offered in a traditional format (Allen & Seaman, 2014).In these settings, students are free to choose which delivery mode they prefer.Gratton-Lavoie and Stanley (2009) investigate the characteristics of students who took an online versus a face-to-face introductory economics course.They report significant differences in age, gender, marital status, number of dependents, prior coursework, GPA, and projected major.Their raw data show that online students performed much better than face-to-face students.However, after controlling for individual characteristics, they find no significant difference in learning outcomes between the two groups.Johnson and Palmer (2015), compare the learning outcomes of students who were free to choose between taking a linguistics course either face-to-face or online.This study finds that the online students performed significantly worse.Indeed, the course average for the online class was a full letter grade lower than for the face-to-face class.This significant difference, however, is most likely explained by the fact that the average GPA of students in the online class was between 0.255 and 0.424 points lower than that of students who took the face-to-face class for the different semesters that the courses were conducted.Helms (2014) reports a similar finding in an introductory psychology course.Those students who chose to take the online section of the course had significantly lower GPAs.They also had more outside distractions (family and job) and time constraints than the students who chose to take the class in a traditional setting.Not surprisingly, the online students demonstrated significantly lower course performance.
Both studies suggest that some students choose online courses because they believe an online version of a class will be easier than a face-to-face version.These students often have lower GPAs or they are unable to devote the time needed to successfully complete a college level course.Unfortunately for these students, most do not realize that online learning is very challenging and will most likely require more time and discipline than a comparable face-to-face class (Stanford-Bowers, 2008).Indeed, as suggested by Helms (2014), improved advisement systems for online education are needed.

Student Characteristics Impact Performance in Online Courses
When students are allowed to choose whether to take a particular course online or face-toface, each format will most likely be comprised of students with unique characteristics.Jaggars (2014) reports that most students choose to take difficult courses in a face-to-face setting and take what they believe will be easy courses online.Fendler, Ruff and Shrikhande (2011) note that females, non-majors, and students with lower GPAs are more likely to choose an online undergraduate core course in finance than the same course taught in a face-to-face setting.Diaz and Cartnal (1999) note that students with different learning styles tend to choose different settings.Specifically, the authors find that independent learners more often chose to take an online class while dependent learners more often selected the equivalent face-to-face class.A unique feature of this study is that the authors use the Grasha-Reichmann Student Learning Styles Scales (GRSLSS) to measure student learning characteristics.Diaz and Cartnal argue that the GRSLSS (Grasha, 1992) is especially useful for assessing learning preferences in an online course because: (1) it is one of the few learning styles measures designed specifically for college students and (2) it includes a measure for the importance of social interaction, which studies show to be an important aspect of learning for some students (for an excellent review of this literature, see Muilenburg & Berge, 2005).
Several studies discuss a link between individual student characteristics and outcomes in online courses.Yukselturk and Bulut (2007) observe that women perform significantly better than men in an online setting.The authors surmise that a possible explanation for this finding is that women tend to adapt to an online learning environment better than men, but they offer no specific outside evidence to support this assertion.Colorado and Eberle (2010) report that student age and GPA are important determinants of success in online courses.The authors suggest that older and more academically gifted students have the critical thinking and self-regulation skills needed to succeed in an online course.
A large literature discusses relationships between various learning style measures and student performance in online courses.These studies vary by the chosen student learning styles measure, but many indicate that learning styles play a significant role in online performance.Johnson (2007), which uses the Felder and Spurlin (2005) Index of Learning Styles, reports that active learners struggled when placed in online study groups but thrived in face-to-face groups, and reflective learners tended to underperform in online settings.The author also notes that visual learners tended to earn higher scores on quizzes and exams that were given online.Lu, Jia, Gong, and Clark (2007) use the Kolb Learning Style Inventory (Kolb, 1985).They report that in an online environment, convergers and assimilators achieved significantly higher levels of learning than divergers and accommodators.Employing the VARK learning styles model (Fleming, 2001), Eom, Wen, and Ashill (2006) find that learning outcomes are related to learning styles of students.In particular, the authors report that in their data set of 397 college students who completed at least one online course, students with visual and read/write learning styles reported that they learned more in an online course than students without these preferences.

Individual Risk Tolerance Levels
As most students are relatively unfamiliar with online classes, the decision about whether to take a course in an online format as opposed to a face-to-face format is arguably a relatively risky decision.The role that an individual's risk-tolerance level plays in how students make decisions has been explored across several academic fields.Examples of risk taking within a classroom include speaking up in class, openly challenging the teacher's and classmates' beliefs, innovative thinking, and choosing to accept more difficult problems and assignments.Clifford (1991) argues that risk taking in education should receive the same type of research attention that risk taking receives in other fields, such as economics and psychology.A key part of this line of research is the use of self-assessment questionnaires to provide baseline levels of students' risktolerance levels.Clifford finds that learning outcomes can be improved by having students engage in an optimal level of freely chosen educational risk.
In a study of undergraduate English majors at a Chinese university, Wang and Lin (2015) create a self-assessment questionnaire to determine student risk-tolerance levels.The authors find that student risk-tolerance levels are strongly correlated to performance in two of the three measured learning outcomes.Robinson and Bell (2012) investigate risk-tolerance levels and academic risk taking by pre-service teachers in the online portion of a blended class.The authors postulate that active participation in the online format may be viewed as risky by students who are traditionally more familiar with face-to-face interactions in a traditional setting.Using a 12-item self-assessment questionnaire to measure student risk-tolerance levels and a rubric to quantify actual risk taking by students in their contributions to online discussions, Robinson and Bell find a statistically significant relationship between self-reported risk and actual academic risk taking within the class.Specifically, those students who indicate that they are more likely to engage in risky activities in general do, in fact, take more risks in an education setting.
In economics, researchers in the area of human capital theory have proposed that risk taking is positively associated with individuals seeking more education.The theory would suggest that, ceteris paribus, individuals with higher risk tolerances are now more likely to forego essentially known income in exchange for potentially higher, but unknown, income in the future.Belzil and Leonardi (2013) recognize that the amount of investment in additional education is a complex decision, one that balances time, money, effort, talent, and the uncertainties around future labor markets.Using the 1995 Bank of Italy Survey of Income and Wealth that covers 8135 Italian households, the authors find that individuals with higher risk tolerances are more likely to obtain additional schooling.Jung (2015) strengthens these results by untangling the endogeneity problem associated with the possible relationship between education levels and risk attitudes; he shows that more education leads to less risk tolerance, with risk tolerance measured via a Likert scale answer to a single question regarding the individual's attitude to risk.
Risk taking is also a major topic in finance literature.Of particular interest to this study is behavioral finance research that explores the relationship between risk-tolerance levels and investors' potentially bad decisions.In a study of the investment portfolios of German investors, Dorn and Huberman (2005) show that an individual's response on a self-assessment questionnaire measuring risk tolerance is the main driver of both portfolio diversification and trading within the individual's investment account which, they argue, strongly supports the effectiveness of selfassessment questionnaires to measure individual risk tolerance.Further, the authors find that more risk-tolerant investors hold less diversified portfolios and trade more frequently, activities that have been shown to harm investment outcomes.The authors also find a strong correlation between risk taking and investor overconfidence.Barber and Odean (1999) argue that the explanation for this excessive trading by some investors, which is well documented to hurt investment performance, may lie in investor overconfidence: the simple but well documented idea that humans are generally overconfident in their abilities and knowledge and, thus, tend to make poor decisions.Logically, this concept of overconfidence may also apply to students choosing between the unknown, risky online course and the known, more certain face-to-face course.

Objective and Research Questions
If students with different characteristics or different learning styles are more likely to perform better or worse in an online course, then these students should be advised (prior to taking the course) accordingly.Sound advisement is particularly important if, as noted in Johnson and Palmer (2015), Helms (2014) and Fendler, Ruff and Shrikhande (2011), students often choose to take an online course for the wrong reasons (for example, they believe it will be easier or they have many outside distractions).
Therefore, the primary objectives of this study are twofold.First, to empirically investigate the impact that individual demographic, personality, and learning style differences exert on student performance in different class settings (online versus face-to-face).The second objective is to forecast the potential grade difference between class settings.
The specific research questions addressed in this study are: 1. Do individual student characteristics, such as learning style, GPA, gender, planned major, course load, collegiate experience, and risk preference, influence how a student performs in a given course setting? 2. Do individual student characteristics influence performance differently in diverse settings?3.If a student who chooses to take the course in an online setting had instead taken the course in the face-to-face setting, would that student's grade have been the same, significantly higher, or significantly lower? 4. If a student who chooses to take the course in a face-to-face setting had instead taken the course in the online setting, would that student's grade have been the same, significantly higher or significantly lower?

Methods
To investigate these research questions, a regression equation using outcomes (i.e., course grade) as the dependent variable, and personal characteristics (specifically, demographic data, ability, learning styles, and risk-tolerance score) as the independent variables, is estimated for a group of students who took a course in a face-to-face setting.A second, similar regression equation is estimated for another group of students who took the same course in an online setting.The coefficients of the classroom regression equation are then used to forecast the grade that each student who took the online version of the course would have earned (based on their individual characteristics) if they had instead taken the face-to-face class.Similarly, the online regression equation is used to forecast the grade that each student who took the classroom version of the course would have earned if they had instead taken the online class.Finally, the forecasted grade in the alternative setting is compared to the actual grade received in the chosen setting.

Setting
The data used for this study were drawn from 504 students: 219 students who took an undergraduate course in finance in a traditional classroom setting and 285 students who took the same course online.The face-to-face and online sections of the course were purposely designed to be as similar as possible in all ways except setting.Specifically, the course was rigorously structured.All sections of the course used the same textbook, shared a common syllabus, and followed the same weekly calendar.A course coordinator oversaw all aspects of the course and met with section instructors weekly to ensure as much uniformity as possible between all sections of the course.
The course evaluated was a core course in the business curriculum.All business students, regardless of major, must take this fifteen-week course.Eighty-two percent of the students in the study were juniors, fifteen percent were seniors, and the rest were sophomores.Students were allowed to choose the class setting they preferred and had substantial flexibility to switch sections through the end of the second week of the semester.
The data were collected over two semesters.The online class had an average enrollment of 120 students per semester.Several face-to-face sections of about 30 students per section were taught each semester at various day/time combinations.

Sample Instrument
The dependent variable used in the regressions is final course grade.The course average was determined as follows: 10 percent quiz average, 10 percent problem-set average, 20 percent first exam grade, 20 percent second exam grade, and 40 percent final exam grade.Quizzes, problem sets, and mid-term exams were similar but differed some between classes and formats.For example, each individual instructor made up their own quizzes, problem sets, and mid-term exams.Also, all face-to-face class exams were completed in the classroom during regular class hours, while all online students took their exams via the online learning management system.The final exam, however, was identical for all students in all sections of the course.This evaluation instrument was a comprehensive, carefully proctored, closed-note, closed-book exam that all students took in a physical classroom on the same day at the same time.Final averages were converted to letter grades as per strict guidelines provided by the course coordinator.
Because finance is heavily math oriented, assessment of all quizzes, problem sets, and exams used to determine the final course grade (in both the online and face-to-face sections) was primarily objective.That is, student answers were either right or wrong.Thus, grading was very similar between course settings and, because the course was highly standardized, from semester to semester.
The university provided final course letter grades for all students in the sample.The translation of numerical course average to letter grade was specified to each section instructor by the course coordinator.Letter grades were translated into numerical values as defined in Table 1 to use in the regression equations.These are the same point values that the university uses to calculate a student's GPA.

Data Collection
Prior approval from the university's IRB was received for this research project.The university provided GPA, gender, major, total hours of college credit completed, semester course load, and percent of coursework completed at the school for each student in the study.This data was linked with individual survey data (described below), and all student specific identifiers were removed to strictly protect student anonymity.
Additionally, all students were asked to complete the Felder-Soloman Index of Learning Styles (ILS) questionnaire and a risk tolerance assessment quiz during the second week of class each semester.Participation in the research study, indicated by completing these instruments, was voluntary.An incentive of receiving a grade of 100 on a quiz was offered and an alternative for earning this same grade was offered to students who did not choose to participate.
The ILS survey, described in Felder and Spurlin (2005), is an online questionnaire that measures student learning preferences on four dimensions (Active vs. Reflective, Sensing vs. Intuitive, Visual vs. Verbal, and Sequential vs. Global).Each Felder-Soloman learning style is summarized in Table 2.

Learning Style Definition Active Learners
Improve retention and understanding of information by discussing or explaining it to others.

Reflective Learners
Prefer to think about the material first then, after reflection, formulate ideas and opinions.

Sensing Learners
Like learning facts and solving problems using well-established methods; strongly prefers seeing connections to the real world.

Intuitive Learners
Like discovering possibilities and relationships; like innovation and abstract information.Dislike memorization and routine calculations.

Visual Learners
Remember what they see; for example, pictures, diagrams, flow charts, demonstrations.

Verbal Learners
Get most out of written and spoken explanations.Sequential Learners Gain understanding in linear, logical steps.

Global Learners
Learn in large jumps, randomly absorbing material until they suddenly "get it."Table 2

. Felder-Soloman Inventory of Learning Styles
The survey has 44 total questions, with 11 questions related to each of the four learningstyle dimensions.Each short question has only two answer choices.Two example questions from the survey are listed below: Each student's learning preference is classified along a learning style spectrum where learning style (LS) scores can range from -11 to +11 in 2-point intervals.For example, if a student answers all 11 of the Active/Reflective questions in a way that indicates "active" for every question, and thus "reflective" for 0 questions, the student receives an Active/Reflective value of -11 + 0 = -11.If the student's answers indicate "active" for 4 questions and "reflective" for 7 questions, the student receives an Active/Reflective value of -4 + 7 = +3.
A negative score indicates a preference towards the first learning style; a positive value indicates preference towards the second learning style.For example, a score of -7 on the Active vs. Reflective LS, means a student has a fairly strong preference for active learning.A score of +3 on the same LS scale indicates a relatively weak preference for reflective learning.
Validity and reliability of the ILS has been confirmed by Livesay, Dee, Nauman and Hites (2002) and Zywno (2003).These studies report that test-retest correlation coefficients for all four scales of the ILS survey vary between 0.7 and 0.9 for an interval of four weeks after the test is administered and between 0.5 and 0.8 for intervals of up to eight months after administration.These coefficients are all significant at the 0.05 level.Zywno (2003) reports that Cronbach alpha coefficients for the ILS, which measure reliability, exceed the criterion value of 0.5 for all measures.
The risk tolerance assessment quiz that all participants completed is an online questionnaire available through Rutgers New Jersey Agricultural Experiment Station (https://njaes.rutgers.edu/money/riskquiz/).This survey has been used by over 200,000 educators, researchers, financial advisors and their clients (Grable & Lytton, 1999).Risk assessment scores can range from 13 to 47. Higher scores indicate increased risk tolerance.For a discussion concerning validity and reliability of the index, see Gilliam, Chatterjee, and Grable (2010).

Data Analysis
A comparison of mean values and standard deviations for all variables in the sample is presented in Table 3.The final column in the table indicates whether differences in mean values between the online group and the face-to-face group is significant.T-statistic tests were done for the continuous variables and Chi-square tests were conducted for binary and learning style variables (Winer, Brown & Michels, 1971).
Final course score is the average of converted letter grades to GPA point totals.All courses at the university are 3 credit hours.Business majors are classified as either quantitative (finance, accounting, actuarial science, CIS) or qualitative (marketing, management, general business, other).Percent coursework at the university reflects transfer credit and AP credit relative to coursework completed directly at the university.As an urban state university, approximately 35 percent of the students in the sample completed their initial year at a junior college or smaller school in the state.
As shown in Table 3, average values for the two groups are similar in some areas and quite different in others.The online group has a higher average course grade than the face-to-face group (2.92 versus 2.40, respectively).This result seems to suggest that online students have higher learning outcomes, although that is not necessarily the case.The course grade is comprised of three components: midterm exams (40 percent), quizzes and problem sets (20 percent), and the comprehensive final exam (40 percent).
As noted above, the final is a common, in-class, carefully proctored exam.The average grade on this common instrument is essentially the same in both groups (implying no significant difference on a common evaluation instrument).The average grade on quizzes and problem sets is also essentially the same in both settings, since these are used mainly as learning assignments as opposed to evaluation instruments.The average grades on midterm exams, however, are significantly higher for the online group.For this group, strictly timed exams were conducted online, but students were allowed to complete the exams anytime during a 3-day period.The average grade on midterm exams for the online classes is nearly a full letter grade higher than the average grade on the final exam.Such a discrepancy between midterm and final exam grades does not exist for the face-to-face group.The higher grades on online exams may reflect the challenges with monitoring online cheating (Harmon & Lambrinos, 2008) or it may be due to the fact that online students can use books, notes, and other materials to take online exams while face-to-face students are not allowed to use such materials.Additional research is needed to determine the precise reason.online class; on the other hand, it does not reflect the variety of online instructional approaches that a student may face across online courses taught by various instructors.In addition, different instructors in the face-to-face sections may introduce an instructor effect in the non-online course group.Such potential differences in face-to-face versus online classes is a limitation that should be further explored in future studies.
Though care was taken to align the online class with the face-to-face class, differences still inevitably exist (in particular, the online class size was 120 students and the face-to-face average class size was 30).Another potential limitation of the empirical results is that the studied course is a highly quantitative course with a focus on solving math-oriented problems.Thus, the results of this study may not apply well to a more qualitative course.Put differently, the analysis presented in this paper opens the avenue for similar studies comparing face-to-face with online settings for purely qualitative courses.
A third potential limitation of this study is that the sample set is based on students at an urban university who do not generally live on campus and very likely work full or part time.Thus, the results of this study may not apply well to, for example, a school where students generally live on, or near, campus and are full time students.The students at these two different types of schools may have very differing reasons for selecting an online version of a course over a face-to-face version, or vice versa.
Finally, it is possible that some students preregistered for sections of the course that, once the semester started, they realized was not their preferred setting.Even though the drop/add policy at the university is lenient, it is possible that at least some students were prevented from switching.Thus, due to possible constraints on the registration process, choice is not truly 100 percent.While this is a potential limitation of the analysis, it is likely difficult to resolve given that most business schools have similar if not more stringent constraints on student movement across sections and between different class settings, especially after the term begins.On the positive side, this limitation strongly supports development of policies that implements advising in a pre-emptive manner to guide students into the most appropriate setting given their overall profiles rather than merely letting students choose the setting.

OLS Regressions
Ordinary least squares regression coefficients for all variables in the model are presented in Table 4. Final course score is the dependent variable.Based on the R-Square values of .455for the online regression and .459for the face-to-face group regression, the regressions are quite robust in that they explain nearly half of the variation in grades between students in each setting.As expected, the regression coefficient for college GPA is highly significant (at the 1% level) and positive in both the online and face-to-face regressions; that is, for both groups of students, the students with higher overall GPAs tended to earn higher grades in the class.As also expected, the quantitative majors performed significantly better (at the 1% level) than non-quantitative majors in both groups.The variables capturing the students' percent of their coursework completed at the university and their course load in hours during the semester are both positive and significant at the 1% and 5% levels respectively for the face-to-face group, but not significant for the online group.By contrast, the variable capturing the students' total hours of college credit is positive and significant at the 10% level for the online group regression, but not significant for the face-to-face group.

Online Group
Face-to-Face Group Interestingly, the coefficient for the risk assessment score is negative and significant at the 10% level in the regression for the online group, indicating that the greater an individual's risk tolerance level, the worse they tended to perform in the online version of the class; the same variable is not significant in the face-to-face regression.The online group finding corresponds with the literature discussed earlier concerning investors.Dorn and Huberman (2005) find that investors with high levels of risk tolerance earn lower rates of return due to holding less diversified portfolios and high transaction costs due to frequent trading.Barber and Odean (1999) argue that investors with high risk tolerance levels are overconfident and this leads to poor decision making.The significant negative coefficient for risk assessment score for the online group in the current study implies that students with high levels of risk tolerance may be overly optimistic in assessing their ability to succeed in an online class.Basically, they are more apt to make poor decisions concerning the setting for which they are best suited.
For students in the online group, the coefficient for the variable measuring reflective learning style is negative and significant at the 10% level, indicating that reflective learners tended to perform worse in the online class than non-reflective learners.Likewise, for students in the online group, the coefficient for the variable measuring visual learning style is positive and significant at the 10% level, indicating that visual learners tended to perform better in the online class than non-visual learners.Both of these findings correspond with similar results reported in Johnson (2007).Neither of these variables-reflective learning style or visual learning style-are significant in the face-to-face regression.
Finally, females tended to perform worse (significant at the 10% level) in the online group than males, but no significant gender performance differences are observed in the face-face group.The gender relationship for online students in the current study is opposite of the association reported in Yukselturk and Bulut (2007).

Predicting Jumpers
Table 4 reveals several significant differences between the regressions for the two different settings.First, the intercepts are highly dissimilar (-1.4236 for the online regression and -4.3318 for the face-to-face regression), especially given the fact that the dependent variable values only range from 0 to 4.3.Second, some variables significant in one regression are not significant in the other.Specifically, risk assessment score, reflective learning style, visual learning style, total hours of college credit, and gender are significant in the online group regression but not in the face-toface group regression.On the other hand, percent of coursework completed at the university and term course load in hours are significant in the face-to-face group regression but not in the online group regression.Finally, the signs on several of the coefficients differ between equations.In particular, a higher risk assessment score, a more intuitive learning style, and a greater global learning style are associated with a lower course grade (and vice versa) in the online group regression but a higher course grade (and vice versa) in the face-to-face group regression.Conversely, a more active learning style and a greater verbal learning style are related to a lower course grade (and vice versa) in the face-to-face group regression but a higher course grade (and vice versa) in the online group regression.These differences suggest that student characteristics may influence performance differently depending on the setting (i.e., online versus face-to-face).To further investigate this notion, the other setting equation was used to forecast grades for each student had they taken the class in the opposite format.That is, online student data were inserted into the face-to-face group regression to forecast the grade each online student would have most likely earned if they had instead taken the face-to-face class.Likewise, face-to-face student data were inserted into the online group regression to forecast the grade each face-to-face student would have most likely earned if they had instead taken the online class.
First, though, grade distributions were normalized to account for the difference in mean grades assigned in the two learning settings.As the mean final grade in the online group is 2.92 and the mean grade in the face-to-face group is 2.40, comparison of forecasted grades to actual grades would produce biased results without first adjusting for this fundamental difference in the final grade distributions.Thus, for the online students, the distribution of their forecasted face-toface grades across the various grade categories (A+ through F) was scaled to match that of their actual online grades.For instance, students with the highest 3.2 percent of the forecasted face-toface model scores were assigned the grade of A+ as their forecasted face-to-face grade to match the same percentage (3.2percent) of the online students that actually earned an A+ in the class.Similarly, the next highest 16.9 percent of the students in terms of forecasted face-to-face scores were assigned an A grade to match the percentage of A grades actually assigned in the actual online classes.As the distributions across the various grade categories were equalized, the mean of the actual online final grades (2.9) equaled the mean of the forecasted face-to-face final grades for these online students.A similar process was followed for the face-to-face students, with the mean final grade of their forecasted online grades matching the 2.4 final grade mean of their actual face-to-face class grades.
The forecasted grade was then compared in the other setting to actual grade received.A difference between forecasted and actual grade equivalent to one letter grade unit (i.e., from A-to B+ or from C to C+) is called a "jump" of one unit.If the forecasted and actual grades are the same, the jump is 0. However, if the actual grade is two units from the forecasted grade (for example, actual grade is B+ and the forecasted grade is B-), then jump is designated as 2. And a difference between an actual grade and a forecasted grade of three units (for example, an actual grade of C+ and a forecasted grade of B+) is designated as a jump of 3. Jumps do not indicate positive or negative, but merely the potential magnitude of the change.
Note that no suggestion is being made that the forecasted value is the actual grade a student would have received in the alternate class setting.Obviously, many factors determine a student's grade in a class; indeed, more than half of the variation in grades is unexplained by the regressions.However, the regressions do suggest that individual factors-such as gender, learning styles, and course load-seem to influence student performance differently in each setting and the jumper's variable is designed to capture the potential magnitude of this difference.
Table 5 presents the distribution of jumpers between each setting.Jumps of 0 mean no change in grade.Jumps of 1 could be caused by any number of factors not captured by the model as well as random error.A jump of 2 (for example, receiving a C instead of B-or an A-instead of a B if the other format had been chosen) may capture real differences or may still be due to model misspecification.Choosing to be conservative, the authors of this study consider only jumps of 3 or greater to be meaningful.Thus, the results presented in Table 5 suggest that for up to 68.5 percent (i.e., 24.2 percent + 28.3 percent + 16.0 percent) of students who chose to take the online class and for as many as 58.3 percent (i.e., 18.3 percent + 22.1 percent + 17.9 percent) of students who chose to take the face-to-face class, the outcome of their decision would essentially be unchanged.Their final course grade would have been more or less the same in either class setting.  5. Distribution of Jumpers Jumps of 3, 4 or greater represent a change in grade of a full letter amount or more.For instance, a student who received a final letter grade of B would have most likely received an A (and perhaps even higher) or a C (and perhaps even lower) in the other class format.Table 5 shows that for 31.5 percent (i.e., 21.5 percent + 5.9 percent + 4.1 percent) of students who chose to take the online class and for 41.7 percent (15.8 percent + 9.1 percent + 16.8 percent) of students who chose to take the face-to-face class, choice of class setting was highly significant.
Theoretically, then, for a student completing this course in the first semester of their junior year, a negative jump of exactly 3 letter grades in this one course would have reduced the student's overall GPA by approximately 0.03 points (for instance, moving a 3.0 GPA to a 2.97).In the extreme, though, suppose a student could select between online and face-to-face for all classes and this student systematically chose the wrong format across all classes.The student's GPA would then move from a 3.0 to a 2.0.

Jump Up or Jump Down?
The direction of these highly significant jumps is also very important.Specifically, if the majority of the large jumps are positive, then it would appear that most students chose to take the "wrong" class format.Alternately, if most of the large jumps are negative, then students made wise choices.
Table 6 shows the percent of each group that moved in the positive or negative direction.Of the 69 online students who would have jumped 3 or more grade units had they instead taken the face-to-face class, 50.7 percent would have performed worse and 49.3 percent would have performed better.Thus, for the online students, a roughly equally split exists between those who chose the "wrong" class setting and those that chose the "right" class setting.Similarly, of the 119 face-to-face students who would have jumped 3 or more grade units if they had instead taken the online class, approximately half would have performed worse and half would have performed better.

Implications and Future Research
The findings reported in this study have important implications for student choice and advisement.Whereas it may be true in the aggregate that no significant difference exists between the performance of students who take a class in an online setting and those who take a class in a traditional face-to-face setting, this study demonstrates that, for any individual student, choice may be very important.For the sample used in this study, 31.5 percent of the students who took the online course and 41.8 percent of the students who took the face-to-face course would have received a grade of at least 3 full grade units higher or lower if they had chosen to take the class in the other format.Such important differences cannot be ignored because they "wash out" in the aggregate.Exploring the notion that online learning may not be equivalent to face-to-face learning (and vice versa) for individual students is not equivalent to rejecting the format.Instead, this research points to the importance of proper advisement processes for students trying to decide whether to take a class online or face-to-face.
Assuming that the final grade is the most important consideration for students, a relatively large subset of students who chose the online class instead of the traditional class made a less optimal choice.Of those online students who would have performed differently in the face-to-face format, approximately half would have experienced improvement of at least a full letter grade.For example, a student who received a C in the online class would have earned a B or better in the traditional class.
Undoubtedly, many factors influence the format a student chooses.Online classes are convenient and they provide time-constrained students with significant coursework flexibility.Some students may consider these benefits worth the cost of receiving a lower grade.
However, efficient cost/benefit analysis requires that both the costs and the benefits be fully understood before the decision is made.Accordingly, schools should consider creating a model like the one described in this study that would allow students to forecast the possible grade difference in taking a class online versus face-to-face.A unique model would need to be created for every class, but as long as it is simple for students to determine the input variables, student decision making would be greatly improved.Better, more informed decisions about class format will most likely reduce the high drop rates currently characteristic of many online classes.In addition, students who choose to take a class where the forecasted grade is lower may decide to expend additional effort in that format to improve their grades.Creating and providing students with such a model, and analyzing its impact on student behavior, represents an interesting area for future research.
Another area for future research is to more fully investigate whether the jumpers have specific characteristics that set them apart from non-jumpers.Such analysis may be quite complex, however, since there are three different groups (students who jump up, students who do not jump, and students who jump down) in two different class settings (online versus face-to-face).More than likely, a larger data set would be needed to properly identify all unique factors.Nonetheless, such a study would be very interesting, especially a more in-depth analysis of the role that risk plays in driving the performance of jumpers versus non-jumpers in the two different settings.

Conclusion
This study of over 500 students demonstrates that whereas no significant difference may be true in the aggregate, for any individual student the choice of whether to take a class online or face-to-face may be very important.Using a multivariate regression model to forecast the grade that a student would have most likely earned (based on prior academic performance, unique personal characteristics, and preferred learning styles) in the other class setting, potential grade differences for each individual student between each class format are computed.Differences between actual and forecasted grade are classified as "jumps."A student is said to jump by 1 unit if the change in letter grade is one designation (for example, from A to A-or from C to C+), by 2 units if the change in letter grade is two designations (for example, from A to B+ or from B-to C), and so on.Students for whom the potential change in grade is one full letter or higher (i.e., a jump of 3 or more) are designated as jumpers.To these students, the difference in potential grade between class formats is most likely very meaningful.
The specific research questions asked, the process used to examine each question, and the answers suggested by this study are summarized in Table 7.

Research Question
How data analyzed; answer to question 1. Do individual student characteristics such as learning style, GPA, gender, planned major, course load, collegiate experience, and risk preference influence how a student performs in a given course setting?
Two multivariate regressions-one for the online students and one for the face-to-face students-were estimated with course grade (proxy measure of student performance) as the dependent variable and student learning styles, GPA, gender, planned major, course load, collegiate experience, and risk preference as the explanatory variables.The overall significance of the regressions and the significance and signs of the various explanatory variables suggest an affirmative answer to this question.2. Do individual student characteristics influence performance differently in different settings?
Each independent variable coefficient in both multivariate regressions mentioned above was compared.Whereas some independent variables were significant in one regression, they were insignificant in the other.
Additionally, the signs of some variable coefficients were positive in one regression and negative in the other.These differences, coupled with the relatively high R-square values for both regressions, indicate an affirmative answer to this question.3.If a student who chooses to take the course in an online setting had instead taken the course in the face-to-face setting, would that student's grade have been the same, significantly higher or significantly lower?
The face-to-face regression equation was used for each online student to forecast the grade that the student would have most likely earned if he/she had instead taken the face-to-face class.Actual grades received are then compared to forecasted grades to identify "jumpers"students whose grade difference would have been a full letter grade or more.For the data set analyzed, 31.5 percent of all online students were jumpers.4. If a student who chooses to take the course in face-toface setting had instead taken the course in the online setting, would that student's grade have been the same, significantly higher or significantly lower?
The online regression equation was used for each face-toface student to forecast the grade that the student would have most likely earned if he/she had instead taken the online class.Actual grades received are then compared to forecasted grades to identify "jumpers"-students whose grade difference would have been a full letter grade or more.For the data set analyzed, 41.7 percent of all face-toface students were jumpers.Table 7. Research Questions and Data Analysis Summary A deeper probe of jumper groups indicates that approximately half of the jumpers who took the online version of the course would have received a higher grade if they had instead taken the face-to-face version of the class; likewise, about half of the jumpers who took the face-to-face version of the class would have earned a full letter grade or higher if they had taken the online course.Thus, in terms of learning, these students appear to have chosen the wrong version of the class.
Indeed, for these individuals in particular, this research study suggests that students could benefit tremendously if they possessed the information provided by a forecasting model prior to choosing a particular class setting (i.e., online or face-to-face).Developing an advisement model that a student could use to predict the possible magnitude and direction of performance between different course formats represents fertile ground for future research.Because each online course is distinct and student characteristics impact online learning differently in different subjects (Xu and Jaggars, 2013), each predictive model will be unique.However, as shown in this study, much of the data required are simple to collect and the predictive model is easy to create.Providing students with this type of information will allow them to make more informed decisions which may improve online retention rates.Additionally, this information may motivate greater student effort in situations where the jump is negative but the student chooses the setting for a particular reason (e.g., choose online for convenience and flexibility).

Figure 1 .
Figure 1.Example Questions from the Felder-Soloman Index of Learning Styles questionnaire 1.I understand something better after I: a. try it out.b. think it through.2. When I think about what I did yesterday, I am most likely to get: a. a picture.b. words.

Table 1 .
Translation of Course Letter Grades to Numerical Values

Table 4 .
OLS Regressions with Final Course Score as the Dependent Variable

Table 6 .
Direction of Movement for Highly Significant Jumpers