Innovative Orientation Leads to Improved Success in Online Courses

A team of instructional designers, educators, and the School of Liberal Arts (SLA) academic program coordinator from a nonprofit online college, collaborated on producing short voice-over videos with interactive elements that address the most common technology frustrations of beginning students. These videos were inserted into the “Start Here” page of the pilot study courses. Courses were selected that had a higher than average withdrawal rate and that also tended to have a broad grade distribution for students who completed the course. The videos with voice-over covered basic navigation, such as posting to a discussion board, submitting an assignment to a drop box, reviewing a grading rubric in the grade book, and opening a graded copy of an assignment to view instructor feedback. The orientation element contained interactive tools that allowed students to check their learning of the skills covered in the videos. The orientation videos were available throughout the 8 weeks of the course. After viewing the videos, the students were asked to complete a short survey, which included both quantitative and qualitative feedback. The responses to the survey were extremely positive, ranging from 84.48 to 90.67 on a 100point scale. The team also examined course withdrawal rates and the grade distribution before and after the insertion of the videos. The data for almost all of the pilot study courses showed a drop in withdrawals after the use of the video orientation element. Several courses showed a significant drop in withdrawals. The examination of the preand postintervention grade distribution showed an improvement in grades for almost all of the pilot courses. This improvement in grade distribution was statistically significant in some


Introduction
Online education is growing quickly across all colleges and universities (Wladis, Wladis & Hachey, 2014).With this enrollment increase come new challenges pertaining to retention and persistence.Degree completion is progressively becoming a larger issue, particularly within online programs.According to the National Student Clearinghouse Research Center, completion rates for students who are older or part-time have increased.According to their 2008 data, 42.1% of enrolled adult students completed their degree (2014).For the purpose of this study, we are defining adult students as those who are 24 years of age and older.This age range is the reporting measure used for adults by the National Center for Educational Statistics (2014) and the National Student Clearinghouse Research Center (2014).It is also reflective of this study's population.
The purpose of this study is to address retention and persistence concerns in adult online education.The authors propose that if instructors embed an orientation within the class itself, students are more likely to complete the course.Park and Choi enumerated factors influencing adult student persistence in online classes, listing the learners' skills as an area that has an effect on the students' success but is outside of the control of the instructional design (2009).Given this assertion by Park and Choi, the authors felt that instructional design and orientations can be developed for the purpose of augmenting these skills.Such augmentation could lead to the students' overall success and completion of the course.
This study was designed to determine if providing just-in-time tutorials and course orientation would improve course completion rates.

Theoretical Significance of This Study
When discussing adult retention, it is beneficial to review what "adult learner" really means in the United States (since this study was conducted at an online college in the United States, it makes sense to work within this context).The word for adult education is andragogy; the term has been around for a long time.It was coined by Alexander Kapp in 1833 and meant "education in a man's age" (Reischmann, 2004).Since Kapp's time, many scholars have written about adult education.However, in the United States, Malcolm Knowles is attributed with creating both a theoretical perspective and an operational framework for approaching adult education.Knowles' theory has become foundational for those working in the adult learning field (Reischmann, 2004).Knowles wrote that it is erroneous to think of adults in the same way we think about children and adolescents when it comes to learning (1973).Adults are more mature and live complicated lives in which every moment is accounted for.Knowing that adults' lives are complex, Knowles created a set of assumptions that are meant to guide how educators tailor their classes to better serve adult students.These assumptions are the following: (1) adults are more self-directed; (2) adults bring rich experiences; (3) adults are more ready to learn; (4) adults want to apply what they learn directly to their world; (5) adults learn through problem solving; and (6) adults want to know why they are learning what they are learning (Knowles, 1980;Glancy & Isenberg, 2013;Chan, 2010).Out of this list of assumptions, the one most relevant to this study is adults' self-directedness.
Online education requires students to be more self-directed (Jones, A., 2013).Online courses are more flexible and malleable to students' individual situations (Park & Choi, 2009) and require students to take more initiative in their learning processes.While this malleability is a positive attribute of online courses, it can also be a hindrance.Despite the flexibility, the amount of work can be overwhelming, creating cognitive overload (Jones, A., 2013).The courses often are text heavy, requiring an intensive amount of reading and writing.An instructor who meets students face-to-face during the week to assist in keeping the learning on track is absent.These students must be completely reliant upon themselves to complete assignments and discussions in a timely fashion.Students also have to be prepared to make a large time commitment alongside their other everyday expectations.Candy (1991) writes that faculty must orient, guide, and foster this independent learning among adult learners, whether online or face-toface.One way to set adults up for success in an online environment is to ensure that they feel comfortable in that setting.
A report produced by Complete College America gives suggestions on ways to increase student course and degree completion.They write that one way to improve retention is by providing integrated support in "college-level gateway" courses (2014).Excelsior College, the setting for this study, is a degree-completion college with a very low residency requirement.Many students take as few as one or two courses with Excelsior before graduating.Thus, for this study, "gateway courses" can be defined as the first course taken by a student at the college.It is essential that students learn how to orient themselves to the course upon college entry.
Orientations have the ability to assist students in understanding how to use the learning management system (LMS), available services, and course expectations.A rural community college created a mandatory online orientation program for its students and found that students were better prepared for this educational experience as a consequence (Jones, K. R., 2013).Orientations, however, are not the panacea for increasing retention.Another factor contributing to student success, according to the Complete College America report, is the use of a system of intrusive just-in-time advising (2014).While the report is geared to young adults, this factor can be extrapolated to adult online students.Students who are returning to college with a plethora of credits need support that is tailored to their needs at that moment.Instead of taking the voluntary online orientation, students tend to jump into their classes without knowing much about what to expect or how to navigate in the LMS.Thus, looking at a pathway for adult students to succeed requires just-in-time support inside the LMS that is quick, easy to use, and interactive.Hence, not only can an orientation serve to get students prepared for how to participate in the course, the orientation can also provide just-in-time tutorials to assist students when they are stuck on how to work within the course.

Study Background
Excelsior College is a not-for-profit online institution serving adult students with an open admissions policy.The mission of the college focuses on adult learners and those who have been historically underrepresented in higher education.The School of Liberal Arts (SLA) allows the most transfer credits of any school within Excelsior College.Students come to the SLA when they are near degree completion; quite often they are also engaging in their first online learning experience.The focus of this study was to see if incorporating an orientation video with interactive elements into the opening page of courses could improve student success in these online learning experiences.Online learning can create additional challenges for students beyond those that they might encounter in face-to-face classrooms, especially for this student population, who have often been less than successful in traditional colleges.As an open-access college focused on adult returning students, often from underserved populations, retention and persistence rates are a primary concern for the institution.The five schools of the college have been challenged to submit proposals for projects that will improve retention and persistence and, thus, graduation rates for their students.Small internal grants were awarded to cover the cost of these pilot interventions.The authors, who are from the SLA, proposed to create simple orientation videos with narration that included interactive elements.The proposal was accepted, and the authors were awarded a small grant in October 2012.The concept was to create short video tutorials that walked students through the most basic procedures they needed to master to be successful in their course.Each video was less than four minutes in length.A menu was provided so that students could skip around or repeat any element.Topics covered include the following: • How to get started in the course  The orientation begins with a welcome to the course video as shown in Figure 1, which invites the student to feel comfortable with the course environment.The entire video orientation and navigation product were created using the two software packages: Articulate Storyline and Camtasia Studio.Viewing the videos and participating in the interactives takes no more than 10 minutes.These orientations were inserted into selected courses that met the following three criteria: (1) high enrollments, (2) often one of the first courses taken by new students, and (3) higher than average withdrawals.The videos and interactive elements work in concert with one another.The inclusion of brief interactive self-assessments inserted directly after the short video allows the students to check their understanding of the information presented.Figure 2 is an example of such an activity.The mixture of videos and interactive elements was created for the purpose of building student success in the course.

Method
Student success was evaluated using both a qualitative and a quantitative measure.The end of the orientation video section contained a short six-question survey designed to evaluate students' perceptions of the video orientation.Additionally, the withdrawal rates and grade distributions for each course were examined before and after the insertion of the video orientation.
The survey was designed to allow the students to give a "grade" for their response to each of the stem items from "0 (strongly disagree)" to "100 (strongly agree)."It was felt that a 100-point scale would be a familiar format for students-similar to a grade-and would have intrinsic meaning for them.The final two items of the survey allowed for an open-ended response.Students were not required to view the video orientation, nor were they required to take the survey.The survey was constructed and responses were compiled using the Qualtrics survey system.
The survey consisted of the following six items: 1. Yes, the video was very clear and easy to understand. 2. Yes, the videos were informative.
3. Yes, the interactive elements improved my ability to navigate inside the course.4. Yes, as a result of this orientation, I feel more knowledgeable about how to work inside my course.5. What were the most helpful parts of the orientation?6.What areas could be improved to better address your questions or concerns about navigating inside your course?The video orientation first appeared in the 2013 Summer I term in HUM300 and BIO212.In May 2014, it was adapted for three additional courses and added to HIS101, SOC101, and ENG101.ENG101 is an introductory composition course that runs for 15 weeks.The other courses all run for 8 weeks each term.As stated previously, these are courses that run many sections each term and are often taken early in a students' enrollment with Excelsior College.
The courses selected had a stable population of instructors and had not been revised or substantially changed in the year prior to the video, during which time the video orientation was included.The withdrawal rates for each course were examined for the one-year period before the insertion of the video orientation.These rates were then compared to the withdrawal rates during the time when the video orientation was in the course.The grade distributions in each course were also examined for the year prior to the video.This data was then compared to the grade distributions during the period when the video orientation was present in the course.This data was collected using the data management system of the college where information for each section of each course is accumulated.The college currently uses the QlikView Business Intelligence system.This software allows specific date-range selection along with the ability to track specific courses for withdrawals and grade distributions.
The response rate varied somewhat for each question and for the additional comments on Questions 5 and 6, which were optional.The response numbers for individual survey prompts 1 through 4 appear in Table 1.The number of responses for the first four prompts in the survey ranged from 802 to 817.For the first four Likert-scale prompts, the average N was 810 responses.

Survey Results
The results from the survey responses are presented in Table 1.The results for all of the survey responses from all of the courses were combined for this report.The final two survey items were questions and allowed for open-ended responses.The responses were coded, and we then looked for common elements in the parts of the orientation identified.The number of responses for each is given below.
Question 5, "What were the most helpful parts of the orientation?" received 100 responses.Twenty percent of these students identified the navigation instruction as the most helpful element.Additionally, 29% identified the interactive topic review exercises as most helpful.The rest of the responses were varied.Examples of comments received from Question 5 included the following: • The use of the cartoon characters was fun • I learned that the syllabus and all the course information is always at my fingertips whenever I get confused.This is my first online course • The exercises • How to locate the discussion and assignment links in the course • The way the video moved around and used visual teaching • The interactive activities Question 6, "What areas could be improved to better address your questions or concerns about navigating inside your course?" received 95 responses.A total of 55.8% of the students specifically stated that nothing could be improved, 17% identified difficulty with playing the videos or getting the interactive to work, and the rest had specific design suggestions.A sample of comments received for Question 6 included the following: • I wish it said "assignments" instead of "home page and module" After the insertion of the orientation video, four of the five courses involved in the study showed a reduction in the percentage of students who withdrew compared to the previous year.Two of these four courses showed a statistically significant reduction in these withdrawal percentages.One course (SOC101) showed an increase in withdrawals, but this number was not found to be statistically significant.Table 2 shows the results of the withdrawal rate comparison.The number of students in each course before and after also appears in parentheses following the percentage change in Table 2 The grade distributions for these courses were also examined before and after the introduction of the orientation video.Excelsior College awards only whole letter grades.The passing grades (A-C) were looked at as one data point, and the D and F grades were also separately examined for each course.There were statistically significant increases in the percentage of A-C grades awarded in three of the courses: HIS101, BIO212, and HUM300.SOC101 experienced a statistically significant decrease in the percentage of A-C grades.No other changes were significant.The following paragraphs provide further detail.
HIS101 showed an improvement in the percentage of A-C grades awarded after the introduction of the video orientation.This group showed a 15.4% increase, which is a statistically significant change (p = .0019).In HIS101 there was a slight decrease (2.1%) in students receiving a grade of D, which not statistically significant.There was an increase (3.1%) in the percentage of F grades awarded, which was not statistically significant.
BIO212 showed an improvement in the percentage of A-C grades awarded after the introduction of the video orientation.This group showed an 8.4% increase, which is statistically significant (p = .0002).There was an increase in the percentage of students receiving either a D grade (2.1%) or an F grade (0.9%), neither of which was statistically significant.
HUM300 showed an improvement in the A-C grades awarded after the introduction of the video orientation.This group showed an 8.0% increase, which is a statistically significant change (p = .0032).In HUM300 there was a slight increase (2.1%) in the percentage of students receiving a grade of D, which was not statistically significant.There was a decrease in the percentage of F grades awarded, which was not statistically significant.
ENG101 showed a slight improvement in the A-C grades awarded after the introduction of the video orientation.This group showed a 3.5% increase, which is not a statistically significant change.In ENG101 there was a slight decrease (0.5%) in the percentage of students receiving a grade of D, which was not statistically significant.There was also a slight decrease (0.3%) in the percentage of F grades awarded, which was not statistically significant.
SOC101 showed a slight decrease in the A-C grades awarded after the introduction of the video orientation.This group showed a 4.8% decrease, which was a statistically significant change (p = .0032).In this course, there was a slight increase (0.6%) in the percentage of students receiving a grade of D, which was not statistically significant.There was a slight increase (0.9%) in the percentage of F grades awarded, which was not statistically significant.

Discussion
The results seem to indicate that a video orientation with interactive elements can improve desired outcomes when inserted into online courses that are frequented by novice online learners, have high withdrawal rates, and have broad grade distributions.These types of video orientations are also favorably received by students and seen as informative and helpful by survey participants.This was a small pilot study, and more data from a wider variety of courses needs to be collected before any definitive conclusions can be drawn.The statistically significant improvement in grades and reduction in withdrawals seen in some of the courses is a hopeful indicator that embedding these simple, economical orientation videos into online courses can improve student success under certain conditions.
It should be noted that almost all of the statistically significant positive change was experienced in the mid-to upper-level courses (BIO212 and HUM300).One possible reason could be the much larger after-video N values for BIO212 and HUM300 versus the other courses.Or perhaps the student population in introductory courses is markedly different than that of these other high-enrollment classes.More research is needed to test such hypotheses.We will be continuing to collect data and expand this video orientation to additional courses in the next year.
The results from the SOC101 course are troubling.The researchers are attempting to track down any unique elements that impacted this course during the study period.
There are several limitations to this study.The number of courses selected was small, and the courses were not randomly selected, but targeted first-time online course takers.The results provided may show a correlation between the use of the orientation video and improvements in student success, but directly linking these two events is problematic.An additional limitation is the use of only one college.In many ways Excelsior College attracts a unique student population.Most have accumulated credits at other institutions and through military and civilian credit-bearing activities outside the classroom.This preparation and experience makes the Excelsior student body dissimilar from a traditional student population.

Conclusion
Much has been written about how to engage in "entering services" (Chickering & Reisser, 1993).One facet of this engagement is the orientation.To date, orientations are delivered through a variety of methods, from face-to-face to blended to online.Most online institutions have an orientation course that students can take voluntarily.However, many busy adult students don't engage in or complete it.They really need another form of what Chickering and Reisser (1993) call "initial assimilation."Students need to adapt to the online environment in order to be successful.The more familiar the course tools and navigation are, the easier it is for the students to reach completion, as they don't have to think about how to get around in the LMS; this correlation is evidenced in the improved completion rate in four of the five courses in this study.This relationship between familiarity with course tools and navigation and student completion is especially true for short and intensive courses done in 4 to 8 weeks.
This article starts a conversation as to what methods work best when orienting students to the online environment.In this study, the authors found that using just-in-time tutorials to assist with both initial assimilation and practical assistance worked in most but not all cases, which raises questions about what other factors come into play with regard to online adult student success.Along with creating welcoming orientations with just-in-time tutorials, what else can be created for the purpose of improving student success?Might there be changes in the design, navigation, or instructions that impact student completion?Or might there be changes within the orientation itself, such as changing how videos open so that these can play in any browser or on any computer, that have a positive effect?Of course, there is always the possibility that because these are adult students who live full lives, sometimes living life can get in the way of completion.
Nonetheless, the authors found that student orientations are important for familiarizing learners with the educational environment and lead to improved rates of course completion in most cases.The opinion of the authors is that online learning has the potential to open up new methods to encourage student educational engagement both for online and face-to-face institutions.The virtual world allows us to explore new and creative ways to ensure that students can be successful.

Figure 1 .
Figure 1.The welcome to the course video.

Figure 2 .
Figure 2. One of the interactive elements for self-assessment.

Table 1 .
Responses to the survey for the four Likert-scale prompts.

Table 2 .
. Withdrawal rates before and after the video orientation was introduced into the course.