The Effect of a Multifactor Orientation on Student Performance: Organizational Skills, Goal Setting, Orientation to Classroom, and Academic Support

This quantitative study examined the effects of an online synchronous orientation on the final grade of students enrolled in a first-term online class. The multifactor orientation focused on several factors which are thought to contribute to online student success, including organizational skills, goal setting, orientation to the online classroom, and academic support. Six course sections (three treatment and three control) were included in the study. In each of the three treatment sections, students had the option of participating in the orientation during the second week of the term; about half in each section did so. Controlling for course performance at the second week, students who attended the orientation earned significantly higher end-of-term grades, compared to either control students (who were not offered the option of the orientation) or to students who were offered the extra seminar but did not attend it.


Organizational Skills
Research has indicated that students enrolled in online classes may lack the cognitive organizational skills to navigate an electronic classroom or to work independently (Cherif, Movahedzahed, Adams, Martyn, & Dunning, 2015;Shaw, Burrus, & Ferguson, 2016). Many students need preparation for the self-discipline and self-direction that online classes require. Dupin-Bryant (2004) reported that these two factors were critical to the success and the retention of online students. Sorensen and Baylen (2009) demonstrated similar findings and indicated that today's colleges need to teach students the organizational skills required to remain focused on their academics. Though related to students on a traditional campus, Stelnicki, Nordstokke, and Saklofske (2015) asked students what the factors were that contributed to their success. The students indicated which characteristics were helpful and which were not. Using content analysis and a Q sort of the students' replies, 73.7% of the students stated that executive functioning was important to their success in college (p. 220), while 31.6% (Stelnicki et al., 2015, p. 221) noted that a lack of good executive functioning kept students from reaching their goals. Executive functioning focuses on planning, self-monitoring, prioritizing tasks, and adapting to challenges.
Almost a third of the students in the Stelnicki et al. (2015) study asserted that they did not have a well-developed set of skills to allow them to plan, focus, juggle, and remember information. These tasks are not done in isolation. In other words, planning is difficult to do if students cannot focus their attention or remember information.
In another study, Cherif et al. (2015) collected 610 responses from two-and four-year college instructors; most of the responses came from faculty who had taught for more than 11 years. The faculty overwhelmingly indicated that their students were not ready for college due to lack of organizational skills, such as time management and ability to set priorities, to successfully complete college-level work. The responses to this survey indicated that incoming college students did not know how to ask for help when needed, take notes, or organize their information. Selfregulated learning pertains to a student's ability to understand the learning environment and control and manipulate it enough to develop an effective learning environment. These students also achieve their objectives by selecting, adapting, and/or inventing the strategies that will help them achieve a specific learning goal (Shuy & TEAL Staff, 2010). Paris and Paris (2001) reiterated the importance of cognitive organizational skills and found that successful students are notetakers, time managers, askers of questions, and effective users of available resources. Kim and Ra (2015) interviewed 46 academically successful college juniors, who, with final grades at or near 4.0, were considered high achievers. The researchers indicated that the students credited self-control with time management, note-taking, and goal orientation as the three most important factors in their success. The successful students believed that their ability to self-regulate, set goals, and control their time was crucial to their success.
Overall, for students to meet the unique demands of the online environment, they must be encouraged to self-regulate, stay focused, and manage their time efficiently (Cohen, 2012;Dupin-Bryant, 2004;Kim & Ra, 2015;Sorensen & Baylen, 2009).

Goal Setting
When Kim and Ra (2015) asked students to identify factors they deemed to be important to success in college classes, goal setting was identified as one of three important factors. Conversely, it has been shown that students who do not have clear goals in place may drop out (Kibby, 2015). Although students may not enter universities with the knowledge of how to set specific goals or understand the role of goal setting in working to achieve learning objectives, teaching them self-evaluation skills can improve goal achievement as well as academic performance (Coon & Walker, 2013). Student confidence and optimism are also strongly tied to goal setting: Programs designed to increase student confidence and optimism tend to lead to strong goals, and programs designed to strengthen goal setting by focusing more on students' strengths rather than weaknesses can lead to increased levels of hope, optimism, and academic persistence (Kibby, 2015;Snyder et al., 2002).
Helping students learn to develop strong goals can help them focus on objectives and overall success (Flynn, 2016;Kibby, 2015). Breaking larger goals into smaller goals has been found to help motivate students, especially toward achieving those larger goals. In fact, college students who are committed to the idea of goal setting perform better academically than those who aren't committed to goal setting (Hao, Branch, & Jensen, 2016).

Academic Support
Although achieving success in online learning rests to a significant degree on the student, faculty and academic support play a large role as well (Flynn, 2016;Kim & Ra, 2015;Snyder et al., 2002). Faculty can also integrate some academic supports into the design of the course. For example, Taylor (2015) conducted a study at a not-for-profit online university with an openenrollment policy and found that course withdrawals were reduced by the creation of quickly accessed and easy to use interactive videos. These videos provided information about overcoming challenges that students could access at convenient times in their schedules.
However, many students-particularly those in their first term-may not understand their own academic strengths and weaknesses, and whether or how they should access available supports. Faculty can help students develop this understanding by serving as a mentor, helping them define specific steps toward achieving academic goals, sharing effective time-management strategies, and helping them develop an awareness of and willingness to utilize different avenues of academic support (Coon & Walker, 2013;Flynn, 2016;Kot, 2014;Taylor et al., 2015;Zie & Huang, 2014).
Providing personal help to students by routinely reaching out with encouragement may be the difference between success and failure (Flynn, 2016). Offering one-on-one time and becoming more involved with students, whether it is providing an extra seminar opportunity (as offered in this study) or one-on-one discussions, can also be extremely beneficial (Kim & Ra, 2015;Snyder et al., 2002).
The focus should be to move students from dependence to independence, but Flynn (2016), Kim and Ra (2015), and Snyder et al. (2002) all found that faculty and academic support are strongly correlated with online student success.

Orientation to the Classroom
At a minimum, an orientation to the online classroom should help acclimate students to the software platform and its technical demands. When adult learners are more adept and comfortable with an online learning system, they may access the course room and materials more often, persist to overcome challenges, and take control over their learning (Khatib, 2010;Kitsantas, Winsler, & Huie, 2008). In addition, an online orientation should include a student self-assessment of their level of preparedness for online college, self-discipline, organization, communication, technology access and literacy, and academic support (Fotia, Holtzman, Dagavarian, & Ciocco, 2010). Such self-assessment is crucial for improving the quality of online learning and to help students internalize and master the critical elements of preparedness for online success (Beckford, 2015;Fotia et al., 2010;Mensch, 2015;Orellana, Hudgins, & Simonson, 2009).
Crosling, Heagney, and Thomas (2009) found that an orientation class at the start of a student's college experience increased engagement by helping them grasp the most important elements of online preparedness. Taylor et al. (2015) developed an orientation for first-year students with a combination of multimedia resources. Hendricks and Bailey (2014) asserted that online students must be technologically prepared to overcome the variety of multimedia content in online classes. Categorically, an orientation to the classroom should include assessment for identifying technological challenges, organizational skills, goal setting, and academic support, to affect student learning. Barak, Dori, and Hussein-Farraj (2016) and Milligan and Littlejohn (2016) demonstrated that students new to online learning had higher rates of reaching goals and persisting to overcome challenges when faculty reinforced organizational skills, goal setting, and orientation to the classroom, and provided consistent academic support. Thus, in addition to introducing students to their online classrooms and preparing them for the various technical demands they will encounter, an online orientation would ideally address each of the other factors above. However, the research literature provides little information on structured orientations that integrate all these factors and their potential impacts on first-term college students' academic outcomes.

Integrating Multiple Factors Into Orientation
The present study was undertaken to determine if a multifactor approach to online orientation, focused on organizational skills, goal setting, an orientation to the online classroom, and academic support would improve the final grades for online students in a first-term course.

Setting and Intervention
This quantitative study was conducted at a large national online university. Approximately 75% of the student population is female. Most of the students live in the United States, but it is not unusual to have students living in other countries. Many of these students are military or from military families and can be stationed anywhere in the world.
New classes start every month and are 10 weeks long. One department in the university has a first-year class, Academic Strategies for the Business Professional, which is taken by most of its students. The course consists of weekly synchronous online sessions ("live seminars") and other asynchronous interactive exercises, focused on providing a foundation for secondary learning, including orienting students to the university and the learning management system (LMS) platform. All sections of the course used the same syllabus and master course shells. The course utilizes a standard rubric for each section of this class. Part of the student's grade is based on participation in the live seminars. Most students are adults in their 30s, many with young children and working full-time. For many, this is their first time in college, and most are new to online education.
The three instructors in this study are full-time faculty. Each instructor taught two sections of the course, and their course sections were randomly assigned to them, as is standard practice at this university. Instructors had no prior knowledge of the students who would be in their course. Prior to the start of the term, each of the three instructors randomly selected one of their two sections to receive an invitation to the additional orientation seminar. The extra seminar would be offered in the second week, since the first week tends to be overwhelming for new students as well as being add/drop week. During the first regular live seminar, each faculty member invited their "treatment class" students to attend the extra orientation seminar the following week (that is, during Week 2, after the add/drop window had closed). Faculty informed students that the optional extra seminar would focus on resources for organizational skills, goal setting, orientation to the online classroom, and academic support. Similar to the other seminars throughout the course, students attended the extra seminar via laptop or personal computer. They could see a PowerPoint presentation, which was created by one of the long-time faculty members who was part of the research team. The script for the PowerPoint focused on skills students need to successfully complete online college classes: organization, time management, setting deadlines, and gaining access to writing center resources. Students could hear the faculty member speaking and interact by using text in a designated chat area. They could not verbally communicate with anyone; only the instructor had the ability to verbally communicate with the class. No grades, extra credit, or other compensation was offered for participation in the extra seminar.

Participants
Students who remained on the roster at the end of the course were included in the study. (We could not evaluate the effect of the students who withdrew from the course. When those students are removed, the computer system removes their names and their data.) Across the six course sections, there were 200 students, with 106 in the three treatment sections and 94 in the three control sections. Among those in the treatment sections, 42% participated in the extra seminar (n = 45, Treatment: Attended) and the remainder did not (n = 61, Treatment: Did Not Attend).
Little demographic information was obtained, as there was no nonintrusive way of obtaining that data. Gender was the easiest demographic to obtain, as the student's name indicated gender or the student made some comment during the term to indicate his or her gender (see Table  1). Unless a student mentioned it, the instructors had no information regarding the age of any student. Many students were connected to the military in some way, but not all students mentioned a connection.

Data Collection and Analysis
Course tests and quizzes were autoscored by the LMS. To understand whether the control and treatment groups differed at Week 2, grades of the student participation in the required seminars were reviewed in Week 2. At the end of the term, the final course grades were collected, and the participating instructors sent the data to a fourth instructor for analysis. The fourth instructor also taught the same course but did not offer the extra seminar and did not have any contact with any of the students involved in the study.
To determine whether data could be collapsed across instructors, the three instructors' final grades were compared using a single-factor ANOVA. To determine whether the Treatment: Attended group differed from the other two groups (Control and Treatment: Did Not Attend) at Week 2, the three groups' Week 2 seminar grades were compared via a single-factor ANOVA. Finally, the final grades for students were compared via a single-factor ANCOVA to determine if the Treatment: Attended group had higher final grades, controlling for Week 2 participation. Table 1. An ANOVA revealed that there was no significant difference among the instructors. The result suggests that teachers applied the course rubrics evenly across the classes and that data could be pooled across instructors. The ANOVA comparing Week 2 grades across the three student groups indicated that there was no significant difference between the groups, indicating that the final results would not have been due to some form of self-selection into treatment.

A summary of the characteristics of the groups by instructor is included in
The ANCOVA comparing final grades, and including the grades for the required Week 2 seminar as a covariate, indicated that there was a significant difference in the final course grade between the students who attended the treatment, the controls, and those who were offered the treatment and did not attend, F(3,195) = -1.367, p = 5.188E-06. Bonferroni corrected t-tests indicated that the final grades of the students who were offered the seminar but did not attend were not significantly different than the final grades of the students in the control group. The final grades of the students who attended the extra seminar were significantly different than the grades of the other two groups. A Bonferroni t-test indicated that the final grades of the students in the control group were not significantly different from the final grades of all the students in the treatment classes regardless of their attendance at the extra seminar. The null hypothesis is rejected.

Discussion
The findings from this study suggest that a multifactor orientation had a positive impact on student grades. The researchers found that the instructors' methods were similar and that any differences in the scores of the students were not related to an individual instructor's grading methods. In addition, students who selected into the extra orientation seminar did not seem to differ from students who chose not to attend, or who did not have the option to attend, the orientation. The protocols for administering the extra seminar were standardized, delivered with the same PowerPoint content. In this study, students in the control group earned average final course grades in the low-D range, or just above the failing point of 60%, with an average of 62%. Students who were offered the treatment and did not attend earned final course grades mostly just below the failing point, with an average of 55%. Students who attended the extra seminar earned average final grades in the C or low-B range, with an average of 77%.
Researchers have suggested that first-time online college students could benefit from additional orientations at the beginning of their college career (Robichaud, 2016;Taylor et al., 2012). This study seems to lend some support to the idea. This study's results echo that of Whiteside and Dikkers (2016). They studied high school students getting ready to graduate and attend college. These courses were first-term courses; students attending these courses could have been recent high school graduates or not. Additionally, those students who attended the extra seminar may have found social connections, which may have moderated feelings of being distressed or overwhelmed, especially in the first week or two of class. Support helps buffer any psychological distress, which may have helped the students to persist. Researchers Song, Kim, and Luo (2016) have shown that having a strong connection with instructors can "indirectly affect learning outcomes through student-teacher relationship satisfaction" (p. 437). Several studies have demonstrated how positive course interaction with instructors is a vital influence on students' ability to complete an online course (Arbaugh, 2008;Eom, Wen, & Ashill, 2006;Marks, Sibley, & Arbaugh, 2005).

Limitations
There were a number of limitations to be considered when interpreting the results of this exploratory study. The sample size was small and heavily weighted for females, although the demographics are in line with research and gender trends for education (Lopez & Gonzalez, 2014). Statistical tests typically require a larger sample size to ensure a representative distribution to generalize results. Since the treatment group included only 45 students not identified by age and mostly female, these findings may not translate to male students or a wider range of ethnicities or ages. The measure used to gather the data came from a pre-and postcount of students enrolled who finished the course. In the future, researchers may find it more effective to revise the method for gathering data. A pre-and postassessment tool identifying readiness skills in students may help pinpoint specific areas in need to be included in an orientation. The voluntary nature of the participants also limited the results, since those students who did not participate may have been different from the volunteers. Although the students who declined to attend the extra orientation were similar to the control group in Week 2 grades, they did perform slightly worse than control students on their final grade, but this difference was not statistically significant.

Recommendations for Practice and Future Research
This study suggested that a combination of supportive factors in an extra orientation seminar can improve students' final grades. It did not attempt to isolate which factor or factors had the greatest impact on the final grades. Future researchers could measure each factor (organizational skills, goal setting, orientation to the classroom, and academic support) separately to determine whether one or the other had more impact on students. Pearson product-moment correlations could be used to analyze any relationship between factors. If the studied group were large enough, a regression analysis might be computed to determine the respective weights of each of these factors on the final grades.
This extra seminar was offered at the end of the second week or beginning of the third week; it is possible that different timing could alter its effect. The variable of timing may have caused students to miss the opportunity to participate in the offered seminar, as it must be given at a random time that may overlap with other mandatory seminars students have scheduled. In the future, researchers might identify an optimal time during the term, and perhaps an optimal day of the week, as well. In addition, the extra seminar could be offered at various times and, therefore, more students might participate since they wouldn't be constrained by the seminar being offered on only one day and at one time.
Future researchers could utilize a larger sample size, with an equal distribution of females and males. The issue with a sample that includes roughly equal numbers of males and females is complex, in that there might an imbalance in these numbers in practice. At this particular school, for example, males tend to be approximately 25% of the population.
Researchers may also examine a wide range of ethnicities, and ages. Future researchers might find it helpful to identify ESL students, for whom online learning is a poor fit. Researchers can also separate ESL students from non-ESL students to get a more equal distribution of participants.
Finally, determining the exact reason for online student success is complex, but as part of the multifactor orientation extra seminar, instructors can screen for technological readiness in order to pinpoint their needs, thus making an immediate impact in the online class. As a best practice, therefore, it seems prudent that universities design their programs to include orientation sessions.