Perceptions of the Persistent: Engagement and Learning Community in Underrepresented Populations

In an effort to characterize perceptions of learning community and engagement in relation to success for underrepresented populations of online learners at a public institution in southeastern United States, a survey was conducted in Spring 2016. The results of the survey were paired with institutional data to create a baseline engagement and learning community profile for the online student population, which comprised 22% of total enrollments. The subpopulations of interest were: Age, gender, race/ethnicity, disability, Pell grant eligibility, first-generation, and orphan. For all students, a very strong positive relationship was observed between student perceptions of engagement and learning community and student outcomes (grades). This strong and positive relationship was confirmed across the subpopulations, but there were a few noteworthy exceptions: Hispanic and Black students were more engaged than Whites but earned lower grades. Younger students and students with disabilities were less engaged than their counterparts, but earned equivalent grades. These patterns corresponded to withdrawal statistics, which revealed a higher percentage of young minority males withdrawing from online courses.

Challenged to find ways to better engage online learners and improve retention, a study was designed to characterize student perceptions of engagement and learning community in relation to success (grades).Demographic, success, and withdrawal data were paired with the survey data to generate a holistic view of the online population.The subpopulations of interest were characterized by gender, age, race/ethnicity, disability, first-generation in college, Pell Grant eligibility, and orphan.The purpose of the study was to establish a baseline from which future instructional design and retention efforts could be compared.

Engagement
Engagement is a broadly researched topic in education due to its impact on academic success and its malleability (Carini, Kuh, & Klein, 2006;Fredricks, Blumenfeld, & Paris, 2004).Csikszentmihalyi's "flow" theory (1990) describes the ultimate engagement as a state of "flow" in which students are so intensely involved in an activity that nothing else seems to matter.At the other end of the spectrum, withdrawal from the academic institution exemplifies the ultimate disengagement.Academic engagement has a long history of research support in terms of thinking deeply and staying focused, as Chickering and Gamson (1987) so succinctly stated, "Time plus effort equals learning."The social aspects of engagement became integrated with academic aspects through the work of Astin (1984) and Tinto (1997) among others.Drawing from the K-12 literature, engagement has been deemed a multidimensional construct including behaviors, emotions, and cognitions (Appleton, Christenson, & Furlong, 2008;Fredricks, et. al., 2004Fredricks, et. al., , 2016;;Wang & Holcombe, 2010).Behavioral engagement includes participation, effort, persistence, and attention.Emotional engagement includes a sense of belonging, positive or negative feelings about the instructor, peers, the course, or the institution.The need to belong is a fundamental human motivation."Key self-esteem processes, such as relatedness, are hypothesized to have energetic functions; they are considered catalysts for engagement or disaffection.Engagement is a key construct in motivational models because it is considered a primary pathway… to learning" (Furrer & Skinner, 2003, p. 149).Cognitive engagement includes self-regulated learning, deep learning strategies, and exerting oneself to grasp complex ideas.
One of the most recognized measures of engagement in higher education is the National Survey of Student Engagement (NSSE, Kuh, 2001).The NSSE instrument assesses five key educational practices that support engagement, including • Academic challenge • Active and collaborative learning • Student interactions with faculty • Enriching educational experiences • Supportive campus environments As summarized by Kuh (2008, p. 542), "Student engagement represents both the time and energy students invest in educationally purposeful activities and the effort institutions devote to using effective educational practices."Thus, engagement can be viewed from both institutional and course-level perspectives.
Considering course-level engagement, Joosten (2015) adopted a systems perspective to characterize the impact of several input variables on engagement, learning community, and other social and communication processes in the classroom.Students attributed the key input variables to be instructional design (42%), instructional support (22%), and assessment and evaluation (15%).Regarding learning community, students identified assessment as most significant (27%)."It is clear that assessment and evaluation... may be greatly overlooked as having significant impact on predicting course communication, specifically media richness, social presence, and learning community" (Joosten, 2015, p. 52).Both engagement and learning community contributed to students' perception of learning.Joosten's descriptive model is depicted in Figure 1 (Joosten, 2015, p. 69).Learning Community Tinto (1997) and Rovai (2002) emphasized the importance of community in reducing attrition through feelings of connectedness and shared learning, which motivated learners to persist."The broader process of academic and social integration can be understood as emerging from student involvement with faculty and peers in the communities of the classrooms" (Tinto, 1997, p. 617).Whereas Tinto's work focused on the face-to-face community college classroom, Rovai focused on the possibilities of establishing effective virtual learning communities.To this end, Rovai (2002) described seven factors that affected the development of a virtual learning community: Dialogue versus instructor-controlled conversation, social presence (defined as "...a measure of the feeling of community that a learner experiences in an online environment" (Tu & McIsaac, 2002)), social equality (everyone has a voice), small group activities, group facilitation by the instructor, learner-centered teaching, and optimal community size (20-30 students in a class with active discussions).Angelino, Williams, and Natvig (2007) and Liu, Gomez, and Yen (2009) recommended beginning courses with "icebreaker" activities to stimulate conversation, incorporating introduction discussion posts, and using group projects to develop a sense of community.They posited that once social connections were formed, learning could occur."Clearly the path to student engagement… is not about the type of activity/assignment but about multiple ways of creating meaningful communication between students and with their instructor -it's all about connections," said Dixson (2010).Johnson (2011) and Arbaugh (2002) found gender differences in online course communication patterns (females communicated more than males and formed more social networks), but performance was gender neutral.Tsai, Liang, Hou, & Tsai (2015) found females adapted better to online discussions than males, whereas males participated more actively than females in face-to-face classroom discussions.
Garrison, Anderson, and Archer developed the Community of Inquiry model in 2001."An interactive community of learners is generally considered the sine qua non of higher education," (Garrison & Cleveland-Innes, 2005, p. 135).The three domains of the Community of Inquiry are cognitive presence, social presence, and teaching presence.From the student perspective, these can be viewed as interaction with content, interaction with peers, and interaction with the instructor (Garrison, Anderson, & Archer, 2001;Moore, 1989).

Retention
Models of Retention.Tinto (1993Tinto ( , 1997Tinto ( , 2006) ) described the "complex web of events" that impact student retention and the uphill challenge institutions faced to improve this metric in light of diminishing resources.Tinto's initial work described only the traditional, four-year, residential situation, but his later work developed the complex web to include cultural, economic, social, and institutional forces segmented by institution type (2-year, 4-year, residential, nonresidential).First year involvement was deemed critical, yet it was unclear how best to operationalize engagement, although research supported the development of learning communities and emphasized the importance of faculty involvement in the process (Tinto, 2006).Full implementation of effective programs included faculty ownership of student retention and assignment of more experienced faculty to the critical freshmen courses.Bean and Metzner (1985) created a theoretical model to explain the forces impacting the nontraditional student's decision to drop out.The nontraditional student was defined as not 18-24 years of age, not residential, and not full time.These forces included low grades, psychological factors (e.g.goals, stress, satisfaction), background factors (e.g.past academic performance, demographics), and environmental factors (e.g.hours of employment, finances, family responsibilities, ability to transfer)."The chief difference between the attrition process of traditional and nontraditional students is that nontraditional students are more affected by the external environment than by the social integration variables affecting traditional student attrition" (p.485).Rovai (2003) synthesized the persistence models of Tinto (1993) and Bean and Metzner (1985) to incorporate the unique needs of online learning students, resulting in a composite persistence model.The unique needs of online students included (1) remote access to institutional policies, procedures, and course catalogs, (2) self-confidence to handle the learning management system and workload, (3) integration into the college and learning community, and (4) remote access to support services including bookstores, library, financial aid, and advising.Rovai (2003) concluded that no simple formula could ensure student persistence, but institutions should strengthen orientation programs and support services for online students, while course design must encourage the building of learning community.
In Braxton, Hirschy, and McClendon (2004), Tinto's persistence model was empirically tested and differentiated between traditional four-year residential and commuter institutions.For commuter institutions, only two of Tinto's thirteen propositions were validated: 1) Student entry characteristics affected the initial level of commitment of the institution, which 2) then affected the subsequent institutional commitment.Braxton, et al. (2004) searched, identified, and advocated nine exemplary retention programs, including a student-led Campus Retention Committee, a college process review, minority outreach programs, a proactive Decision Tree survey to ascertain students' intentions to stay/leave, freshmen support and tracking programs, and undergraduate research programs.
Online versus Face-to-Face.While educators have come to accept equivalency of learning outcomes between online and face-to-face courses, there is widespread belief that online students drop out at a higher rate than face-to-face students (Allen & Seaman, 2015).Explanations for the retention gap include: Online students carry more work and family responsibilities than face-to-face students, thus external factors prove more pressing; online enrollments may be more exploratory than in the traditional university; and online students are completing courses rather than programs, thus appear to be dropping out (Allen & Seaman, 2015;Boston, Ice, & Gibson, 2011;James, et al., 2016).(Note that dropouts must be differentiated from transfer students yet tracking online students across institutions has proven challenging.)Empirical research providing evidence for the greater attrition for online students includes the Xu and Jaggars (2011) study of 323,528 Washington State Community and Technical College students in which fully online students had an 8% lower completion rate than face-to-face students.Interestingly, hybrid student completion rates were equivalent to face-to-face.
There is further evidence that hybrid students are retained more than face-to-face and online students.After controlling for background factors, Shea and Bidjerano (2014) found that community college students who took some of their early courses online were more likely to attain their degree than students who took exclusively face-to-face classes (N>18,000).One conceptual possibility used to explain the success was "transactional adaption" of the institution to provide choice, flexibility, and convenience through online learning to nontraditional students (Shea & Bidjerano, 2014, p. 110).Similarly, in a large study across fourteen institutions in the Predictive Analytics Reporting (PAR) Framework (N=656,258), James, et al. (2016) provided empirical support for improved retention through blended and online coursework throughout a student's program.

Demographic Factors.
Research has yielded mixed results regarding the influence of demographic factors on retention.Park and Choi (2009) concurred with Aragon and Johnson (2008) and Willging and Johnson (2004) that demographics (age, gender, race, and educational level) were not primary influences.Park and Choi (2009) stated relevance and family and organizational support were the most important factors influencing adult learners to persist.Other researchers found demographic factors to have significant influence on the online student's decision to persist or dropout.James, et al. (2016) found older online students were retained at higher rates than younger online students.Packham et al. (2004) found that older working students were more likely to withdraw from online courses and sometimes lacked realistic expectations of the work involved.Levy (2007) found educational level to be a factor, but not age or gender.In a large (N=40,000), five-year study of students enrolled in 34 community and technical colleges in Washington state, Xu and Jaggars (2013) found White women were more likely to successfully complete online courses and that young Black males with lower incoming GPAs were more at risk.This finding was consistent with traditional retention statistics, but the performance gap was thought to widen by the additional challenges associated with online learning.

Purpose of the Study
Challenged to find ways to better engage learners in online courses, a study was designed to characterize student perceptions of engagement and learning community in relation to success (grades) within the online student population.The purpose of the study was to establish a baseline from which future instructional design and retention efforts could be compared.The research hypotheses include: H1: Online student success (grade) will be related to student perceptions of engagement and learning community.
H2: The relationship between online student success (grade) and student perceptions of engagement and learning community will vary across subpopulations (age, gender, race/ethnicity, disability, Pell grant eligibility, first-generation, orphan).

Participants
The participants came from a public four-year institution with a headcount of 15,076 students in Spring 2016.The institution was founded as a community college and now offers fouryear degrees.In 2016, the institution awarded 2102 associate degrees and 278 bachelor's degrees.The college has a robust online program, and in Spring 2016, approximately 3000 students were fully online, a 10% increase over the previous year.In Spring 2016, online enrollments accounted for 22% of all enrollments.
Enrollment and withdrawal demographics are compared in Table 1.The online student population has a higher percentage of older, White females who are part-time students.Withdrawal rates were higher for online students than face-to-face students in Spring 2016.Total online courses enrollments were 9716 at the close of the term following 868 withdrawals (8.9%).In contrast, face-to-face enrollments totaled 33,844 and withdrawals totaled 1,904 (5.6%).Minority males disproportionately withdrew from both face-to-face and online courses.Table 2 identifies the number of Early Alerts processed during Spring 2016 for face-toface and online students.The Early Alert system is an established intervention system to support at-risk students.A lower percentage of online withdrawals passed through the Early Alert system as compared to face-to-face withdrawals and, of those, only 22% persisted and passed their courses.The primary reasons for online withdrawals were personal (39%), academic difficulty (22%), employment (20%), health (11%), and instructor (3%).The primary reasons for face-toface withdrawals were slightly different: Personal (45%), academic difficulty (18%), employment (16%), instructor (8%), and moving out of local area (3%).

Instrumentation
From the pool of question sets provided by the DETA Research Center, two measures comprised of Likert response options were chosen: One measure operationalized engagement and another operationalized learning community (see Appendix A).An exploratory factor analysis (EFA) was applied to half of the collected survey results, then a confirmatory factor analysis (CFA) was applied to the other half.Based on EFA results, three engagement items did not factor well onto the measure and were therefore removed (questions #1, 16, and 17, see Appendix A for item wording).After running the CFA, learning community questions #1 and 8 were also removed and engagement question #9 was moved into the learning community set, because it aligned with the learning community component.Varimax rotation was applied to diversify the loadings on each factor as much as possible.The resulting two components-Engagement and Learning Community-clearly emerged, accounted for 67% of the variation, and were used for all analyses in this report.
Reliability measures for the two sets of Likert questions showed strong internal consistency as shown in Table 3.There were three qualitative questions included in this study, but only the third one was analyzed: "How does your interaction with the course materials or other individuals in the class influence your success?"

Measures
The research hypotheses include: H1: Online student success (grade) will be related to student perceptions of engagement and learning community, after accounting for known demographic confounders of student outcomes (see Figure 2).H2: The relationship between online student success (grade) and student perceptions of engagement and learning community will significantly vary between underrepresented populations and their more privileged counterparts (age, gender, race/ethnicity, disability, Pell grant eligibility, first-generation, orphan) (see Figure 3).Predictor variable definitions are detailed in Appendix B.

Procedures
After receiving Institutional Review Board approval to conduct the study, the survey instrument was converted to a Qualtrics® survey, online faculty were notified, then the instrument was manually inserted into upcoming modules within all online courses.The survey ran for two weeks.The survey was voluntary, took about 20 minutes to complete, and data were centrally collected on the Qualtrics® server.At the end of the term, the Institutional Research team merged demographic and success data with the survey data, de-identified students, and returned the data to the author.The de-identified data were uploaded to the DETA National Research Center to contribute to the national study.In addition, local analyses were conducted to explore the relationships between success and perceptions of engagement and learning community within the groups of interest.

Data Analysis
All quantitative data analyses were conducted in SPSS 24.0.An alpha level of 0.05 was used for all significance tests in this study.For the post hoc univariate tests following MANOVA, an alpha level of 0.015 was used.
Of the 933 surveys collected, incompletes and duplicates were removed to yield 643 results.Frequency distributions of age, grades, engagement, and learning community scores were generated, as were regression plots of grades, engagement, and learning community combinations.To address Hypothesis 1, hierarchical regressions were run to statistically control for known demographic confounding factors on student grades.This allowed the analysis to investigate the unique associations between student-perceived engagement and sense of learning community.To address Hypothesis 2, MANOVAs were generated to honor the complexity of factors impacting grades, engagement, and learning community.MANOVAs allowed for the simultaneous investigation of between-group differences for underrepresented populations.
Finally, inductive qualitative analysis was conducted on the open response question, "How does your interaction with the course materials or other individuals in the class influence your success?"Responses were grouped, then the groups were organized into themes according to the method of inductive content analysis (Elo & Kyngas, 2007).
Demographic data are summarized in Table 4 for the overall sample population, and for each subpopulation.Highlights include the greater age of orphan and first-generation subgroups, the greater full-time employment and part-time student status of the orphan subgroup, the language challenge for the first-generation + minority subgroup (36% report English as a second language), and the higher engagement of orphan and first-generation subgroups.Men and students with disabilities scored lower in engagement.Grades were generally consistent across all subgroups.

Characterization of Data: Frequency Distributions
One of the more striking aspects of the sample population was the gender imbalance (77% female).The average online student was 29 years of age in contrast to 24 years for the total student population (Figure 4).Based on a grade scale of 0 to 4, Figure 4 shows that final grades were nonnormally distributed, with skewness of -1.480 (SE = 0.097) and kurtosis of 1.817 (SE = 0.193).The average grade was 3.24+/-1.026.

Hypothesis Testing
Hypothesis 1. Hierarchical regression analysis was used to examine Hypothesis 1 for the total sample population.Hierarchical regression involves introducing predictors into the analysis on a theoretical basis (Petrocelli, 2003).Because demographic characteristics are known mediators of the relationships between grades, engagement, and learning community, these were entered as Tier 1 independent variables.The "static" demographic factors included age, gender, race/ethnicity, physical disability, Pell Grant eligibility, first-generation, and orphan.This allowed for the evaluation of the pure effect of the "dynamic" and continuous variables of engagement and learning community on grades over and above the demographic mediation (Petrocelli, 2003).
Engagement.Grades were plotted versus mean engagement scores in the left panel of Figure 5.For all students, a statistically significant difference in grades was observed according to student perception of engagement (F4, 537 = 4.560, p<.001; R 2 =0.022; η 2 =0.033).Learning community.Grades were plotted versus mean learning community scores in the right panel of Figure 5.For all students, a statistically significant difference in grades was observed according to student perception of learning community (F4, 516= 5.845, p<.001; R 2 =0.028; η 2 =0.044).
An interesting uptick in engagement and learning community scores occurred for failing students and accounted for the low R 2 values.The uptick is a hopeful sign that faculty were reaching out to struggling students.

Regression of engagement versus learning community scores.
Student perceptions of engagement and learning community were strongly correlated (R 2 =0.459; p<.001).The average engagement score was 63 +/-14 and the average learning community score was 26 +/-7.

Hierarchical regression analysis.
Table 5 provides the results of hierarchical regression analysis across three models.For all models, the dependent variable is grades, which were categorical in nature (A to F). Model 1 is a simple regression of engagement and learning community scores on grades to establish an association between our focal independent variables and student outcomes (grades).Model 2 controlled for demographic factors in Tier 1 and entered engagement and learning community scores in Tier 2, which allowed for testing of the effects of engagement and learning community scores on grades above and beyond the variability due to demographic factors.Because there was a theoretical basis for learning community to be incorporated into the engagement term as the "social" aspect of engagement (Fredricks, et al., 2004(Fredricks, et al., , 2016)), Model 3 placed the sum of engagement and learning community in Tier 2 (relabeled "Total Engagement").Model 3 omitted the concern of collinearity between engagement and learning community by creating one measure.Motivation for using Model 2 originated from the factor analysis which showed engagement and learning community to be independent ideas (Table 3).For all models, Hypothesis 1 was supported, meaning student perceptions of engagement and learning community were positively correlated with final grades.Comparing Model 1 to Models 2 and 3, it was apparent that demographic factors were significant in accounting for some of the variability in grades.The demographic factors that emerged as significant were race/ethnicity and first-generation, and these had a negative effect on grades.The overall contribution of demographic factors to grade variability was 2.1%.In both Models 2 and 3, student perceptions of engagement and learning community were significant in accounting for an additional 3.2% of the variability in grades for a total of 5.8% (p<.001).
MANOVA was used to explore differences across subpopulations with respect to engagement scores, learning community scores, and grades.MANOVA was run twice, first using Model 2 with the dependent variables of grades, engagement, and learning community; then again using Model 3 with dependent variables of grades and total engagement.Results were similar; Model 3 results are reported below.
In Part 1 of the MANOVA analysis, multivariate MANOVA was used to characterize differences between grades and total engagement scores across the underrepresented populations.MANOVA takes into account all demographic factors simultaneously, including the intercorrelations between them.Box's test of equality of covariance matrices failed as expected due to the known non-normality of grade distributions (Box's M = 174, F(93,3779)=1.552,p<.001).
In Part 2 of the MANOVA analysis, univariate tests were run for each of the independent variables showing significance in Part 1, with the tightened alpha value of 0.015.Post hoc Levene's test of equality of error variances passed for total engagement (F(41, 398)=1.034,p>.005) and, as expected, failed for grades (F(41, 398)=1.750,p<.005).To summarize the MANOVA results, a two-way MANOVA revealed a significant multivariate main effect for age, race/ethnicity, and disability.Given the significance of the overall test (adjusted R 2 =0.917 for grades and adjusted R 2 =0.954 for engagement), the univariate main effects were examined.Significant main effects for minority were obtained for grades and total engagement.Significant main effects for age and disability were observed with respect to total engagement.These results support the hypothesis that engagement and learning community influence grades irrespective of demographic factors; however, some demographic factors stand out: Hispanic and Black students were more engaged than Whites but earned lower grades.Younger students and students with disabilities were less engaged than their counterparts but earned equivalent grades.

Qualitative Analysis
Students were asked an open response question: "How does your interaction with the course materials or other individuals in the class influence your success?"This question aligns to Moore's (1989) segmentation according to interaction with content, interaction with peers, and interaction with instructor.

Content.
With respect to the nature of the course materials, students noted that interacting with multiple types (i.e., media, reading materials, and web-based learning tools) resulted in increased understanding of course concepts.Students also discussed conditions they believe contributed to their success, including course organization, clear expectations, consistency among course materials and alignment between course materials and learning outcomes.Older students placed a significantly greater emphasis on learning from course materials than did younger students as measured by frequency of course material codes (33% vs 24%).

Time spent interacting with content.
Students perceived that time spent interacting with course materials significantly impacted their success such that more time is associated with greater success: "The more interaction and time spent online regarding the class, the more successful I am."Indeed, students reported that time spent reviewing the material resulted in an increased understanding.One student asserted that "more time with the material will ensure [their] success."Unfortunately, some students reported that time restraints (e.g., work obligations) "limit [ed] [their] ability to fully take in the material and interact."One student stated that "I wish I could be more involved but with my work schedule it is hard to be more involved." Interaction with multiple types of materials.Students reported that interacting with multiple types of materials (i.e., media, reading materials, and web-based learning programs) increased their understanding of course concepts.Media, especially videos and PowerPoint presentations, were described as both enjoyable and helpful.One student stated, "I enjoyed the presentations.It put everything into an easier concept to grasp."Another student stated, "Watching all the video instruction helps with interaction and MasteringChemistry®." Beyond increased understanding, it appeared that PowerPoint presentations and videos contributed to a sense of "being in a classroom." Many students reported that reading was significantly correlated with their success in the course.Some indicated that textbooks were crucial to learning: "Reading teaches me 95% of the course material."Some students expressed dissatisfaction about the importance of reading in relation to their learning: "There is no teaching done by any professor yet in this program, all learning must be done by reading the text and the professors are merely 'graders' who grade assignments." In addition to media and reading materials, a few students reported interacting with webbased learning programs.One student mentioned the learning benefits of having access to an interactive course on how to program (i.e., Codeacademy, https://www.codecademy.com/):"Codeacademy [made] it easier for me to understand this course, allowing me to further progress through this course without any major problems."Another student found McGraw-Hill Connect®, an adaptive learning assignment and assessment platform, increased understanding of course concepts.

Course organization and clear expectations.
Students wrote that course organization, including organization of course materials, was important to success in their course.One student reported that they were "successful in [their] class so far because [the] course materials are organized and laid out for [them] so there is no confusion about what [they] have to do." "Assignments that stay in a flow or specific pattern" was also found to be helpful.Students reported that clear expectations significantly contributed to their success.One student asserted that clarity in instructions for assignments influenced their success while interaction with others did not: "The interaction with others does not [affect] my success.The material provided and instructions for assignments [are] clear and therefore [do] impact my success."

Consistency and alignment with learning outcomes.
Students reported that course materials were most beneficial when there was consistency among course materials and alignment with learning outcomes.One student noted, "The course materials are a great influence on this course.However, they would be more beneficial if they were relevant to the course and modules themselves."Another student stated that inconsistency between materials and assessments resulted in a lack of learning: "The materials have nothing to do with the homework or the exams.I don't feel like I'm learning anything."In one case, a student relayed anger regarding inconsistencies and contradiction between course materials: Course materials, what a joke.Watching three-hour videos that [have] nothing to do with my book is ridiculous.Also, the information in the videos contradicted my book, then when taking a quiz, which source was I supposed to use?Why even make us purchase a book?Interactions with peers.Themes related to interactions with peers revealed that peer interaction is most often associated with success.Students reported that discussions between peers increased their understanding of course concepts.Perspective sharing was reported to increase understanding, encourage open-mindedness, and improve critical thinking skills.A few students also noted that the sharing of perspectives was a particularly enjoyable aspect of the course.In addition, students reported that feedback from their peers was useful and that they felt they could rely on other students for assistance when needed.Also reported was that students found interaction with their peers to be motivating.Although the majority of students noted the benefits of peer interaction, some reported a preference for independent learning and believed that interaction with peers did not influence their success in the course.

Discussions increased understanding of course concepts.
Students reported that discussions with peers often increased their understanding of course concepts: "I learn and discuss with others, which is very conducive to my success."Students indicated that many conversations occurred on discussion boards.One student wrote that "reading other students' questions and answers in the discussion boards helps me understand the material better at times."Another student found the discussion boards helpful because they could read how other students worded their understanding of course materials: "The interaction with the students in the discussion boards influences my success because it sometimes gives me a better understanding of the reading material in another student's words."Some reported that discussions "helped [them] when the book [did] not" and that "students [brought] up subjects relating to [the] course to shed more light on the text."Although the majority of discussions appeared to take place on discussion boards, one student noted that they "seem to learn better and [are] even more successful when there is live interaction and opportunity for discussion live." Discussions provided multiple viewpoints.Students reported that they valued discussions with their peers because it provided them with multiple viewpoints.Some reported that the availability of multiple viewpoints provided them with additional insights into the material: "Upon reading others' responses to discussions, it sometimes [gave] me a different perspective or an insight into something that I was unaware of."Other students indicated that multiple viewpoints helped them to think critically about the topic at hand: "The discussion boards are great because I can see how other people interpret the assignment and it makes me think harder."One student indicated that discussions with their peers was the most important contributor to their success: The replies to the discussion posts are what most influenced my success.Not every student agrees on the prompt and doing the replies helps me to think critically about my own point of view and the points of view of others.
Many students wrote that peer discussions that allowed for the sharing of multiple viewpoints helped them to "become more open-minded" and "grow as a person." Peer feedback reported to be helpful.Many students indicated that they found feedback from their peers to be helpful: "Interacting with other individuals gives feedback and constructive criticism to be more successful in [mine] as well as their work."They noted that peer feedback provided them with new ideas and that "you really learn from each other."One student reported valuing peer feedback even though they do not enjoy interacting with other students: "I personally do not like having to keep in contact with other students, but the feedback from them is sometimes helpful." Peers viewed as reliable "helpers."Some students viewed their peers as available to them in case they needed help: "I feel that we can contact each other if needed."One student stated that their peers helped them when they were having difficulties with assignments: "They really provided helpful assistance for me while I was struggling with some of the [assignments]."Similarly, another student reported that other students help them "to understand the material we are working on."In general, students reported that "there are others who can help you and have your back" and "if we do not understand, we can always help each other." Interactions with peers described as motivating.Many students reported that they found interactions with their peers to be motivating.Some students indicated that their motivation was derived from not wanting "to be the student who does the bare minimum."Most others, however, found their interactions with peers to be encouraging.One student wrote that "getting a response from others is encouraging" while another reported that "seeing how the other students respond to the material drives [them] to better [themselves]." Preference for independent learning.Although many students found feedback from their peers to be beneficial, some reported that they prefer "to learn independently and do not require feedback from [their] peers."These students tended to believe that they are more successful if they are able to work alone.One individual stated, "I am an independent learner, I really need time with the materials to absorb and comprehend information." Interaction with peers did not influence success.Although many students found that discussions with peers were beneficial, some believed that peer interaction had no bearing on their success in the course unless it was required.One individual stated, "Interaction with the class does nothing to influence my success."Contrary to some findings, a few students reported that group discussions were not helpful and "contribute [d] nothing to the learning process."Some viewed the instructor as the primary influencer of their success, while others reported that it was their own ability to teach themselves the course materials and complete assignments.One student reported that they "do not speak to individuals in [their] online classes unless it is necessary."Indeed, one individual wrote, "I am the only one that influences my success." Interactions with instructor.Students felt connected to the instructor through assignment feedback, class announcements, emails, and discussions.The majority of students had positive interactions with their professors and appreciated the feedback and support.A minority of students felt disconnected because the instructor was not responsive to emails, did not provide any feedback on assignments, or was out of sync with the course schedule.
Instructor feedback increased success.Many reported that timely feedback from their instructor was critical to their success.Students indicated constructive feedback was the primary way in which their instructors aided their performance and enhanced their motivation to succeed.One student boldly stated, "The professor's feedback makes or breaks the class."Another student commented: I like when the professor leaves comments about my assignments that are not generic like, "Good work."It makes me feel like they actually read my work and truly appreciated it.Even if it is constructive criticism it feels like they're treating my work like I am a real person and not just another name in the gradebook.
Delays in response and a lack of helpful feedback were described as inhibiting motivation: Well my interaction with the teacher influences my success.If the teacher is responsive and gives good feedback, I strive to do better and continue.When the teacher takes a long time to respond, or her answer does not completely answer my question, or the feedback is only "Nice Work" but I got a B, and there is nothing telling me what I could have improved on, then I feel less inclined to work harder.If the teacher doesn't give a damn, why should I?
Beyond the quality and timeliness of feedback that their instructors offered, students reported that regular interaction is key to maintaining motivation to succeed.

Overall Model
Engagement and learning community perceptions were strongly correlated, thus supporting the importance of social connections within the engagement construct (Fredricks, 2004(Fredricks, , 2016;;Kuh, 2001Kuh, , 2008;;Tinto, 1997Tinto, , 2006)).Although factor analysis showed engagement and learning community to be separate ideas (Appendix A), the questions from the DETA engagement survey encompassed primarily emotional and cognitive engagement prompts and the one question that had social elements (#9) crossed over to learning community in the factor analysis.The significance of the three aspects of engagement-emotional, cognitive, and behavioral-has been strongly supported in K-12 environments (Appleton, et al., 2006(Appleton, et al., , 2008;;Fredricks, et al., 2004Fredricks, et al., , 2016)).This study summed student perception of engagement and learning community scores into a multidimensional independent variable called total engagement and, when regressed against grades (after controlling for demographic variability), provided strong statistical support for Hypothesis 1, which stated perceptions of engagement and learning community positively contributed to student success (grades).Hypothesis 2 explored whether the relationship between engagement, learning community, and grades varied across demographic subpopulations, specifically age, gender, race/ethnicity, physical disability, Pell Grant eligibility, first-generation, and orphan.The model held true for all subpopulations except these: Younger students and disabled students were less engaged but earned equivalent grades, and minority students were more engaged but earned lower grades.

Retention
In Spring 2016, 8.9% of online enrollments withdrew in comparison to 5.6% of face-toface enrollments.This is consistent with the 3% difference noted by Xu and Jaggars (2013), and superior to their earlier work (Xu & Jaggars, 2010, 2011) and the Instructional Technology Council's 2015 Trends in eLearning report, which states online withdrawal rates are typically eight percentage points higher than face-to-face withdrawal rates (Lokken, 2015).One could conclude from the withdrawal statistics that institutional operations seem to be effective, yet there is room for improvement.Withdrawals occur for myriad reasons, but fundamentally represent inefficiencies and loss of human potential.Hence an effort was made to establish a baseline and withdrawal metric by which future improvements could be assessed.For the withdrawals that did occur, the reasons cited were personal (39%), academic difficulty (22%), employment (20%), and health (10%).The academic difficulty category, which translated to 190 students withdrawing, became the focus for future intervention strategies as described in the Recommendations section.

Demographics
Age.The online student population was older than the overall student population (29 vs 24 years) and older students demonstrated strong statistically significant engagement and learning community scores compared to younger students, which positively affected grades (see Appendix C).On the flip side, older students are reported to carry greater work and family obligations (Park & Choi, 2009), which negatively affect school retention, but in this case did not affect course retention (Table 1).This is consistent with the motivational study of Stewart, Bachman, & Johnson (2010), who found employed, older learners were more motivated to take online courses and complete their degrees.Withdrawals were greater among younger students, further supporting the idea that older students were more persistent.Both younger and older students acknowledged the significance of peer interactions as contributing to their success; however, older students were much more likely to mention the importance of studying course materials thoroughly.
Race/Ethnicity.The withdrawal data for this one semester at this institution showed a tendency for the online student population to be older, more female, and more White.The same demographic shift occurred in the face-to-face population, but to a lesser extent.Why were young males, particularly young minority males, not persisting?MANOVA results showed younger students were less engaged, which aligned, but minority students were more engaged.Why would more engaged minority students withdraw?Recall minority students, although more engaged, earned lower grades.Perhaps this subpopulation accounted for those who withdrew due to academic difficulty.These results are consistent with the work of Xu and Jaggars (2013), who suggested young minority males may be more challenged to adapt to online learning and that online learning exacerbates performance gaps that are known to exist in face-to-face classrooms.English language learners were concentrated within the minority subpopulation, and average grades were lower among English language learners, which could prove another contributing factor.The disconnect between success (grades) and the greater engagement of the minority subpopulation is an important area requiring further investigation.For this baseline study, the greater total engagement scores registered for the minority subpopulation were encouraging and perhaps reflect cultural differences as well as effective institutional intervention and outreach programs.

Disability.
The sampled population had a small number (N=16) of students requiring accommodation, and of these, the primary disability identified was attentional focus.The disabled subpopulation was 95% successful in course completion, but its mean total engagement score was lower than that of the overall population (73 vs. 62).Future work should seek to identify students using the Office of Adaptive Services and characterize its role in student success.
Gender.Nationally, online student populations registered a 60/40 ratio of female to male (Lokken, 2015).This was confirmed by the institution's overall student population (61/39), but not online enrollments (68/32), and less so by the sampled population (77/23).Perhaps females were more motivated to complete surveys.Considering their higher engagement and learning community scores (Table 4 and Appendix A), females expressed motivation in this and other ways.Some researchers claim a higher social orientation for women translates well to a collaborative online learning environment, whereas others state females value the flexibility of online learning in balance with family and work responsibilities (Aragon & Johnson, 2008;Johnson, 2011).It is important to note that this study showed equal success (grades) for males and females, which is consistent with the literature (Arbaugh, 2000;Johnson, 2011).
Orphan.The sampled population had a small number (N=19) of students self-identifying as orphans.The orphan subpopulation was older than the overall online population (39 years versus 29 years) and was equally successful in terms of grades and completion.The majority of students in this subpopulation was employed full time (68% vs. 48%).
First-generation.The first-generation in college student was older (31 years versus 29 years) and more likely to be awarded a Pell grant.This subpopulation appeared equally engaged as the overall online student population (75 vs 73) and earned equivalent grades (3.1 vs. 3.2).Isolating first-generation minority students, language factors emerged (36% English as a second language versus 15%), yet engagement appeared stronger (77 versus 73).Average grades were lower between the first-generation minority subpopulation and the overall population (2.9 versus 3.2).
Pell grant eligibility.The lack of connection between income and successful course completion in this study was inconsistent with the literature, including the work of Terenzini, et. al. (2001);and Tinto (2007), who stated, "For too many low-income students access to higher education has become a revolving door, the promise of a bachelor's degree unfulfilled" (p.12).Sixty-one percent of the sampled population was Pell grant eligible, which rose to 85% of the firstgeneration students and 75% of the disabled students.One explanation for the discrepancy between this study and the literature lies in the definition of income as a self-reported "Pell grant eligible" versus FAFSA-based data.

Recommendations
Since student retention must be the goal of the entire institution, meaningful and relevant retention metrics should be established for individual departments and service units.One meaningful metric might be the ratio of successful credits over attempted credits by subpopulation.This metric is more meaningful for students in degree programs who stay to graduate from the institution.With the fluidity of online learning and state-driven efforts to unify course catalogs, registrar, and advising functions, credit tracking will get trickier at the local level.
Once a meaningful metric is established, pre/post analysis of instructional design experiments and retention initiatives are possible.The following recommendations are focused on course-level engagement and organized according to Joosten's (2015) systematic model into categories of (1) learner support and (2) instructional design.The multidimensionality of engagement in learning requires both social support and instructional design be addressed.

Learner Support
Learner support will prove particularly important for the younger online student who may not have yet developed the self-regulation skills necessary for online learning and who may not proactively communicate and seek out support.Faculty involvement is key to building learner selfconfidence, providing feedback to clarify what the learner does/does not know, and guiding the student to support tools and services.Support tools can be enhanced: Adding an online readiness survey tool to the college website will help to set expectations for the unique demands of online coursework, requiring learning management system (LMS) orientation for all online students (currently only first-year students are required) will alleviate initial start-up troubles, and building more just-in-time support tools within the LMS will improve the user experience (Rovai, 2003).
As far as support services, the efficacy of current academic support services for online students should be regularly evaluated in comparison to those offered to face-to-face students.Online tutoring, for example, is accomplished through partners such as Tutor.com®,Smarthinking®, and Grammarly®.Academic preparation was unaccounted for in this study but has been shown to play a significant role in success (Xu & Jaggars, 2013).Future work should incorporate prior GPA with Early Alert and tutoring usage into the analysis, especially in relation to those students within the pool of withdrawals due to academic difficulty.Aragon and Johnson (2008) surveyed 305 students who withdrew from online community college courses and found 28% of these students attributed poor course design or lack of communication for their decision.This concurs with earlier work by Swan (2001) of 1406 online students in the State University of New York system, in which three course design factors emerged as significant: Clear and simple design, interactions with instructors, and lively discussions.Considering the diversity of academic preparation and skillsets by which primarily nontraditional learners enter the online classroom, and the inherent self-regulation required for online learning, instructional designers face significant challenges.Instructional designers should adhere to research-based approaches such as Merrill's First Principles and Keller's ARCS theory of motivation (Keller, 2009;Merrill, 2002).Both Keller and Merrill recommend capturing attention through a relevant task or problem, demonstrating new concepts, applying to the relevant task/problem with feedback, then integrating into the real world.

Instructional Design
Assessment.Joosten (2015, Figure 1) highlighted the importance of assessment in driving classroom conversation, resulting in greater learning community and ultimately impacting students' perception of learning.Instructional design should "begin with the end in mind" and craft authentic assessments as the backbone of the course, about which conversation naturally flows.Adult learners require relevancy, so real-world problem-solving scenarios lend great value.
Formative assessments play an important role in developing mastery in preparation for summative assessments (Bloom, 1968).Regular, low-stakes quizzing has proven effective in clearing up misunderstandings early and enhancing learning (DePaulo & Wilkinson, 2014).Gamification elements such as choice, risk, progress bars, and leader boards could prove useful in formative assessments and appeal to at-risk young males.

Content.
Students attributed multiple types of course materials as being important to their success.Reading and media presentations were deemed valuable; students recognized that more time spent with these materials improved their learning.This student feedback was consistent with Universal Design for Learning principles which encourage flexible access to content for use by the wide variety of learners (https://library.educause.edu/~/media/files/library/2015/4/eli7119-pdf.pdf).Future efforts will include adding more direct media-rich instructional content (e.g., narrated Articulates®, OfficeMixes®) in combination with readings.Direct instructional content has the added benefit of increasing instructor presence in the course, which builds a sense of learning community (Garrison, et. al., 2010;Moore, 1989).
Organization of content and alignment with learning outcomes were two aspects deemed important to students while busywork was criticized.Alignment with learning outcomes is consistent with Quality Matters® standards and establishes the framework for effective assessment (https://www.qualitymatters.org/qa-resources/rubric-standards/higher-ed-rubric).An intuitive course structure is more important in online learning environments which requires primarily self-guided work.

Interactives
Peers and instructors.Rovai (2002) found a "positive significant relationship between a sense of community and cognitive learning" (p.328).Garrison & Cleveland-Innes (2005) cautioned that "simple interaction, absent of structure and leadership, is not enough," identifying the critical role of the instructor to guide discourse (p.145).This study provided empirical evidence for the importance of discussion boards in support of collaborative learning.Given the wide array of courses sampled, there were undoubtedly varying degrees of instructor facilitation and effective structure.More careful analysis of discussion prompt structure and instructor facilitation would prove an interesting study.When prompts and facilitation are thoughtfully approached, students are more likely to learn through the interaction.Additionally, the effectiveness of media-rich responses could be compared to traditional text-based discussion boards.
Peer review, peer presentations, and group projects are other ways to construct knowledge through learning community.Faculty play an important role in setting the expectation for and facilitating the formation of learning community.
Learning activities.Students valued technology-based learning activities such as those provided by CodeAcademy (https://www.codecademy.com/),Pearson's MasteringChemistry®, and McGraw Hill Connect®.Such content interactives can provide the guided practice essential to the construction of knowledge (Keller, 2009;Merrill, 2002).Institutions should evaluate commercial digital products through the lenses of 1) alignment with learning outcomes, 2) grounding in learning science, 3) depth of tiered feedback, and 4) ease of integration with the LMS.Interactives must advance students achievement of learning outcomes to be worth the investment of time and dollars.
Learning activities provide the student with essential practice of concepts and should be both predictable and varied.Predictability is achieved through consistent requirements, e.g.weekly readings, reading quiz, vocabulary flash cards, and problem sets.This predictability contributes to the intuitive nature of the course.On the flip side, too much predictability can be boring, so periodically varying the form of a learning activity can pique interest.For example, periodically require students to post audio responses to a discussion board rather than text.Finally, wrapping up a learning module with a reflective exercise is best practice to build metacognition and selfregulation skills.

Limitations
This study establishes the characteristics of Spring 2016 online enrollments at a public, 4year higher education institution and may not be representative of other institutions.Surveys were voluntary and had a return rate of less than 10%.Although inserted into every online course, they were most likely completed by more motivated (either negatively or positively) students.Judging from the preponderance of high grades, more positively motivated students completed surveys, skewing results.Sample population was skewed with a higher percentage of females than the institution's overall online population (77% versus 68%.)Several demographic factors were selfreported, such as income, work, orphan, and marital status.Ideally a baseline would include more than one semester of data.Grades were letter grades and non-normally distributed, which affected the analyses.

ENGAGE21
.652 The class was a waste of time (R).

ENGAGE2
.511 The learning activities required me to think critically.

ENGAGE20
The activities were not active (R).

ENGAGE19
.421 I was not engaged in the learning activities.

ENGAGE11
.513 I was willing to put in the effort needed to complete the learning activities.LRNCOMM3 .470 .508 I developed a personal relationship with my instructor.

ENGAGE9
.636 The learning experiences were active and collaborative.

LRNCOMM4
.686 I was able to communicate sufficiently with others.

LRNCOMM7
.691 I did not develop relationships with my classmates (R).

LRNCOMM5 .698
The learning activities encouraged contact between myself and my classmates.

LRNCOMM9
.706 There was little cooperation in completing assignments with my classmates (R).

LRNCOMM2
.771 I developed personal relationships with my classmates.

LRNCOMM6
.784 My classmates and I cooperated in completing assignments.Two sets of Likert-style questions were chosen from the DETA Data Kit, one set for engagement and another for learning community.An exploratory factor analysis (EFA) was applied to half of the collected survey results, then a confirmatory factor analysis (CFA) was applied to the other half.Engage questions #1, 16, and 17 were removed based on EFA results.After running the CFA, Learning Community questions #1 and 8 were also removed.Varimax rotation was applied to diversify the loadings on each factor as much as possible.The resulting two components -Engagement and Learning Community -clearly emerged, accounted for 67% of the variation, and were used for all analyses in this report.

Figure 1 .
Figure 1.Systematic view of classroom academic and social processes

Figure 2 .
Figure 2. The general research model for Hypothesis 1.

Figure 3 .
Figure 3.The general research model for Hypothesis 2.

Figure 4 .
Figure 4. Age and grade distributions for total sample population

Figure 5 .
Figure 5. Mean engagement and learning community scores vs. grades.

Table 1 . Demographic and Withdrawal Characteristics of Overall Face-to-face and Online Enrollments Spring 2016 Data Criteria Total Student Population Face-to-face Enrollments* Face-to-face Withdrawals Online Enrollments Online Withdrawals
-face enrollment demographics were estimated based on face-to-face demographics.

Table 2 .
Withdrawal and Early Alert Data, Spring 2016

Table 3 .
Reliability Measures for Engagement and Learning Community Likert Questions

Table 4 .
Demographic Data for Online Students

Table 5 .
Summary of Hierarchical Regression Analysis for Variables Predicting Final Grades (N=514)

Table 6 .
Demographic Factor Results of Multivariate MANOVA Analysis and Univariate Tests for Independent Variables Showing Significance.Demographic factors evaluated: Age, gender, race/ethnicity, disability, Pell grant eligibility, first-generation, orphan.