Supporting Learning Engagement with Online Students

University students increasingly desire that traditionally taught courses be made available online. While quality standards have been developed for the format and organization of online courses, professors often are left wondering what activities contribute to learning engagement for online students. To investigate this question, an online survey of all students taking an online course during a spring semester was conducted in one state university. A total of 417 students completed the survey, which included three standardized scale variables for learning engagement and two open-ended questions. Course activities that had statistical significance in relationship to students’ reported learning engagement include those that changed their understanding of a topic or concept, connected their learning to societal problems, linked their learning to prior experiences and knowledge, and were interpreted as fun. A regression model using these variables, along with control variables of student age, gender, and out-of-school work, resulted in an R2 of 0.484, suggesting that almost half of the variance in learning engagement can be explained via this model. Analysis of responses to the open-ended questions revealed that students found certain aspects of online discussions and interactive assignments engaging, especially those that prompted students with thought-provoking questions that relate to real-world situations and invited students to share diverse opinions and develop personal perspectives.


Supporting Learning Engagement with Online Students
Colleges and universities are recognizing students' interest in online courses and programs.Students seek online educational opportunities for various reasons, including greater access, flexibility, convenience, ubiquity of the technology and format, and cost (see, e.g., Larreamendy-Joerns & Leinhardt, 2006;Simmons, 2001;Willging & Johnson, 2009).Likewise, online technologies promise benefits for institutions, such as increasing reach and sustainability in a competitive higher education market (Angolia & Pagliari, 2016).In 2016, approximately 31.6% of students took at least one online education course, and the trend toward growth in online enrollments has steadily increased, despite a decline in overall higher education enrollments (Seaman, Allen, & Seaman, 2018).While many faculty remain skeptical of online learning (see Allen, Seaman, Poulin, & Straut, 2016, p. 26), technical reformations in education that "put the consumer in charge" are viewed by some policy makers as a means to make higher education better and more cost-effective (for states and students).
As most faculty know, however, it is a mistake to think that the creation of effective online courses is a matter of simply transferring classroom content, assignments, and exams to a Webbased setting.Certain standardized and peer-reviewed criteria (e.g., Quality Matters) have helped to guide the development, formatting, and organization of online courses, and other innovative technologies (e.g., learning management systems, online tools, interactive content, videos, etc.) offer both opportunities and obstacles for professors, who may utilize available and emerging mechanisms to enhance their own teaching techniques, even in the traditional classroom.Changing environments also shape student learning.Technological advances thus offer not promises but possibilities, and there is no guarantee, particularly if applied inappropriately, that these will translate into benefits for students.Even when appropriately designed, online learning requires more self-discipline and initiative on the part of students, alters the roles of faculty members (Bawane & Spector, 2009;Bejerano, 2008;Bonnel, 2008;Fein & Logan, 2003;Volery & Lord, 2000), and may create challenges for promoting academic priorities (Barbera, 2004;Bothel, 2002) as well as social integration, including faculty-student and student-student interactions (Bejerano, 2008; although Baglione &Nastanski, 2007, andBlake, 2000, argue the alternative).
Regardless of course format, faculty are committed to ensuring a higher education learning experience that promotes academic depth, critical thinking, and active learning engagement.For online courses, however, professors often are still left pondering the following question: What activities contribute to learning engagement for online students?The purpose of the research presented in this article was to explore this question.

Learning Engagement Research and Definition
Early studies of student engagement from Alexander Astin (1985) initially postulated "learning engagement" as composed of physical and mental energy, being situation-specific, and including quantitative and qualitative components.Nearly a decade later, Astin's research of college students identified student engagement as being significantly strengthened through academic involvement, involvement with faculty, and involvement with student peer groups (Astin, 1993;Feldman, 1994).These early studies support the social construct of learning engagement as incorporating three key areas: behaviors, emotions, and cognitive engagement (Hew, 2016;Wang, 2008;Woo & Reeves, 2007).Behavioral engagement involves students participating in activities, such as course assignments.Emotional engagement refers to students' affective attitudes toward professors, peers, or even the course in general.Cognitive engagement relates to students evaluating course content as both relevant and important.
As faculty know, such components of student engagement are important regardless of the delivery mechanisms of a course.Garrison, Anderson, and Archer (2001), however, developed the Community of Inquiry (CoI) model of learning engagement to explain what takes place in online courses in terms of interactions.This CoI model of learning engagement dictates that there are three presences for effective engagement: social presence, teaching presence, and cognitive presence (Garrison & Arbaugh, 2007;Lear, Ansorge, & Steckelberg, 2010).Social presence is the ability of learners to project their personal characteristics and present themselves as "real people."This connection is comprised of being part of and maintaining relations with other classmates (Rourke, Anderson, Garrison, & Archer, 1999).Teaching presence is defined as the design and facilitation of cognitive and social processes so students realize personally meaningful and educational learning outcomes (Anderson, Rourke, Garrison, & Archer, 2001).Cognitive presence is the extent to which students are able to construct meaning through sustained communication (Garrison, Anderson, & Archer, 2001).These three presences are not only interrelated, but also related to student engagement.For example, Peter Shea and Temi Bidjerano (2009) found that 70% of the variance in online students' cognitive presence scores accounted for perceived levels of teaching presence and social presence, and a study of students in 30 online classes at midwestern universities found a significant correlation between students' perceptions of sense of community and learner engagement (Lear, Ansorge, & Steckelberg, 2010).
Today, research on learning engagement consistently recognizes engagement as essential for student satisfaction and course completion (Hew, 2016;Kuh, 2003;Robinson & Hullinger, 2008) and is considered one of the most significant factors in a student's academic success.In online courses and programs, because of the virtual and often asynchronous nature of communication and other interactions, students' perceptions of engagement are even more critical.For example, one study found that reasons why students dropped out of an online course or program included "too many low level assignments," standards "too difficult/demanding," "lack of one-to-one interaction with the instructors and students," and "lack of interest in the material or the program didn't meet expectations" (Willging & Johnson, 2009, p. 124).Another study by Dumford and Miller (2016) of online and traditional students at 541 U.S. institutions suggests that online courses encourage quantitative reasoning at the expense of collaborative learning, which could undermine perceptions of and opportunities for engagement.
The National Survey of Student Engagement (NSSE) identified five academic components related to learning engagement: (1) level of academic challenge, (2) a supportive campus environment, (3) enriching educational experiences, (4) student-faculty interaction, and (5) active and collaborative learning (NSSE, 2005, p. 11).Following these findings, Marcia Dixson (2010)in "Creating Effective Student Engagement in Online Courses: What Do Students Find Engaging?"-determined that "there is no particular activity that will automatically help students to be more engaged in online classes" (p. 1).With this disappointing outcome, she conducted more research and was able to identify that communication between students and between students and the instructor via "multiple ways of interacting" is related to higher engagement in general and that "active learning assignments, particularly discussion forums and web pages, may serve the secondary purpose of helping to develop students' social presence" (Dixson, 2010, p. 7).
Based on her research in this area, Dixson developed a validated scale for measuring online student engagement, the Online Student Engagement (OSE) scale (Dixson, 2010).It measures online students' study skills as well as how they feel about their learning and connections with the instructor and their peers.Later expanded upon, the OSE scale correlates significantly and positively with application and interactional student learning behaviors (e.g., discussion posts, more formal written assignments, quizzes) but not observational student behaviors (Dixson, 2015).
Our work draws upon these findings to explore what activities contribute to learning engagement for online students.Understanding the construct of learning engagement as incorporating behaviors, emotions, and cognitions, we developed a survey for students enrolled in at least one fully online course during a spring semester at a state university that used an OSE scale for identifying participation behaviors and two NSSE scales for identifying emotions and cognitive engagement in order to answer the following question: What activities contribute to learning engagement for online students?Additionally, two open-ended questions were asked in order to provide more insights into student-identified activities and behaviors.

Sample Selection
During the 2017 spring semester at a small state university, 2,008 students were enrolled in at least one of the 186 fully online courses (30 graduate courses and 156 undergraduate courses) offered in the following colleges: education, health professions, liberal arts, and science and technology.After obtaining IRB approval, we contacted all 2,008 students via their university email addresses and invited them to complete a confidential online survey near the end of the term.The e-mail to students included a description of the study and a link to a 14-question SurveyMonkey survey.Some professors offered extra credit for completion; if a student was enrolled in a course by one of these professors, the survey requested the student's university ID number, and professors were provided a list of university ID numbers of students in their courses who completed the survey.No other data were provided to professors, and students could choose to leave this ID-number question blank.
Other than possible extra credit offered by their professors, students received no rewards or compensation.The survey response window was three weeks (from April 18 through May 8), and, in addition to the initial invitation, eligible students were sent three follow-up reminders (for a total of four separate e-mails).A total of 417 students (a 20.7% response rate) completed the survey.Demographics of the participating students reflected the university's general student body.The majority of student responders were female (80%), full-time (84%), and undergraduate students (87%).They lived off campus (82%) and were employed off campus (71%).In terms of overall online course experience, 21% were currently enrolled in their first online course; 41% had taken two to four online courses; 32% had taken five to 10 online courses; and 6% had taken more than 10 online courses.

Instrument
Based on a review of the literature and borrowing from other validated scales (Dixson, 2015;NSSE, 2015), we developed several drafts of an online engagement survey and pilot tested them with both undergraduate and graduate students.The university IRB approved the final instrument, which focused on four distinct engagement variables: a global learning engagement score; higher order learning engagement; reflective and integrative learning engagement; and participation engagement.We also added two open-ended questions at the beginning of the survey in order to obtain, even briefly, students' own thoughts about what they found engaging and disengaging in their online course before they were exposed to constructs found in the validated scales.

Open-ended questions.
We posed two open-ended questions at the beginning of the survey.The first asked students to identify, either in their current online course or any other, what assignments, activities, or other aspects encouraged them to be interested in the class and/or its topics.This was then followed by asking them to think about the same course and identify what, if anything, discouraged them.It was important to include these open-ended questions at the start of the survey so as not to bias students with ideas about engagement from the subsequent questions that used OSE and NSSE scale variables.Although it was not possible to obtain in-depth qualitative data from this format (as one would, e.g., in a face-to-face semistructured interview), these two questions sought uninfluenced opinions about what the students themselves found engaging and disengaging in an online course.

Global learning engagement.
Similar to the open-ended questions, the survey next invited students to think of one of their online courses and the aspects that encouraged them to be interested in the overall course topic and those that discouraged them.We used this prompt to stimulate students to thoughtfully reflect on their course and then assess their level of learning engagement using a scale from 1 to 10, with 1 being I really don't care and 10 being I am highly engaged and excited.For this variable, students did not have to justify their score; this simply revealed the affective or emotional engagement a student felt toward the course.
Higher order learning engagement.This variable provides valuable information regarding the academic challenge students felt their online course emphasized with regard to required behavioral activities.The variable is composed of four statements, which were rigorously tested both qualitatively and quantitatively by the 2015 NSSE survey.Similar to the NSSE's approach, for each statement we included a four-point response set of never, occasionally, a moderate amount, and a great deal.Students assessed these four statements in the following way: Thinking of your online course, how often have you done the following: 1. Applied facts, theories, or methods to problems or situations 2. Analyzed an idea or experience in depth 3. Evaluated a point of view, decision, or information source 4. Formed a new idea or understanding from various pieces of information Reflective and integrative engagement.This variable measures the degree to which students extend mental breadth and depth within their online course and reflects cognitive aspects of learning engagement.It also is taken from the 2015 NSSE survey and includes a four-point response set of never, occasionally, a moderate amount, and a great deal.Students assessed seven statements: Thinking of your online course, how often have you done the following: 1. Combined ideas from different courses when completing assignments 2. Connected my learning to societal problems or issues 3. Included diverse perspectives (political, religious, racial/ethnic, gender, etc.) in course discussions or assignments 4. Examined the strengths and weaknesses of my own views on a topic or issue 5. Tried to better understand someone else's views by imagining how an issue looks from his or her perspective 6. Learned something that changed the way I understand an issue or concept 7. Connected ideas from my course to my prior experiences and knowledge Participation engagement variable.This variable is Dixson's (2010Dixson's ( , 2015) ) descriptive engagement variable and includes six engagement behaviors, such as chats and discussions.Just as Dixson did, we used a five-point Likert scale of not at all, not really, moderately well, well, and very well.Students assessed the following six statements: 1.I have fun in online chats or discussions 2. I participate actively in discussions 3. I help fellow classmates 4. I engage in online conversations 5.I post regularly in discussion forums 6.I get to know other students Respondent demographics.In the last section, students disclosed their gender, year of birth, enrollment status (part-time or full-time), the number of online courses they had taken prior to and including the current term, whether they lived on campus, whether they were employed outside of the university, and whether they were currently enrolled only in online courses or an online-only program.

Data Analysis of Learning Engagement Survey Results
The quantitative analysis consisted of descriptive analysis of all items, followed by scale variable development, and then correlational and regression analysis.The higher order learning scale and reflective and integrative learning scale variable development followed the procedures used in the NSSE studies.Scale scores were determined using procedures from originating scale variables (NSSE, 2015).The higher order learning scale and reflective and integrative learning scale utilized four-point scales, with never = 0, occasionally = 1, a moderate amount = 2, and a great deal = 3.The options of don't know or does not apply were not provided.The given scores then were converted to match the scoring styles of the originating validated scales.For example, the four-point Likert scale used here was recoded to match the NSSE's 60-point range (never = 0, sometimes = 20, often = 40, and very often = 60).The participation engagement scale variable from Dixson's study used a five-point Likert scale, which we did too.Then, the mean scores for all scale variables were determined.
Pearson's correlation analysis to the global learning engagement variable was conducted on all scale variables, as well as individual items within these variables.Regression analysis models were conducted for variables with statistically significant moderate or higher correlations.The first regression analysis was conducted with three control variables (age, gender, and external employment) and three scale variables (higher order learning, reflective and integrative learning, and participation engagement).The findings were not useful, as the scale variables were not specific activities.So we conducted a second regression analysis with all scale items that had a 0.5 or higher correlation coefficient to the global learning engagement variable.

Data Analysis of Open-Ended Survey Questions
The two open-ended questions, regarding what encouraged or discouraged students to be interested in an online course and its topics, also were examined.To analyze students' responses to these questions, we developed a coding scheme using a constant comparative method (à la Glaser & Strauss, 1967;Strauss & Corbin, 1997, 1998) for the purpose of quantifying and examining (using mixed methods guided by Creswell & Plano Clark, 2018) those activities that students considered most and least engaging.First, the principal investigator conducted open coding of all responses.This involved identifying themes from the (dis)engagement-related phenomena students described in their responses and categorizing them, whether using in vivo terms or creating a label that incorporated similar responses or ideas.From this, the principal investigator drafted an initial code list.All three authors then independently coded the open-ended responses using this initial code list, achieving a 49.5% interrater reliability (IRR).We felt it was important to code responses independently to obtain perspectives that were as broad as possible before discussing them as a group.The middling initial IRR may have been due to differences in the authors' backgrounds regarding online courses: one develops and teaches online courses regularly; one has taught some online courses but primarily teaches on-site courses; and one is a graduate student who completed an online undergraduate degree.Such varied experience with online courses may have engendered different initial reactions to students' responses in the openended questions.Another reason may be that, since student responses were collected via a nonface-to-face format and thus were brief (it was up to students to determine how to answer these questions and to what extent), context was at times difficult to parse.For the final pass with regard to coding the open-ended data, all three authors met to review every response and discuss differences in our perspectives, recategorizing phenomena and even revising the initial code list until we were all in agreement.During this process, we used a combination of inductive and deductive thinking to create a framework of how the initial open codes might relate to one another and to identify the top five activities that students described as engaging or disengaging (as a means of identifying, using our limited set of open-ended data, rudimentary "axial" and "selective" codes per Glaser & Strauss, 1967, and Strauss & Corbin, 1997, 1998, that could be investigated further in a more in-depth qualitative research project).

Survey of Student Learning Engagement
Four variables measured students' learning engagement, and all revealed similar findings, with 42-46% of students being highly engaged and 13-22% being minimally engaged (see Table 1).The global learning engagement score was the only single-item variable, in which students simply rated their learning engagement on a scale from 1 to 10.As Table 1 indicates, the scores on this variable were distributed similarly to the scale variables.As previously described, the two NSSE learning engagement scale variables-higher order learning and reflective and integrative learning-were composed of multiple statements and rated on four-point Likert scales, with scores converted to a 60-point range (with never corresponding to 0 and a great deal corresponding to 60) to match with NSSE scoring.The mean Higher Order Learning value in this sample, which included both undergraduate and graduate students, was 40.59, and the mean Reflective and Integrated Learning was 40.76.This is comparable with seniors at public institutions, as found in Gonyea and Kinzie (2016): In a national survey performed for the Council of Independent Colleges, first-year students at public institutions had a mean Higher Order Learning value of 38.7 and a mean Reflective and Integrated Learning value of 35.1, whereas seniors at public institutions scored 40.9 and 38.5, respectively.The items within each of these scales and their frequencies are found in Table A1 in the Appendix.One item, "I connected ideas from the courses to prior experiences and knowledge," had the highest percentage of a great deal agreement (41%) and the fewest students who stated never (3%).All other items had a range from 31-38% of a great deal and 5-11% of never.
The last scale variable measured participation engagement using Dixson's OSE instrument.Students' mean score in our sample was 3.8, which is below Dixson's observed mean score of 4.2 (Dixson, 2015).The items and their frequencies are found in Table A2 in the Appendix.The distribution of students' responses varied more with this scale variable than the previous variables.The three items with the lowest positive responses included (a) students get to know other students (7%), (b) students have fun in online classes (11%), and (c) students help fellow classmates (11%).Yet the highest-rated items in this scale were that students post regularly in discussions (34%) and participate actively in discussions (32%).All four learning engagement variables were found to be significantly correlated with each other, as indicated in Table A3 in the Appendix.

What Activities Contribute to Learning Engagement for Online Students?
In analyzing students' responses to the survey, this question was first answered with the quantitative data and then expanded with qualitative findings.First, we examined Pearson's correlations and determined that all scale-engagement variables were significantly related to global learning engagement and to each other.Then we analyzed items within the NSSE and OSE scale engagement variables.While all items again were statistically significant, correlated at p ≤ .01,there were four items with a moderate correlation of .5 to .7.Students (1) have fun in an online classroom, (2) connect ideas from prior experiences and knowledge, (3) changed the way they understand an issue or concept, and (4) connect their learning to societal problems or issues.
The question then became which variables were most predictive of overall learning engagement.We conducted a regression model using these variables, along with student age, gender, and whether the student worked outside of school as control variables (see Table 2).This was significant (R 2 = 0.484, p < .000),suggesting that almost half of the variance in global learning engagement can be explained via this model.Furthermore, all noncontrol variables were statistically predictive of the global learning engagement level, with "changed the way I understand an issue or concept" having the greatest impact.Note: R 2 = .484,df = 7, n = 415, p < .000

Quantitative Content Analysis of Open-Ended Survey Questions
On the survey, students had the option of answering the two open-ended questions and could write as little or as much as they wished.Most offered detailed (and thus "codable") explanations to both, although some students gave a generalized response in one or the other, such as everything was engaging or disengaging, nothing was engaging or disengaging, they liked or did not like the course (without further explication), or simply that the course was required for their degree or program.These generalized responses provided no useful insights, and although we categorized them as indicated in Table 3, these responses were not included in the final codefrequency counts for what engaged or disengaged students.

Most Engaging Activities Identified by Students
The five most mentioned engaging activities included discussions, interactive assignments, specific topics covered in the course, use of media, and long-term or in-depth individual assignments (see Table 4).Discussions.More than one third of responding students mentioned discussions as an engaging activity in their online courses (139, or 35.6%).Many just wrote "discussions," but others elaborated a bit-for example, "I felt I could be more vocal than what I normally would be in class"; "The online discussion was a great way for shy students to speak up"; and "Discussions really got me involved in my online classes!"Because so many respondents identified discussions as engaging, we reviewed these answers to investigate whether students provided greater detail and/or explanations as to why.In doing so, three threads emerged.Students seemed to enjoy (1) hearing different points of view, (2) sharing their own perspectives, and (3) responding to thought-provoking questions.Regarding the third thread, sometimes the professor provided questions, and other times the students posted their own questions.Examples are listed in Table 5. Thought-provoking questions "Discussion boards with questions that made me really think and imagine different scenarios.""Constructing questions about course material and answering them among students.""Asking classmates two questions of your own." Assignments.One fifth of responding students (82, or 21.0%) reported interactive assignments as engaging, and a smaller percentage of students (34, or 8.7%) found individual assignments engaging.Assignments were coded as "interactive" if they involved activities that forced students to do something outside of their online learning environment or textbook.Students described many creative assignments as engaging, such as creating WebQuests, making a video, interviewing and structured observations, playing an online game that tested decision-making based on life's "uncertainties," and participating in online simulations.Assignments were coded as "individual" if they were a more traditional long-term and/or analytical activity in a course (e.g., writing a paper or conducting further research), as opposed to shorter term homework assignments.In examining students' responses in these categories, themes of independence and interactivity emerged-even within guided and more traditional assignments.Examples are listed in Table 6.

Table 6
Encouraging Activities for Student Engagement: Assignments Themes presented by students for encouraging engagement in assignments Illustrative student quotes Interactive "Creating webquests and doing projects.""Incorporating my personal experiences and life endeavors into assignments.""When the activities allowed me to interact outside of the online learning environment."Individual "None really encouraged me, but I did enjoy doing assignments that allowed me to pick from a list of items to write a research paper on versus being given just one specific topic.""The [assignments] that encouraged me to be most interested […] were the ones that gave me questions that I could find through the book.It actively engaged me in the chapter that we were discussing." Course content.Regarding course content, 51 students (13.1%) found specific topics in their classes engaging, and 41 (10.5%) were engaged through the professor's use of media (videos, podcasts, or other nontraditional media and visuals).Threads that can be found in these students' comments include content (1) that was presented in a more interactive or illustrative fashion and enhanced understanding of the more traditional materials and assignments in a course and (2) that students could connect with their professional careers and/or everyday life.Examples are listed in Table 7. Use of media "The fact that [the] teacher used an online tool instead of the book for learning.It makes me want to be more engaged.""Medical Terminology […] had fun online simulations to learn the proper terms to relate to the anatomy of the body.""The stories, videos and extra readings that back up the information that we are studying for at the time.They can be from history or everyday real life.""I enjoyed the video clips of teachers talking us through the power points and giving examples, etc.Just posting a power point for us to review does not accomplish teaching.""Links to direct websites or videos that were real world scenarios."

Disengaging Activities Identified by Students
Regarding the five most mentioned disengaging activities, this included course workload, individual assignments, general discussions, course organization, and professor feedback (see Table 8).Course workload.More than one sixth of responding students (49, or 17.5%) mentioned a course's workload as being disengaging.Students disliked "a lot of reading," "having so many assignments due in a little amount of time," "daily [d]eadlines [that] were sometimes difficult to remember," "the frequent amount of discussions that were due in such a short amount of time," "excessive amounts of weekly requirements (5 discussion posts and responses and two one hour quizzes every week)," and "having every single assignment from the publisher assigned to me."

Individual assignments (and homework).
Student responses about individual assignments (47, or 16.8%) as well as homework (30, or 10.7%; see Table A5 in the Appendix) being disengaging support the finding that interactive assignments are more engaging, especially those that are relevant to the course material and/or a student's life.For example, one student said that "research papers that have nothing to do with course, just doing it to draw out [the] course" are not engaging.Similarly, another student stated, "Some assignments just seemed like busy work [sic], and when the professor didn't reply to any emails [this] made it seem like the class wasn't worth the input."Students appeared to value practical application and connection to the real world of what they are learning."I was only discouraged by the course because it lacked practical application," another student emphasized, saying "an example would be in a clinical setting utilizing the terms you have learned."Likewise, similar to the finding reported above that students found sharing their own opinions engaging, one student explained that not being able to do so in assignments was discouraging: "The journal assignments [were disengaging] because we couldn't put opinions in it.What good is a journal if you can't write your own thoughts?"Repetition, either in the form of assignments or how material was presented, also was problematic from a student perspective.One explained that "topics were presented in the same way every week, assignments seemed pointless, and it was too easy to fudge the labs," and another mentioned "seemingly unnecessary 'busy work ' [sic] just to have something to turn in." General discussions (and criteria for discussions).The fact that one quarter of responding students referred to discussions in some manner as disengaging (47, or 16.8%, regarding discussions in general and 24, or 8.6%, regarding discussion criteria; see Table A5 in the Appendix) calls into question the finding that more than one third of respondents found discussions as an engaging activity in an online course.Examining students' more detailed responses may shed light on this.Many students reported disliking the requirements associated with a discussion post, such as "having to reply to everyone who replied to you on discussion posts" or "hav[ing] to repost to two other classmates['] posts," "having a word count," "discussion posts that took longer than 20 min[utes]," and "deducting points from discussions because I posted later in the week."Students also found discussions that appeared to be "busywork" and lacked challenge or practical application problematic.For example, students reported as disengaging "discussion topics that seemed to have little relevance to the course material, they seemed like just a way to have a grade" and Pointless discussion questions.Writing a discussion question is fine, but expecting students to react to a discussion question online is a little far-fetched.It is obvious when you read the mandatory reactions to what you posted as a discussion that, 1. people don't really have a conflicting opinion and if they do they won't post it.2. They are writing words so that they get credit for that part of the assignment.Thus, criteria regarding discussions as well as the selection of topics may contribute to learning engagement or disengagement.Students deemed topics disengaging if they were too "straightforward," did not have practical application, and did not challenge students' thinking and perspectives.As one student explained, discussions that were disengaging involved "topics that weren't good enough to have a discussion.Usually they're straight answers and everyone agrees."Another stated, "Personally, I dreaded the discussions because they were fact based rather than application.I like applying knowledge rather than looking up and relaying knowledge."Others, like the following student, felt discouraged by "responding to classmates who wrote about the same thing.I understand what discussions are trying to accomplish, but they don't accomplish a discussion about anything.It's just another writing assignment." Course organization.Additionally, one out of seven responding students (40, or 14.3%) found course organization and assignment instructions too confusing or lacking in application.Student responses here included "paragraphs of words [in] directions make the assignment seem harder than it really is"; "changing/adding to the course topic, and the changes were not standing out"; "odd deadlines or no notifications of assignment deadlines"; "most assignments were due all at the end of the week which made them add up, unlike a regular class where assignments would have been distributed through[out] the week"; and "the assessments had nothing to do with the course." Professor feedback.Lastly, 35 (12.5%) of the responding students stated they found the level of feedback a professor provided or the professor's lack of responsiveness and timeliness of grading as disengaging.One student summed up what many of the others in this category said: "I feel as though taking a class online, I dont [sic] get as much one on one with my professer [sic] as I would like, to answer questions or just for guidance."Other students provided comments such as "not being able to reach [the] instructor when I need understanding," "lack of interaction and communication from the professor," "not having assignments graded in a timely manner," "the teacher did not give good feedback," "bad rubrics," and "when the professor is not an active participant and does not give feedback […] regularly."

Discussion
The following research question drove this study: What activities contribute to learning engagement for online students?We first developed an instrument from prior research (NSSE, 2015;Dixson, 2015) to define and characterize activities that support student learning engagement in online courses.Our analysis of student surveys revealed that students appear to be more engaged when they learn something that changes the way they understand an issue or concept, connect their learning to societal problems or issues, have fun in online chats or discussion, and/or connect new learning to prior experiences and knowledge.These learning concepts then were clarified with practical learning activities identified through the analysis of the two open-ended questions.Specific activities that enable engaged learning included discussions, interactive assignments, and certain topics (and/or the means by which topics were presented) that allow students to (a) present their own perspective, (b) listen to other perspectives, (c) bring in personal experiences, and (d) address significant social issues.These activities are initiated with thought-provoking questions, created either by the professor, the student, or fellow class members.
Both the quantitative and qualitative data elucidate avenues that professors and administrators can explore to increase learning engagement for online students.

Connection
An overarching thread in the qualitative findings suggests that what students in online classes seek is connection-to oneself, to others, and/or to course material.Students reported being more engaged when such connections were present.(This likely is also true for on-site students.)Connection may merely be a synonym for learning engagement, but viewing our findings through the lens of connection offers concrete examples of what engaged students.
With regard to connecting with oneself in an online class, students valued independence and initiative in activities, such as choosing paper or discussion topics (e.g., "I dislike the professor picking the topic of a discussion post that I may not be very interested in") or completing assignments (e.g., looking up answers in the textbook to questions posed by the professor, as opposed to consuming a lecture, or conducting research beyond the textbook); they also wanted to share their opinion and have their voices heard.The former contributes to a sense of discovery and individual accomplishment and the latter a feeling that individuals and their contributions matter.
Connection to others, which also may be easier to foster in an on-site class than online, contributes to social presence and can in turn bolster connections to oneself and course material.These reasons may explain why one third of students in this study ranked discussions as an engaging activity in their courses.Discussions not only enable students to share their own opinions and have their voices heard, they allow for feedback, dialogue, and debate, which can change the way students understand an issue or concept or help a student connect a topic to their own knowledge or societal problems.Those who do so are likely to describe discussions and chats as "fun."However, as our findings illustrate, connection to others via discussions must be authentic, with questions or topics that are challenging and not rote, and must occur in an environment in which students feel able to express differing opinions and engage with one another about nuances and divergences.This places an onus on instructors not only to draw out experiences and opinions but also to ensure that students can do so in an appropriate and respectful manner.As other researchers have discussed (see the "pedagogy of discomfort" in Chenneville, 2017, and the use of "counterstories" in Farver & Dunn, 2017), professors walk a fine line when raising controversial topics in the classroom that can (and many times should) make students uneasy, and this may be more precarious in an online setting.Thus, in this sample, discussion forums were most popular as engagement tools among students when they involved thought-provoking questions, provided opportunities for students to share their perspectives (and possibly even create and direct the discussion), and enabled students to hear and understand the voices and perspectives of others.
On the contrary, students in our study found discussions disengaging when they were associated with burdensome requirements or amounted to routine busywork that failed to foster connection to other classmates or to course material and/or lacked practical application.Discussions were preferred to be authentic as well as "just right" in number: not too many and not too few.(One student, for example, stated that the "number of responses should not exceed 4 people," while another disliked "forced discussion without replies.")While workload can be an issue in an on-site course, the challenge may be greater online, as online classes often must find ways of delivering material and creating interactions that occur more naturally face-to-face.
Connection to course material was another theme evident in students' responses.Students preferred interactive assignments that took them away from consuming information (e.g., reading a textbook) or relaying information (e.g., written responses or quizzes that prioritized knowledge over insight or analysis) and instead encouraged connection to prior experiences or other courses as well as practical application.Students identified as disengaging assignments that were less interactive, especially ones that seemed irrelevant to their learning and, thus, a waste of time.

Deep Learning
These findings seem to corroborate other research results regarding learning engagement, most especially those of the NSSE.Other studies, however, provide additional insights.Robinson (2011) studied student participation in online discussions, which were not required or graded, and found that 84% of students rated the discussion forum useful, with 64% of students accessing the discussion board from one to three times per week.Convenience and enjoyment of reading and writing posts were the primary motivators for using the discussion boards.However, he noted that students' content generally was not related to their coursework.Johnson (2016) examined the content and depth of online discussions to determine how much students were engaged in their learning.Discussion posts by students were examined for whether students interpreted previous posts, expanded on the topic, and shared a new or different understanding of the topic.It was found that students were able to collaborate and develop meaning, often with the help of an instructor's discussion prompts.Although not all students participated in the discussions, those who did engaged in deeper knowledge building.Johnson concluded that it is essential for instructors to assess students' level of discussion contribution-not by length and frequency of posts but, rather, by quality and how well posts contribute to deeper, more meaningful construction.Students should be allowed to explore ideas with each other and share their personal experiences, but then they must build upon this larger base of experiences to construct a new, deeper understanding of a topic.
As opposed to superficial learning, such as simple memorization and other rote approaches, Czerkawski (2014) has described deep learning as involving higher level or active cognitive processes: "Deep learning occurs when students are able to connect with course topics, find value in them and see how to apply them to real-world situations" (p.30).In relation to deep learning, several reviews and online studies support our four primary findings.
Enhancing (often modifying) one's understanding of an issue.College students often enter their programs and courses with perspectives taken from parents or developed during their youth and have not had ample time or opportunities to thoroughly understand-let alone explainthem.One core action for deep learning introduced by Hacker and Niederhauser (2000) is to require students to construct deep explanations, justifications, and reasons for what they think and do.Hacker and Niederhauser found that this exercise consisted of much scaffolding, but by the end of the course students did achieve a significant level of deep learning.
Enabling understanding of diverse perspectives.This reflective learning is not easy to achieve, as noted by others (see Fear & Erikson-Brown, 2014).Yet students in our study, similar to those in others' research, noted that they not only enjoyed hearing others' views during online discussions, but appreciated expanding their own viewpoints.Czerkawski (2014) recommends that instructional designers provide students with "increasing meaningful dialogue between course participants" (p.36).This is dialogue in which participants are "open to other people's views and acknowledge the different roles and feelings presented" (Czerkawski, 2014, p. 36).
Incorporating past experiences into one's current perspective.This is an especially difficult area to develop for students and instructors, yet requiring students to bring into discussions or even their papers their past personal experiences was one of the activities that seemed to engage students in our study.For professors, such a task may be challenging, as much teaching at the university level is and must be evidence-based.However, it appears students need this addition of past experiences to help them engage in their learning.Fear and Erikson-Brown (2014) identified key factors for successful online discussion posts from students, and they found that bringing in "personal experience" relevant to the topic as well as "divergence" added greatly to the quality of discussions (divergence meaning that discussions often must first have some disagreement or conflict in order for students to reach a level of learning).York and Richardson (2012) also found support for this in their review of online learning quality, emphasizing that self-disclosure and personal examples added significantly to engaging student discussions.The sharing of a student's personal experiences and then incorporating them into specific content in a course are learning activities highly relevant to deeper, engaged learning.
Improving understanding of significant societal issues.Most students welcome the incorporation of current events into courses and course material, but for true engagement we found that students must stretch their understanding of events or issues.Many business professors, for example, advocate problem-and case-based learning as effective activities for producing this level of thinking (DeLotell, Millam, & Reinhardt, 2010).Using real cases or societal problems helps students make the connection between course topics and real-world application.With problembased learning, students are given questions that relate to actual issues or problems, and they are then expected to gather information, explain their findings, and come to a solution.With casebased learning, actual case studies are shared, which students analyze and respond to using their own perspectives and opinions.The findings in our study suggest that both approaches would be useful for increasing engagement with online students, as these types of assignments may be more interactive and, at the very least, wouldn't be viewed by students as "pointless" or "irrelevant" (i.e., the types of assignments students reported as disengaging).

Recommendations
The research presented here seems to confirm that truly engaged online learning is "instructor facilitated and student owned" (Schroeder-Moreno, 2010, p. 28) and that engaging students in online learning requires mastery of both content and online teaching strategies.These are challenging tasks; however, our findings, as well as those of others, offer some areas for improvement-all of which are geared toward enabling students to build on their current knowledge, change some former perceptions, and see their learning as significant to societal issues.
Selection and presentation of topics.In a course, the selection of specific topics seems to further students' engagement.Of course, professors may not always have full control over what topics can or must be included, and students' personal preferences and experiences may influence what topics they find more engaging or disengaging.Students taking a course in their major, for example, may find topics more engaging than those completing a course as a general requirement.However, students in our study stated that they were engaged by topics that were connected to the "real world," timely, and/or controversial.Course materials or assignments related to any topic can be found or crafted that draw on such aspects.Moreover, if this category of "specific topics" from our data is examined in relation to another aspect some students reported as engaging, "use of media," how a professor presents a topic online might make a difference.The use of nontraditional materials (videos and other media, interactive games or software, etc.) can help with the delivery and exploration of course topics beyond foundational textbooks.
Interactive assignments.Fundamental to more engaged online participants in a course are well-planned questions and activities that encourage students to reach beyond the online community and assist them with bridging theory and practice.These often employ nontraditional materials and media and include assignments such as labs, hands-on exercises, simulations, and other student-driven projects (e.g., WebQuests, research-based presentations).This may be difficult, however, in an online environment, where students may not readily have access to needed resources, such as lab space or equipment or expensive software.Moreover, instructors likely need greater administrative and pedagogical support to find, update, and pay for such components, as well as to effectively develop and assess nontraditional assignments that meet course and program objectives.
Authentic and challenging-but-respectful discussions.Discussions are engaging when they move beyond the rote, allow students to create connections with classmates and real-world issues, and challenge students with opportunities (in a safe environment) to explore differences and disagreement.Some students in our study particularly appreciated discussions when they could pose their own questions, rather than just respond to questions developed by the professor, and when they were prompted and felt comfortable presenting contradictory details and opinions.Certain aspects of online discussions, however, can be disengaging, such as the number of discussions and how they are graded.As noted above, students may feel disconnected with too few discussions or unclear grading requirements, but they may feel overwhelmed and like they are just doing busywork if there are too many discussions or grading requirements that are too onerous.Student responses from our study thus suggest that a "Goldilocks" approach is needed with regard to discussion requirements and criteria-that is, not too few and not too many (although students often are not able to understand or value tasks and assignments that in the present might seem difficult or time-consuming but overall contribute to learning).
How discussions are executed also is important.Doing so effectively requires clear expectations and rules about "netiquette," as well as monitoring and engagement by the instructor so that students feel connected with the professor as well as safe in offering diverse reactions.As Johnson (2016) noted, it is vital that professors actively manage online discussions and offer insightful, guided responses to students.Clarke and Bartholomew (2014) reached similar results, though they noted in their analysis of instructors' participation in online discussions that instructors overwhelmingly provided comments that were encouraging, positive, and reaffirming, while students wanted more feedback and "constructive criticism that would lead to new understanding of content" (p.19).Additionally, several authors found effective the use of bi-and tri-level questions, and their work potentially supports our findings for engaged learning: Level 1 questions are those where the answer can be found in course materials; Level 2 questions require students to relate materials to a personally relevant answer (using their personal experiences); and Level 3 questions ask students to find connections between course materials and broader social contexts (Fear & Erikson-Brown, 2014;York & Richardson, 2012).While professors can create some of the aspects and environment needed for such engaging discussion experiences, support from the university also might be necessary, including resources to assist instructors with feedback and assessment and training students in appropriate ways to craft, share, and receive diverse opinions in a classroom setting.
Presence of the professor and clear organization.Students in our sample suggested that regular participation by the professor and in-depth and timely feedback contribute to engagement.Additionally, manageable expectations in an online course and clear directions and due dates are key.For example, for students who stated they felt overwhelmed by or lost as the result of heavy workloads or confusing instructions, notions of independence and learning faded, and frustration grew in their place.While this also can be a challenge in on-site courses, those students can meet face-to-face with the professor during class or office hours, and a professor might add further explanations or walk through an example in class, adding verbal guidance to what is written.Thus, the organization of an online course and the accessibility of and feedback provided by the instructor can affect engagement.Some organization, accessibility, and feedback issues are under the direct control of a professor; however, these also depend on the platform a university uses and the support administrators provide to both instructors and students.More supplementary resources likely are needed in supporting professors in the development and delivery of online content and assignments as well as in preparing and assisting online students-especially when universities offer "flex-term" or expedited sessions that require students to complete full courses in less than a typical semester, when many of today's students are juggling school with work and family obligations and/or expect, especially in an online course, that assignments and deadlines should work around their schedules.

Limitations
The limitations of this study should be acknowledged.Although all students enrolled in at least one online course at the university were invited to participate in the survey, survey completion was voluntary.Thus, a self-selection bias might be present.Moreover, some of the students might have intentionally chosen to take an online course because they like this format better, whereas others might have registered for an online course because this was their only option.More research is needed to understand distinctions in learning engagement between students who voluntarily opt for online courses versus those who take them due to external circumstances.Additionally, some professors allowed students to earn extra credit for completing the survey, which also might have influenced responses.Furthermore, all students who opted to participate completed the survey during the last few weeks of the spring semester.Their perspectives and opinions may have been biased by typical end-of-semester (and end-of-school-year) pressures and/or relief.Lastly, all students surveyed attended one public university.The online platform used (Blackboard) and peculiar online policies and practices at this university may mean findings should not be universalized.
Further research also is needed to replicate and/or expand on our initial findings and to create more concrete suggestions for effective online strategies and activities.The survey in this study focused on students' perspectives; follow-up studies also should examine instructor (and even administrator) perspectives.Lastly, a more in-depth qualitative study based on semistructured face-to-face interviews with online students (and others) would provide greater detail and nuance about what students find engaging and disengaging in these courses.Professor feedback

Appendix
Including the level of feedback as well as the professor's responsiveness and timeliness of grading 5 (1.3%) Course effort

Convenience
The course fitting into a student's overall schedule better and/or the ability to complete assignments on one's own time 10 (2.6%)

Easiness
Whether in terms of assignments, assessments, or pace (time spent each week or ease of getting ahead)

Table 1
Learning Engagement Measures and Their Distributions

Table 2
Linear Regression Model for Global Learning Engagement Score

Table 3
Number of Students Who Effectively Did Not Answer the Open-Ended QuestionsThus, out of the 417 students who completed the survey, 390 (93.5%) provided a detailed answer to the first open-ended question about what activities they found engaging in their online course, and 280 (67.1%) provided a detailed answer to the second open-ended question about what activities they found disengaging.In these detailed responses, students often mentioned more than one activity (e.g., interactive assignments and discussions), and we coded each response with one count of as many applicable codes from the code list.Frequency counts of codes are therefore reported out of a possible 390 student responses to the first open-ended question and 280 possible student responses to the second open-ended question.For example, in the first open-ended question about what students found engaging in their online course, 139 out of 390 student comments (or about 35.6%) mentioned discussions.For a complete list of codes and their frequencies, please see TableA4and TableA5in the Appendix.

Table 4
Most Engaging Activities Identified by Students in an Online Course

Table 5
Encouraging Activities for Student Engagement: Discussions

Table 7
Encouraging Activities for Student Engagement: Course Content

Table 8
Most Disengaging Activities Identified by Students in an Online Course

Table A4 Code
List and Frequencies for Activities Students Found Engaging in an Online Course

Table A5
Code List and Frequencies for Activities Students Found Disengaging in an Online Course