How much does student engagement with videos and forums in a MOOC affect their achievement

Engagement in Massive Open Online Courses (MOOCs) is based on students who self-organize their participation according to their own goals and interests. Visual materials such as videos and discussion forums are basic ways of engaging students in MOOCs. Student achievement in MOOCs is typically measured using assessments distributed throughout the course. Although there is research on the basic forms of student’s engagement and assessment in MOOCs, little is known about their effect on students’ achievement in the form of students completing a MOOC. Using binomial logistic regression models, this paper addresses this gap in the literature by presenting the degree to which student engagement with videos and forum posts can predict students’ probability of achievement in a MOOC. It also explores the extent to which participation behaviors and their intention to receive the course certification can be used to predict achievement in MOOCs. Using qualitative content analysis, this paper discusses the quality of the forum posts exchanged by participants in this MOOC. The findings from quantitative analysis support MOOC’s pedagogical assumptions, showing that students’ engagement in forums and with videos increases the probability of course achievement. It also shows that intention to certify plays a moderator effect on the number of videos watched, enhancing achievement in MOOCs. The findings from qualitative analysis reveal that most students’ posts in forums display more information acquisition than critical thinking. Implications for practice suggest MOOC designers and MOOC instructors foster engagement in forums by implementing discussion prompts that foster interactions about deep meaning of concepts or application of concepts covered in the MOOC. In regard to videos, implications for practice suggest the creation of interactive videos that promote students’ engagement and control such as inserting guiding questions and segmenting the video content. Future research comprising multiple MOOC cohorts is suggested to validate the empirical model presented in this study.


How Much Does Student Engagement with Videos and Forums in a MOOC Affect Their Achievement?
Massive Open Online Courses (MOOCs) are open learning environments that have the capacity to enroll a large number of participants.In MOOCs, participants freely engage and disengage with available resources and share their learning experiences with other participants in discussion forums.MOOC resources are commonly comprised of online reading materials, videos, quizzes, discussion forums, and assessments.Due to their open nature and students' autonomy, MOOCs are also known for having a high number of students drop out (Ho et al., 2014;Coetzee, Fox, Hearst, & Hartmann, 2014).The combination of students' autonomy and students' dropout rates has drawn the attention of researchers for better understanding the forms of students' engagement and its contribution to students' achievement in MOOCs (e.g., Onah, Sinclair, & Boyatt, 2014).
In this paper, we explore the effect of students' intention to complete the MOOC and their engagement with videos and forums on their achievement in the Creativity, Innovation and Change MOOC (CIC MOOC) delivered by The Pennsylvania State University.Binomial logistic regression models were used to present the degree to which learner's engagement with videos and forum posts can be the basis for predicting course completion and receiving a certificate.We use qualitative content analysis to gain knowledge about the quality of the forum posts exchanged by participants in this study.For that, we make use of the Interaction Analysis Model (IAM) provided by Gunawardena, Lowe, and Anderson (1997) to evaluate participants' online messages in terms of co-construction of knowledge.Afterwards, we discuss the results through the lens of the literature and indicate implications for practice and research.

Review of Related Literature
The most common ways to engage students in MOOCs are insertion of visual materials such as videos and the use of discussion forums.Formative assessments (low or no point value quizzes) and summative assessments (graded evaluations) are often used to evaluate participants performance in MOOCs.This section presents a brief review of the literature about students' engagement with videos and forums, students' completion and their assessment in MOOCs.

Student Engagement with Videos in MOOCs
The majority of MOOCs delivered through platforms such as Coursera, edX, and Udacity heavily rely on videos to deliver course content.Videos give flexibility to participants and allow scalability for MOOC providers in delivering course content (Lee et al., 2015).These videos vary in length, in position within the course, and in purpose.According to Morris and Lambe (2014), MOOC videos are characterized by: (a) introductory videos in which course instructors explain the course and its purpose, (b) animations with audio narration in which the course content is explained, (c) video lectures given to real students, (d) documentary style video, (e) interviews or conversations among instructors and guests, and (f) video with built-in questions.A MOOC may make use of a particular video style or a combination of different styles according to its purpose of learning.
Benefits of videos in online learning can be found in the literature (Triay, Sancho-Vinuesa, Minguillón, & Daza, 2016;Morris & Lambe, 2014).For instance, videos can be paused, repeated, or skipped through, allowing flexibility in the learning process and reinforcing student autonomy in MOOCs.Short videos intertwined with quizzes emulate one-on-one tutoring and tend to fit into a manageable period of time that students can dedicate to MOOCs (Glance, Forsey, & Riley, 2013).
Although videos are the primary vehicle for delivering content in MOOCs, research seems to suggest that high quality videos are not enough to provide high quality experience to participants (Guo, Kim, & Rubin, 2014;Coetzee et al., 2014).Lately, Lee et al. (2015) are investigating the potential of including time-anchored commenting interfaces along with videos that allow participants to watch the video and at the same time to exchange comments with others about the content being learned.This initiative has the potential to transform engagement with videos from passive to active.

Student Engagement with Forums in MOOCS
MOOC forums have many purposes.They can be a space for students to interact with other participants, get to know their peers, and learn through their experiences (Young, 2012).They can also be a space for students to check their understanding of the subject matter and to ask questions regarding a task or a problem (Young, 2012;Darabi, Arrastia, Nelson, Cornille, & Liang, 2011).Forums are an environment for cooperation among students (Coetzee et al., 2014) in which students can learn as much from their interactions with others as they do from their interactions with course materials (Thomas, 2002).Forums can be a space where participants create new knowledge by interacting with others about concepts and techniques (Dubosson & Emad, 2015).
According to Dubosson and Emad (2015), Young (2012) and Koller (2012), MOOC forums have been proven to be a good environment for peer assistance, in which students tend to answer each other's questions without instructor intervention.However, research is unclear regarding the effectiveness/benefits of discussion forums.Some researchers found that discussion forums promote high quality discussion of course content, allowing students to reflect upon course materials and upon each other's comments (e.g., Walker, 2007).Additionally, the implementation of discussion forums allows students the opportunity to initiate discussions and to drive their own learning (Darabi et al., 2011).In contrast, critiques of the use of discussion forums state that only a small fraction of participants contribute to forums, producing a high quantity of posts in relation to most participants (Coetzee et al., 2014).Threads may cover different topics, in different languages, showing problems of organization, which may intimidate participants and diminish their engagement (Anderson, Huttenlocher, Kleinberg, & Leskovec, 2014).As the number of posts and discussion threads become overwhelming, participants may feel less confident about engaging in forums (Dooley & Wickersham, 2007).In this sense, forums become spaces in which most students' posts tend to display information acquisition instead of critical thinking (Kanuka & Anderson, 2007).For these authors, forums may support students' increase of knowledge, but still fall short in presenting evidence as a venue for development of students' new knowledge.

Student Completion in MOOCs
In open online environments such as MOOCs, participants are allowed to choose how they want to pursue their engagement.MOOC completion has emerged as an important metric being used by MOOC researchers to define course performance (Belanger & Thornton, 2013;Breslow, Pritchard, DeBoer, Stump, Ho, & Seaton, 2013).MOOC completion here is understood as completing course requirements and earning the Statement of Accomplishment certificate.The reason for choosing participant completion as a metric in MOOC is related to the straightforward way to collect information from the platform and completion being a variable that can be used to compare students' performance in different MOOCs (Wang & Baker, 2015;Moore & Kearsley, 2011).However, when analyzing the literature of MOOC completion, the high levels of participant dropout emerge as a concern for MOOCs as a pedagogical environment, with completion rates varying between 5% and 15% as presented by Kizilcec and Schneider (2015), Ho et al., (2014), andJordan (2014).
Due to the open environment of MOOC's and participants' autonomy of engagement, different participants may have different perspectives of what completion means.Our view is that completion in MOOCs gains a personal connotation that is aligned with participant's goals in that MOOC.Loizzo et al. (2017) highlight that for some participants completion was related to acquiring new resources and interacting with the MOOC platform.McAuley, Stewart, Siemens, and Cormier (2010) also pointed out differences in participant views when defining MOOC completion.These authors noticed that as participants define completion they tend to do it based on their "learning goals, prior knowledge and skills, and common interests" (p.4).On the other hand, as described by Anderson (2013), many participants who enroll in MOOCs do not have the intention of completing the course.This divergence of perceptions between MOOC providers and MOOC participants has led the field to explore alternative approaches to understanding MOOC completion (Koller, Ng, Do, & Chen, 2013).One of them is to focus on comparing students' performance against their intentions in a MOOC (Kizilcec & Schneider, 2015).

Student Assessment in MOOCs
MOOCs offer regular opportunities for students to verify and test their understanding throughout the course.Thus, participants engage with non-graded quizzes as a way to test their knowledge, and at the end of the course they can elect to take a scored test for a course certificate.Student achievement in MOOCs is measured using assessments, which are mainly distributed into automated assessments (e.g.multiple-choice quiz), peer-assessment (e.g., students evaluate each other's work), and self-assessment (students assess their own work).Since MOOC pedagogy is primarily based on mastery of learning (Glance et al., 2013) and there are a large number of students enrolled, the above approach seems to work well with MOOCs focused on serving a high volume of participants.
As noted by O'Toole (2013), automated assessment may be used in situations where knowledge is fragmented into simple facts, algorithms, procedures, or explicit chains of reasoning.When knowledge is embedded in more complex situations, other forms of assessment may be applicable (e.g., peer-assessment).In a broad view, the literature on assessment indicates a need for online courses to create assessments for learning and not only assessments of learning (Admiraal, Huisman, & van de Ven, 2014).It is important to create assessments and feedback that are scalable, in which students can benefit from a reliable evaluation process with usable feedback that fosters opportunities for student learning (O'Toole, 2013).

How Much Does Students' Engagement in MOOCs Add to Their Achievement?
Previous research indicates that student engagement in forums and watching of videos are related to their achievement in online courses (Kizilcec, Piech, & Schneider, 2013).The main purpose of the current study is to determine the effect of these two activities on student achievement in a MOOC.When analyzing student achievement in online learning, one must consider that achievement is related to a unique combination of the course characteristics and participants' profile.This means that other MOOCs with similar course characteristics (videos and forums) may not produce the same degree of student achievement as seen in the MOOC analyzed in this study.
Thus, this study aims to answer the following research questions: (I) To what extent is students' achievement in MOOCs associated with the number of posts made and the number of videos watched?(II) What effects does intention to certify have on student achievement in MOOCs?(III) What effects does intention to certify have on the number of videos watched and the number of posts made when considering student achievement in a MOOC? (IV) What cognitive activities are performed by participants as they interact in forums?

Study Context
This study was conducted with students from the Creativity, Innovation, and Change (CIC) 2.0 MOOC via the Coursera platform from July to August, 2014.The CIC MOOC was delivered over six weeks focusing on supporting students in achieving their creative potential, empowering them to transform their personal lives, organizations, and community.No pre-requisites were required to register for this course and students had to submit their work on a weekly basis for the full six weeks.Weekly lessons were comprised of students' engagement with videos about explanations of new concepts and tools (e.g., CIC mindset and Intelligent Fast Failure), performing self-assessment upon their creative style, completing projects (e.g., shoes tower), readings, exercises, quizzes, and engaging in forums (Jablokow, Matson, & Velegol, 2014).For students who were interested in obtaining a certificate of completion, two options were offered.To receive a Statement of Accomplishment certificate, students had to submit six weekly tasks.To receive a Statement of Accomplishment with Distinction certificate, students had to complete the requirements for a Statement of Accomplishment certificate and submit 12 peer reviews throughout the course.

Data Collection and Data Analysis
Participation in this study was voluntary (N = 222), and recruitment was done upon students' completion of a survey that explored the effect of groups on students' success (course completion).Students of the CIC MOOC were invited to participate in an online survey that requested information regarding demographics, employment status, intention to complete the course, and preferred language.Data from the survey was used to gain knowledge about the characteristics of participants (demographics) and their intentions to complete the course.The data on number of videos watched, number of posts, and post content were retrieved from the Coursera platform.
Students' responses to the online survey along with their course interactions (i.e., number of videos watched and number of posts) were used in statistical analyses to build answers to research questions I, II, and III.Student's forum posts were qualitatively analyzed using the Interaction Analysis Model (IAM) developed by Gunawardena et al. (1997) in order to build answers to research question IV.In answering the research questions I, II, and III, we created statistical models using stepwise binomial logistic regression.Age, gender, number of posts, and number of videos were the initial independent variables used to predict participants' probability of achievement in the course.Participants were categorized into six age levels: 15-25 (25.87%), 26-35 (24.13%), 36-45 (18.53%), 46-55 (17.83%), 56-65 (9.44%), and 65 and above (4.20%).
Participants self-identified themselves as female (63.06%) and male (36.94%).Number of posts is comprised of participants (N = 222) posting new discussion threads and replying to each other's posts.Number of videos watched represents all instances in which participants clicked the play button of a video in this MOOC.By design, data collection did not focus on participants' engagement within videos such as pausing, fast-forwarding, and skipping video parts.Intention to certify were measured using a 5-likert scale 1-5 (strongly disagree -strongly agree).Participant achievement was coded as none, normal, and distinction, and later recoded as a binary variable indicating whether they achieved certification in the course.Next, we describe the Interaction Analysis Model used to build answers to research question IV.
The IAM model (Table 1) was used as a tool to examine the cognitive activities performed by participants in their forums interactions.Data collection was comprised of all posts (450 posts) generated by the 222 participants across all forum categories (e.g., week1, week2, etc.).The IAM model comprises five phases of knowledge co-construction that occur during the online discussions (Gunawardena et al., 1997).They are: phase 1-sharing or comparing information, phase 2-discovery and exploration of dissonance or inconsistency among ideas, concepts or statements, phase 3-negotiating of meaning or co-construction of knowledge, phase 4-testing and modification of proposed synthesis or co-construction, and phase 5-agreement statement(s) or application of newly constructed meaning.Four hundred and fifty forum posts were analyzed according to the model.The unit of analysis was participant posts in the discussion forums.Each post was independently coded by the first and the third authors according to the level of cognitive activity (see column code, Table 1) and later checked for consistency and divergence.If any disagreement occurred in the coding, the code selected was the one based on a majority amount of evidence presented in the post.The inter-rater reliability was Cohen's Kappa = 0.99, indicating substantial degree of agreement between the two coders.

Participants Demographics
Participants who took part in this study (N = 222) came from all over the world.Table 2 presents participants distribution and the countries where they were located.Data shows that Chinese participants accounted for the largest number of volunteers who participated in our study (21.6%), followed by participants from the United States (17.6%).This large number of Chinese students is related to the fact that the course has been translated into Chinese language.
The demographics indicate a sample of 140 females and 82 males.From a total of 222 participants in this study, they classified themselves on a scale of four English levels as Poor (5.4%), Basic (22.5%),Fluent (41.4%), and Native (30.6%).Their age levels were classified into six ranges: 15-25 (30.6%), 26-35 (25.2%), 36-45 (16.6%), 46-55 (7.6%), and 66 and above (4.5%).In regard to their job status, 76 participants stated they are employed full-time (34.23%), 45 participants were full-time students (20.27%), 37 participants were self-employed (16.67%), 17 participants were employed part-time (7.66%), 16 participants were looking for a job (7.21%), 10 participants were part-time students (4.5%), and 21 participants were retired, not working, or on maternity leave (9.46%).Regarding their intention to complete this MOOC, participants were asked to choose from a 5-likert scale 1-5 (strongly disagree-strongly agree) upon the statement "Intent to complete the course".The distribution of participants' answers to this survey question was: Strongly Disagree (3.15%), Disagree (6.70%),Neither Agree nor Disagree (54.30%),Agree (29.40%), and Strongly Agree (6.45%).It shows that in the pre-course survey, thirty-five percent of participants in this study indicated an intention to complete the entire MOOC.Participants' course completion data was collected through Coursera with three levels of completion: none, normal, and distinction.These three levels of completion were recoded as a binary variable indicating whether they achieved certification in the course: complete (the combination of normal completion and completion with distinction) and non-complete.Participants' achievement in this MOOC was distributed as: complete (41.52%) and non-complete (58.48%).On average, participants posted twice in forums and watched 40 videos throughout the course.The number of videos watched includes the possibility that some participants may have watched a video more than once.

Results
This section presents results of the analyses conducted to examine the extent to which student achievement in MOOC is associated with the number of posts made and the number of videos watched.It explores the effects of intention to certify on student achievement and on number of videos watched and number of posts made by students in MOOC.It also presents the cognitive level performed by participants as they interacted in forums.Statistical analyses were performed using the R programming language and the R-Studio Integrated Development Environment.Afterwards, discussion and conclusion are presented.

To what extent is student achievement in MOOCs associated with the number of posts made and the number of videos watched?
Results from binomial logistic regression model 1 (Table 3), presented both NofPost (p = 0.0236) and NofVideoWatch (p = 9.98e-12) as statistically significant when considering participants' achievement (Akaike Information Criterion, AIC = 220.7).Age and gender were not statistically significant regarding participants' achievement.Parameter estimate for number of posts regarding the student achievement was 0.27, meaning that the number of posts is positively associated with student achievement.The 1.31 odds ratio for NofPost indicates that a one-point increase in the number of posts made is associated with the probability of MOOC achievement increasing by a multiplicative factor of 1.31 (Table 4).The estimate for frequency of videos watched on student achievement was estimated at 0.06 which informs that the number of videos watched is positively associated with student achievement.The 1.061 odds ratio for NofVideoWatch indicates that a one-point increase in the number of videos watched is associated with the probability of MOOC achievement increasing by a multiplicative factor of 1.061 (Table 4).In answering our first research question, students' achievement in MOOC is positively associated with the number of posts they made and the number of videos they watched.

What effects does intention to certify have on student achievement in MOOCs?
Next, we added participants' intention to obtain a course certificate as an independent variable in the logistic regression model 2 (Table 5).The results for NofPost (p-value = 0.02297), NofVideoWatch (p = 9.87e-12), and CertificateIntention (p = 0.00447) were statistically significant when considering participants' achievement.The model improves when compared to the previous one, presenting a lower AIC = 213.45.The odds ratio of binomial logistic regression coefficients for model 2 is presented in Table 6.The 1.32 odds ratio for NofPost indicates that a one-point increase in the number of posts made is associated with the probability of MOOC achievement increasing by a multiplicative factor of 1.32.The 1.062 odds ratio for NofVideoWatch indicates that a one-point increase in the number of videos watched is associated with the probability of MOOC achievement increasing by a multiplicative factor of 1.062.Likewise, the 1.708 odds ratio for certificate intention indicates that a one-point increase in certificate intention is associated with the probability of MOOC achievement increasing by a multiplicative factor of 1.7.Answering our second research question, model 2 shows that for the ones who intended to obtain a certificate, their probability of MOOC achievement increases by a multiplicative factor of 1.7 when compared to the ones who didn't intend to receive a MOOC certification.Thus, the results above support the basic model that predicts student achievement in the CIC MOOC.In ANOVA test, model 2 presented a significant reduction of deviance (9.250) when compared to deviance in model 1 (75.799), as presented in Table 7.

What effects does intention to certify have on the number of videos watched and the number of posts made when considering student achievement in MOOC?
To understand the effects that intention of certification has on the number of videos watched and the number of posts made by MOOC students, we explored the logistic regression including interaction terms related to intention to certify and their moderation effects in students' achievement (model 3).For the moderation effect of intention of certification, we used stepwise binomial logistic regression as presented in Table 8.Model 3 showed the best outcome when comparing results from other models (lower AIC).By including the interaction in the model, previously significant estimation of independent terms was reduced, and some independent variables became insignificant.
Analyzing the predictors of model 3 in Table 8, only the interaction between "number of videos watched" and "intention of certification" was statistically significant (p-value = 0.0113) with odds ratio =1.019 (Table 9).Model 3 shows us that intention to certify does not play a moderating effect between the number of posts and student achievement.On the other hand, intention to certify has a moderating effect between the number of videos watched and student achievement.Answering our third research question, model 3 shows that an increase in engagement in videos for the ones who intend to receive a certificate is positively associated with an increase in their MOOC achievement.Synthesizing the results from our first three research questions, this study showed that student achievement in MOOCs is positively associated with the number of posts made and the number of videos watched.From model 2 we learned that a one-point increase in NofPost is associated with the probability of MOOC achievement increasing by a multiplicative factor of 1.32, and a one-point increase in NofVideoWatch is associated with the probability of MOOC achievement increasing by a multiplicative factor of 1.062.For students who intend to obtain a certificate, their achievement in the MOOC increases by a multiplicative factor of 1.7 when compared to the ones who do not intend to obtain a course certificate.We also learned that intention to certify does not play a moderating effect between the number of posts and student achievement.On the other hand, intention to certify has a positive moderating effect between the number of videos watched by students and their achievement in this MOOC.In the next section, we qualitatively analyze participants' posts to depict the cognitive activities performed by them through their interactions in forums.

What cognitive activities are performed by participants as they interact in forums?
We used the Interaction Analysis Model (IAM) from Gunawardena et al. (1997) to examine the cognitive activities performed by participants in their forums interactions.Results from qualitative coding on 450 forum posts are shown in Table 10.Participants' forum posts were coded through the IAM's levels from phase 1 to phase 3 (see Table 10).No forum posts were located under phase 4 (testing tentative constructions) and phase 5 (statement/applications of newly constructed knowledge) of the IAM.Data shows that the majority of participants' posts were situated on making statements of observations related to the course and statements showing participants' opinions, as represented under the code Ph1/A comprising 72.07% of the data.This prominent characteristic of participants' posts may be understood by looking at the forum design of the CIC MOOC.In the forums, it was recommended that participants post to others by using the P.U.R.E approach of giving feedback.According to the CIC MOOC page (https://www.coursera.org/learn/creativity-innovation),the P.U.R.E.approach stands for Positive, Unique, Revision and Education as described in Table 11.
Analyzing the P.U.R.E.framework recommended to CIC MOOC participants as they interact with others in forums, we noticed that the suggested script enhances participants' collaboration, minimizing conflict among them.According to Gunawardena et al. (1997) in discussions where there is little conflict among participants' ideas, participants tend to "accept each others' statements or examples as consistent with what the group members already know or believe and the discussion may never advance out of phase one" (p.415).Another factor that may have contributed to participants' lack of higher levels of cognitive activities might be related to the nature of the posts in this sample.Although data collection comprised all posts from the 222 participants across all forum categories (e.g., week1, week2, etc.), when content of these posts was qualitatively analyzed through the IAM we noticed that many of these posts' content resembled the structure of participants' initial posts in MOOCs, in which they typically introduce themselves and state their goals and expectations for the course without challenging ideas of each other.For example, "Hello I'm Jaime [pseudonym].I'm a software engineer in Taiwan.I like innovative products and want to know more about creativity.Cheers, Jaime."This piece of evidence leads us to hypothesize that these participants may have had a higher engagement at the beginning at the course.

Revision
Third, be a "critical friend" -what suggestions do you have for improvement or revision.The purpose is not so much "to grade" as to interact and suggest.

Education
Close by reflecting on what you learned and how you were educated from the submission as a feedback provider.Qualitative analysis of participants' forum posts showed that they tend to post in the format of statement of opinion followed by a question.For example, "I'm struggling with creative blocks.I think part of it is fear of failure but what are some strategies to get your creative juices flowing?"This intrinsic characteristic of the nature of participants' posts justifies the high quantity of Ph1/A and Ph1/D codes in Table 10.In analysis of CIC MOOC posts, participants also engage with others in forums by making a statement of agreement with other participants' posts (Ph1/B) and corroborating with ideas of others (Ph1/C), for example, "Hello Sidney![pseudonym] As I reflect, I agree that [the] shoe tower exercise is simple but profound.As you head into your new venture it will help you to innovate and build from small failures and challenges along the way." In terms of disagreement and negotiation of meaning (phases 2 and 3 of the IAM), participants' posts were mainly concentrated in Ph2/A in which they stated potential disagreement/divergence with the MOOC's rules and/or with the MOOC's activities, as in the example: I'm disappointed in this choice of exercise.I work with people who own one pair of shoes-actually flipflops-at a time.The choice of shoe tower reflects a cultural myopia that has me worried about the rest of the course...This participant was referring to the task presented in week 2 of the CIC MOOC in which students had to build the tallest shoe tower without using any external support.The shoe tower task was designed to foster participants' creativity dealing with limited resources (available number of shoes) and to help them reflect about the strategies used in their creation process.In the data sample, only two posts presented Ph3/A level and only one post presented Ph3/B level, in which the participant intended to clarify meaning and the weight assigned by another participant to the definitions of the word "bad."Participants' posts in this study did not show tendency to disagreement.In fact, in most of the posts they tended to be polite and friendly, which may reduce situations of dissonance and disagreement as the ones suggested in phase 2 of IAM and beyond.

Discussion and Conclusions
Using data from volunteer participants in the CIC MOOC, this study investigated the extent to which students' achievement in MOOCs is associated with the number of posts made and the number of videos watched.Results indicate that participant engagement in forums has a bigger impact on the probability of MOOC completion (32%) when compared to the contribution of videos watched (6%).Considering direction of coefficients, intention to certify had an amplifying effect on students' achievement, acting as a moderator in enhancing students' achievement.These results are aligned with the literature on MOOCs in which forum participation supports students' completion of the course (Breslow et al., 2013;Kizilcec et al., 2013;Waldrop, 2013;Daniel, 2012).
Qualitative analysis revealed that the forum contributions from volunteers in this study are mainly located in phase 1 of Gunawardena's et al. (1997) Interaction Analysis Model.This empirical result adds evidence to the literature of forums (Kanuka & Anderson, 2007, Thomas 2002) that state that most students' posts in forums tend to display information acquisition instead of critical thinking.In this sense, forums in this study worked as a venue in which students increased knowledge about others and about the course content, but fell short in presenting evidence as a venue in which students develop new knowledge.
Given the importance of participants' engagement with forums and videos in MOOCs, implications for practice suggest MOOC designers and instructors should create discussion prompts that foster interactions about deep meaning of concepts or application of concepts.Knowing that participants in MOOCs tend to engage in forums by posting under the format "statement" plus "question," MOOC instructors can capitalize on this common type of post to foster participant replies to posts of others.In doing so, this participant behavior may lead to the creation of a community of learning in which more knowledgeable participants answer the queries of novices or less knowledgeable individuals.On the other hand, if a forum presents only the behavior of participants asking questions but not replying to each other, it may incur the risk of a forum becoming a pile of posts as described by Thomas (2002).
In terms of videos, this study extends the literature showing the impact of videos in the probability of MOOC completion.MOOC designers can use the results of this study as a rationale to suggest the implementation of more interactive videos as described by Glance et al. (2013), intertwining segments of videos with quizzes, and improving the descriptions of videos so that participants know exactly what they can learn by interacting with that specific video.Although it is hard to control participants' engagement with videos in MOOCs by design, providing better guidance of what videos may contribute to participants' learning paths may help them to make better decisions regarding engagement.
In terms of research, although many may intuitively believe that student engagement with forums and with videos seems to be related to achievement, this study advances the field by showing how much these engagement types have the potential to affect student achievement.The combination of statistical methods and qualitative analysis provide to the literature a model of how to analyze student engagement in MOOCs.Results presented here support students' achievement in heterogeneous student populations, as the ones presented in MOOCs.By understanding the potential contribution from students' engagement with forums and with videos in their course achievement, new approaches can be developed to create and sustain learning through personalization.Knowing features that affect achievement and how much achievement is affected, MOOC designers can develop learning paths aimed at maximizing participants' completion of a MOOC.

Limitations and Future Work
The statistical analysis presented in this study makes use of a case study survey and click data which don't afford explanations regarding the causes and effects among the variables.Thus, our results regarding participants' intention to complete the MOOC are contingent on CIC MOOC students' honesty and disposition to volunteer in this study.The sample size of this study was relatively small when considering the large number of students who enrolled in this MOOC.Due to size effects, the findings may not be generalizable to other samples.This could be overcome with the implementation of longitudinal studies in which multiple MOOC cohorts are analyzed under the same statistical model and under the same qualitative framework.Knowing the potential impact on students' achievement as they engage with forums and with videos, in future work it will be interesting to study how the patterns found from the Interaction Analysis Model informs participants' learning outcomes.

Table 2 .
Distribution of Participants by Country

Table 3 .
Binomial Logistic Regression Presenting Number of Posts and Number of Videos as Statistical Predictors

Table 4 .
Odds Ratio of Binomial Logistic Regression Coefficients for Model 1

Table 5 .
Binomial Logistic Regression Presenting Number of Posts, Number of videos, and Intention of Certification as Statistical Predictors

Table 6 .
Odds Ratio of Binomial Logistic Regression Coefficients for Model 2

Table 8 .
Binomial Logistic Regression Presenting Number of Posts, Number of Videos, Intention of Certification and Interaction Terms as Statistical Predictors

Table 9 .
Odds Ratio of Binomial Logistic Regression Coefficients for Model 3

Table 11 .
P.U.R.E Approach of Giving Feedback Used in CIC MOOC