PREDICTING LEARNING FROM ASYNCHRONOUS ONLINE DISCUSSIONS

Online discussions play an important role in student learning. Currently, asynchronous learning courses and some primarily face-to-face courses at New Jersey Institute of Technology require asynchronous online discussions. Does this requirement improve the students’ perceived learning? An exploratory study designed to investigate this issue was conducted in Spring 2002. The questionnaire drew responses to a post-course questionnaire from 116 students in two undergraduate courses and one graduate course. A set of items designed to measure perceived learning from online discussions formed a reliable, unidimensional index for this construct. The results indicate that online discussions do improve students’ perceived learning. Variations among instructors or courses are associated with differences in perceptions of student motivation, enjoyment, and learning from online discussion. Open-ended questions identified some student concerns about online discussions. The concluding section of this paper presents some implications for improving online discussions and for future research plans.


I. INTRODUCTION
Computer conferencing has become popular in the past two decades.By sharing "some of the spontaneity of and flexibility of spoken conversation" [1], computer conferencing successfully transfers the face-toface conversation into a virtual space.Because of many benefits for learners, computer conferencing technologies have been adopted into the education field in recent decades.Harasim [2,p. 121] defines computer conferencing as "a group communications medium enabling groups of people to exchange ideas and opinions and to share information and resources."As one study indicates [3], the benefits of computer conferencing include convenience, place-independence, time-independence, and the potential for users to become part of a virtual community.Computer-mediated communication (CMC) creates a unique environment for discussions that have removed many barriers for students to participate, because the students can access their online classroom "anytime, anywhere" via Asynchronous Learning Networks (ALN).Phillips and Santoro [4] regard "Asynchronicity" as the main benefit of CMC, and "the idea of having a rapid way to transfer questions and answers seemed to facilitate participation." Online discussions are "a key feature of computer conferencing" [5].Currently, both asynchronous and synchronous computer conferencing have been widely used in distance learning courses.Asynchronous online discussions are our major concern rather than synchronous online discussions that require real-time online participation.This paper describes a pilot study that investigates student perceptions of learning from asynchronous online discussions that were conducted in addition to face-to-face meetings of the classes (e.g., "mixed" or "blended" mode courses, which are becoming increasingly prevalent).Over 50 US online learning program directors who belong to a consortium of colleges and universities that offer fully online university degree programs responded to a survey in October 2002 and projected that the proportion of their students enrolled in totally online courses would increase from 20.2% to 36.6% in the next few years, but those enrolled in "blended" courses would increase even faster, from 7.6% to 21.1% of their students [6].However, there are relatively few studies of the determinants of the effectiveness of online discussions in blended courses.Some possible underlying factors that explain perceived learning from these discussions are explored in this paper, which is structured as follows.Existing research that indicates how online discussion affects student learning is reviewed, followed by a description of an exploratory study conducted in Spring 2002 at New Jersey Institute of Technology.We then show how the use of online discussion as a learning vehicle affects the perceptions of student learning.In the last part, the findings of this study are emphasized and future research is discussed.

II. LITERATURE REVIEW
When online discussion becomes part of learning activities in both asynchronous learning courses and primarily face-to-face courses, does it promote active student learning?Harasim [7] suggests several key differences between computer-mediated and face-to-face discussions: time dependence, place dependence, the structure of communication and richness of communication.Larkin-Hein [8, p. F2G-6] said, "The use of online discussion groups offers a relatively new avenue through which the learner can take an active role in the learning process."However, how computer-mediated discussion affects the students' learning is unclear.Over ten years ago, Hiltz [9, p. 100] asked in the "Virtual Classroom™" project: "whether students do take a more active part in the learning process and take advantage of the potential for more interaction with professor and the other students, despite the absence of nonverbal cues to facilitate this interaction."In later studies, Harasim [2] points out that learners often perceive more learning in online interactions compared with a face-to-face class or on the phone.Althaus [10, p. 158] said, "In theory, online discussions help more students learn better by placing them in an intellectual environment that encourages active, thoughtful, and equal participation from all comers."Althaus [10] examines whether supplementing face-to-face discussion with computer-mediated discussion (CMD) enhances the academic performance of undergraduate students in large lecture classes.One hundred and forty-two undergraduates were involved in this study which found that a combination of face-to-face and computer-mediated discussion provides a superior learning environment compared to the traditional classroom alone.[11] present data results from a pilot study at American University during Spring 1998.

Hein and Irvine
The pilot study focused on assessing the effectiveness of the discussions in terms of students' understanding of physics.Sixty-two students volunteered to participate in the online discussion group.Two purposes of online discussion in this study are addressed: (1) to give students a new vehicle through which to learn physics; and (2) to allow the authors an opportunity to study and assess the impact of the online discussions on students' learning of physics.The findings are: (1) students found the online discussions were beneficial and useful; (2) many students think that online discussion should be continued in future classes; (3) students respond that online participation should remain voluntary because of sufficient other activities required in the course; (4) the instructor's role is not only to provide the opportunity for in-depth investigation into a topic, but also to provide constructive feedback to students in the online discussion process.
Larkin-Hein [8] reports ongoing research designed to address the role of student understanding in physics using an online discussion group format.In terms of gauging student understanding in physics, this research assesses the effectiveness of online discussion groups as a learning and assessment tool.The findings are as follows: (1) Online discussions provide an additional learning and teaching vehicle.(2) Online discussions facilitate the acquisition of higher-order thinking skills.(3) Students became more adept at transferring and applying information learned in class to novel situations.(4) Further study is needed to investigate how personal learning styles affect online discussion.
Thomas' [12] study of online discussion assumed that the nature of computer-mediated communication impacts students' learning.Survey, case study and content analysis methods were used in data analysis.This study found that online discussion forums promote high levels of cognitive engagement and critical thinking.However, the virtual learning space of an online forum does not promote the coherent and interactive dialogue necessary for conversational modes of learning.To overcome incoherence, the role of moderator and facilitator of discussion is important in the online discussion forum.
A large survey was conducted at SUNY Albany in Spring 1999 [13].Thirty eight hundred students who enrolled in 264 courses through SLN (SUNY Learning Network) responded to a post-course survey.The findings show that three factors (consistency in course design, contact with course instructors, and active discussion) significantly influence the success of online courses.The researcher also found that interaction with instructors, interaction with peers, and the value placed on participation in course discussions are the factors most significantly related to student perceptions of learning.These findings support the importance of interaction for online teaching and learning.However, none of the prior studies provide or test a causal model of the determinants of perceived learning from online discussions.Are there differences between males and females in their participation in and benefits from online discussions?There have been mixed results for this question [14].On the one hand, early studies of computer-supported learning showed that women tended to be at a disadvantage because they are less likely to have their own computer and have less confidence about their expertise with computers.On the other hand, females consistently out-score males on tests of verbal ability [15], and CMC discussions are thus potentially a place for female students to do better and feel more comfortable than male students.

III. RESEARCH MODEL DESIGN
What are the processes that determine whether online discussions will result in improved learning?Online discussions that persist throughout the week should motivate students to be more engaged in their course on a continuous basis, and to think through the concepts and issues more thoroughly, because they need to restate the issues in their own words.Secondly, active participation in online discussions, which are student-dominated rather than instructor-dominated, should be enjoyable for the students.It should make learning more active and "fun."These are shown as the intervening processes or variables in the model, and should help to explain variations in resulting perceived learning.
A variety of "independent" or contextual variables may influence whether students will be motivated to take part in online discussions, and whether they will enjoy them.For this initial version of the model, we included three such variables.Those who have taken prior online courses should be more familiar with the process of online discussions and thus should feel less anxiety and uncertainty, resulting in higher motivation and higher enjoyment.Gender is related to reading and writing skills, and thus females may experience online discussions as more motivating and enjoyable than males; on the other hand, they are less confident with the use of computers; these factors may balance out.Finally, the instructor's role in framing and facilitating online discussion activities is crucial to their success.
The following research model (see Figure 1) is our framework to guide this study.

A. Research Questions
Q1: Can we obtain a measure of perceived learning from online discussions that will have high internal consistency?Q2: Can we devise measures of motivation from learning online, and enjoyment of learning online, that will have discriminant validity and high internal consistency?These questions will be tested with Cronbach's Alpha and confirmatory factor analysis.

Motivation And Enjoyment
Instructor Role

Independent Variables
Intervening Variables Dependent Variable

B. Research Hypotheses
The following research hypotheses are derived from the model.

Hypothesis 1:
Students who perceive more motivation from online discussion will report higher perceptions of learning from online discussion.
Hypothesis 2: Students who perceive more enjoyment from online discussion will report higher perceptions of learning from online discussion.
Hypothesis 3: Students who have previously taken more distance learning courses will attain higher perceptions of learning, more motivation and more enjoyment.
In our research model, gender is considered as an independent variable.As Gilligan [16] suggests, males and females have different communication patterns.Women tend to provide mitigating and social comments while men address issues directly and eventually-via discussion-downplay the female contribution.Do gender differences affect the learning from online discussions?Many gender studies, focusing on education, find there are no strong differences between female and male students.Ory [17] did not find any gender differences in the use of and attitudes about ALN, after one year of implementation in a university setting.Another gender study [18] indicates that women think of the virtual classroom as an opportunity for collaboration and networking with other participants, while men emphasize the message content.The results of this study also show that there are no significant gender differences in learning, and moderately significant differences in class participation.Therefore, we expect there will be no gender differences in online discussions context.
Hypothesis 4: There will be no difference between female and male students in perceptions of learning, motivation and enjoyment from online discussions.
We are concerned about the instructor's role in online discussions.From previous studies, the instructor role is a complicated factor in determining students' motivation, and perceived learning.In her "Virtual Classroom™" project, Hiltz [19, p.102] said, "The key activity in the Virtual Classroom is the exchange of ideas and information among students, with the faculty member as an organizer and facilitator of the activities and discussions."Gorham and Zakahi [20] point out that decreased students' learning outcomes are related to increased physical and/or psychological distance between teachers and students.Another study [21] identifies a positive correlation between teacher immediacy behaviors and students' learning, though different racial and ethnic groups have different meanings of immediacy behaviors.However, Gorham and Christopher [22] found that students regarded motivation as being determined by themselves rather than the instructors.A recent study [12] shows that the role of instructor is important in online discussions.Based on these studies, we present our Hypothesis 5 as follows: Hypothesis 5: Instructors will play an essential role in promoting students' motivation, enjoyment and perceptions of learning online.

V. METHOD
Our study was conducted in Spring 2002 at New Jersey Institute of Technology.Online discussions in three courses were conducted on WebBoard™ and WebCT™, both including asynchronous computer conferencing systems.Participants included 116 students from two undergraduate courses and one graduate course.All of these sections were "mixed mode" courses for which asynchronous online discussions were added to face-to-face class meetings.These three courses met three hours per week for the whole Spring semester.Instructors gave the traditional face-to-face lectures in the classroom.They not only assigned students course readings, writing, and projects as usual, but also required weekly asynchronous online discussions via WebBoard™ or WebCT™.The topics of these online discussions were based on course designs and objectives.Students shared their learning experiences and had intensive online participation throughout the whole semester.At the end of the Spring 2002 semester, the students who voluntarily participated in our study signed consent forms and filled out post-course questionnaires based on the whole semester's online discussion experience.The details of student and course information are listed in Table 1.

VI. RESEARCH RESULTS
Descriptive statistics were used to summarize the basic findings and correlation analysis was used to explore underlying relationships among variables.

A. Basic Findings
The raw data gathered from the three courses were analyzed by using Excel™ and SPSS™.Our basic findings are shown in terms of perceived learning effects, motivation, enjoyment and correlations among variables.
Table 3 (see Appendix) presents students' perceptions of learning from online discussion.Over half of the students felt that they learned a great deal from their peers through online discussion.Fifty-seven percent of the students thought online discussion increased their learning quality (mean=3.72).Over 78% of students thought online discussion was a great chance to share opinions among peers and instructor.A confirmatory factor analysis was performed to determine if all of the items designed to measure perceived learning loaded on a single factor; they do.In addition, the Cronbach's Alpha of over 0.90 indicates high internal consistency for the set of items.Therefore, these results answer the research question 1: we have a good measure of students' perceived learning from online discussions.
Students were also asked about the online discussion motivation and enjoyment.Table 4 (see Appendix) shows that about half of the students were motivated by and enjoyed sharing knowledge with their peers through online discussion.These results are less positive than we anticipated, but they do provide considerable variance for testing our model.

Table 4 (see Appendix
) shows the answer for research question 2: we could not distinguish students' motivation and enjoyment based on our current research instruments.Initially, the first four items were designed to measure the extent to which online discussion motivated students to be more interested and work harder.They were intended to form an index to measure motivation.The last four items were intended to form an index to measure the extent to which online discussion was an enjoyable experience.However, a confirmatory factor analysis showed that the eight items comprise a single construct, not two separate constructs.Therefore, they were combined into a single index, which has high internal consistency (Cronbach's Alpha=0.9006).
About 68% of students reported that their instructor played a critical role to motivate effective online discussions, and only 14% of students disagreed.(see Table 5 in Appendix and Figure 2).

B. Correlation Analysis
To explore the underlying dimensions and test our research hypotheses, we used correlation analysis for the data gathered via questionnaires.Looking at the correlations in Table 2, the following statements show that four of the research hypotheses are supported, while one of the research hypotheses is not supported.(Of course, it must be remembered that correlation does not prove causation.)Hypotheses 1 and 2: Students who perceive more motivation and enjoyment from online discussion will report higher perceptions of learning from online discussion.(Supported) It can be seen from the table that a significant correlation of .477indicates that those who perceive more motivation and enjoyment from online discussions also rate themselves higher for perception of learning from online discussions.Hypothesis 3: Students who have previously taken more distance learning courses will attain higher perceptions of learning, more motivation and more enjoyment from online discussion.(Not supported) Hypothesis 4: There will be no difference between female and male students in perceptions of learning, motivation and enjoyment from online discussions.(Supported) Hypothesis 5: The instructor will play an essential role in promoting students' motivation, enjoyment and perceptions of learning online.(Supported)

C. Data Results from Open-ended Questions
From the questionnaires, we also analyzed some responses from open-ended questions.
Students were asked to comment about what they think should be improved in the online discussion process.The students' responses to this question included: • "Instructors need more interesting topics to guide the online discussion, such as case studies" • "Students need the basic tutorial to use the software, such as how to use WebBoard™/WebCT™" • "Students need more feedback from either peers or instructors" • "Instructor needs to spend more time to catch the online discussion" • "Synchronous online discussion should be a good supplement" • "It is not easy to follow a big group.Too many people" • "Difficult to consistently log online to participate in discussion" • "Frustration about the software down" • "Need more clear format for online discussion beforehand" • "Need user-friendlier interface" • "Some comments are not so useful" • "Instructors should be available online for two or three hours everyday so that students can ask questions whenever they want…" • "Assign rules to lead discussion" • "The motivation put forth by the professor is very important" When students were also asked what they liked best in the online discussion, answers included: • "Interaction (Students also expect more interaction with their instructors)" • "Convenience" • "Ease of use" • "Less pressure" • "Anonymity feature" • "Provides more social interaction" • "Flexibility of time and space" • "No transportation needed" • "The dynamic nature of content and the sharing of experiences among peers" • "Being connected between students and instructor" • "Clearly and openly express opinions without fear" Overall, the results of the above questions suggested that students found the online discussions beneficial and useful to them.Many students felt that online discussions have benefited their learning skills and have improved their learning quality.The students enjoyed the flexibility of online discussions.These students' comments are valuable for improving the teaching quality of online discussions because these responses identify some weakness of current online discussions.For example, some students suggest improving user interfaces, needing more guidance from their instructors, and adding more interesting discussion topics.The suggestion that instructors should devote more time online may not be reasonable, given total instructor workloads.More structured online discussions will avoid or decrease inefficient online discussions.To some extent, active student online participation is a matter of students being made comfortable with the medium [19].Especially for students who have less computer experience, better software interface and functionalities are desirable.
Furthermore, these comments provide some clues for the future of online discussion research, because they identify some useful factors that may be included in further studies, such as how unstructured vs. structured online discussions will affect students' perceived learning.

VII. CONTRIBUTIONS
Based on the previous studies, this study contributes in several ways: The study provides an initial research model that may be expanded and generalized for future online discussion studies.Most prior studies focused on broader topics, such as collaborative learning, the evaluation of online courses and so forth.Our research model aims to increase and deepen understanding of how students perceive learning from online discussions.
Our study is also one of the few that identify underlying factors that affect perceived learning from online discussions.We found two highly reliable index variables: one is "perceived learning from online discussions" (Cronbach's Alpha=0.9049) and the other is "students' motivation and enjoyment from online discussions" (Cronbach's Alpha=0.9006).These two variables have been tested by confirmatory factor analysis; so they can be used as validated instruments in future research.

VIII. LIMITATIONS OF THIS STUDY
The generalizability of our findings is limited in many ways.Only a small number of courses and instructors were included, and all of these were "mixed" mode courses at a single university.We need to compare these results to those for totally online courses.Instructor's role and level of activity were measured by a single item, and only in terms of student perceptions.A measure of perceived instructor role should have more items that distinguish among different dimensions of the discussion leader role.In the current study, only students' perceived learning from online discussions was measured; objective learning effects may be different from perceived learning.
The most severe limitation is that a single post-course survey cannot prove "causality."The measures of perceived learning were collected at the same time as the measures of instructor role behavior and perceived motivation and enjoyment.It may be that students do not clearly separate their enjoyment of the process of online discussion from their learning as a result of that discussion.A longitudinal study that measured the levels of enjoyment/motivation and instructor behavior, and of perceived learning at different points in time, followed by an objective measure of learning from the contents of the discussion, would go far towards establishing the causal relationship among these correlated variables.

IX. CONCLUSIONS AND FUTURE RESEARCH
In this paper, the results suggest that online discussions definitely improved students' perceived learning in the "mixed mode," meaning classes that meet face-to-face and also require additional asynchronous online discussions.A research model was generated to explore the underlying variables that affect perceived learning from online discussions.The findings demonstrate that variations among instructors or courses are associated with differences in perceptions of student motivation, enjoyment and learning.There are no significant gender differences in online discussions.A surprising result is that the number of Distance Learning courses that students previously took has no relationship with perceived learning from online discussions.From open-ended questions, students' reactions and concerns about online discussions show that the instructor plays an important role to motivate effective online discussions; more online guidance, more structured discussion topics and considerable time devotion are required for instructors.Future research is needed that looks at a much larger data set, preferably from multiple universities, and that compares pure ALN courses to face-to-face plus online discussion courses, and adds additional contextual variables such as students' learning styles as a new factor into the online discussion research model.It is also desirable to redesign the research instrument to further explore measures of motivation and enjoyment.Another topic for research is how newer mobile PCs and the use of digital audio as well as text-based discussion, may affect the process of participation in, and perceived learning from online discussions.

XII. APPENDIX Table 3: Students' Perception of Learning from Online Discussion
Questionnaire Category: SA=Strongly Agree; A=Agree; N=Neither Agree nor disagree (neutral); D=Disagree; SD=Strongly Disagree; S. D.=Standard Deviation; N=Number of Responses

Figure 1 :
Figure 1: Online Discussion Research Model

Figure 2 :
Figure 2: Instructor's Role in Online Discussions

Table 1 : Number of Subjects in the study in Spring 2002
116Among these participants, 61.2% are pursuing their Bachelor's degree, 33.6% are Master students, 2.6% are in graduate certificate programs, 0.9% are non-matriculated, which means not officially being admitted into a Master's program, and 1.7% are Ph.D. students majoring in Information Systems or Management of Information Systems.Regarding their gender, 34.5% are females and 65.

Table 4 : Online Discussion Motivation and Enjoyment Questionnaire Category:
SA=Strongly Agree; A=Agree; N=Neither Agree nor disagree (neutral); D=Disagree; SD=Strongly Disagree; S. D.=Standard Deviation; N=Number of Responses

Table 5 : Instructor Role Questionnaire Category:
SA=Strongly Agree; A=Agree; N=Neither Agree nor disagree (neutral); D=Disagree; SD=Strongly Disagree; S. D.=Standard Deviation; N=Number of Responses