Asynchronous discussion in support of medical education

Although the potential of asynchronous discussion to support learning is widely recognized, student engagement remains problematic. Often, for example, students simply refuse to participate. Consequently the rich promise of asynchronous learning networks for supporting students’ learning can prove hard to achieve.After reviewing strategies for encouraging student participation in discussions in Asynchronous Learning Networks (ALN), we present a study that investigates how these strategies influenced students’ perceptions and use of the discussion area. We identify and explore factors that encouraged and inhibited student participation in asynchronous discussion, and evaluate student postings to an asynchronous discussion group by content analysis.The results question received wisdom about some of the pedagogic techniques advocated in the literature.Instead, results support the view that the major factors for stimulating student participation in asynchronous discussion are tutor enthusiasm and expertise. It appears that the tutor may be the root cause of engagement in discussions, an important conclusion, given that to date, the tutor’s role has remained relatively unexamined. We also note that participation in asynchronous discussion is inhibited when students allocate a low priority to participation, as may occur when participation is not assessed.Content analysis of an asynchronous discussion in this study reveals that contributions were not strongly interactive and that students were simply ‘playing the game’ of assessment, making postings that earned marks but rarely contributing otherwise. Thus the use of assessment to encourage students’ contributions appears to be only a superficial success; it seems likely that giving credit for postings changes behavior without necessarily improving learning. This finding has significant implications for curriculum design.


I. INTRODUCTION
The potential of asynchronous discussion to support learning is widely recognized. However, learning benefits are not inherent in the technology, but depend upon collaborative activities. Thus, for example, improvements in the quality of learning may be more likely if students actively participate in interactive online learning, if the tutor adopts a collaborative pedagogic strategy, and if students participate in a collaborative assignment [1]. Key to such recommendations is engagement in discussions in an asynchronous learning network.
Despite the fact that students' participation in asynchronous discussion is vital to maintaining interest, motivation, and engagement in active learning, participation remains problematic. Consequently, promoting discourse has become a major role of the e-moderator [2]. In this section, existing research on strategies for encouraging student participation in Asynchronous Learning Networks will be reviewed.
A. Using asynchronous discussion to support learning Mason [3] and others, for example, Paulsen [4], advocate the primary role of the e-moderator as providing a friendly, social environment where effective learning can take place. These writers adopt what might be referred to as pedagogic determinism, proposing that "sensitive and appropriate conference design and the e-moderator's intervention cause the socializing to occur." From this position, a number of techniques or models have been proposed, each of which attempts to specify what an appropriate design and form of intervention ought to be. Warren and Rada, for example, suggest that: … for participants to gain from and sustain potential benefits of online interaction, (a) they must begin contributing to collaborative learning immediately at the start of a virtual course; (b) the tutor must be involved as facilitator, encourager, structure provider and resource; (c) interaction must be required, structured and graded; and (d) participants must be 'mature, motivated learners'… They must also (a) share an interest and commitment to a well-defined task, (b) have ease of access to a reliable computer network, (c) feel a sense of responsibility to the group, (d) experience strong leadership and anticipate a final evaluation of the group task [5]. Salmon proposes an even more detailed five-stage model for what she calls e-moderating, which also takes into account the idea that the nature of communication may change at different points during a course [6].
In spite of guidelines such as these, studies show that the implementation of ALN-based discussion remains problematic. In some cases, for example, students simply refuse to participate.
No matter what I did, I could not get students to use the VC (virtual classroom). I sent them reminders that it was required. They would say, "I don't know how". I would send them a message on how to do it, they would say, "I don't have an account….." Instructor A: Hiltz et al, [1] A lack of participation may result from one of any number of reasons, including technical problems [7], limited access to a computer [8], feeling lost in the discussion, falling 'too far' behind the discussion [8,9], a lack of confidence or understanding [10], not having mastered the medium or specialist language of a particular forum, or simply being a 'freeloader' [6].
Each of these might be seen as a challenge to overcome. However, there is also what might be described as dilemmas, problems for which there simply does not seem to be a 'right' answer. For example, communication tools may be poorly used unless the tutor facilitates discussion; yet at the same time, the tutor's presence in discussion forums can stifle student participation [7]. Such complexities, and their influence on learning, are yet to be fully understood.
It should also be noted that students' non-participation in asynchronous discussion might not be such a bad thing [10], since they can learn a considerable amount simply from observing a dialogue [11].

B. The influence of assessment on participation
One particularly influential element of pedagogy recurs in the literature on ALNs: assessment. Warren and Rada conclude that grading contributions was one cause of increased postings [5]. However, Jones' ethnographic study vividly illustrates the pitfalls that can arise when using assessment to stimulate participation [12]. The course he describes explicitly encouraged online collaboration, and credit was given for such activities. As part of this study, he observed a group of students meet face-to-face in a computer lab; they then created a series of emails to each other, based on prepared notes, in order to maximize their marks. This example shows how students learn how to play the game of assessment, and how drastically it can distort their behavior.
In contrast, Beaudin's study of online tutors questions the centrality of assessment, suggesting that it may be a relatively unimportant strategy in terms of keeping discussion on-topic [13]. Respondents believed assessment to be less influential than, for example, carefully designed questions, providing guidelines for learners, or rewording questions, particularly for tutors who had taught five or more online courses in the last three years. However, interestingly, he also raises the possibility that the differences in perceived importance may also suggest that experienced tutors are more realistic than relative novices about the value of such techniques in keeping discussions on-topic, concluding that further studies might be required to understand what these tutors actually do.

C. Summary
The rich promise of ALNs for supporting students' learning may prove hard to achieve. It cannot be assumed that provision of a technological infrastructure will somehow cause collaboration and learning to take place. However, various factors seem to influence whether or not students engage in asynchronous discussions, ranging from the pragmatic (for example, easy access) to the pedagogic (for example, the influence of assessment). The remainder of this paper describes a study in which the students' views on the relative importance of different factors were analyzed in relation to their actual participation in a range of discussion forums.

A. The educational context of this study
The California College of Podiatric Medicine is a private, non-profit organization previously based in Vallejo, California.
The Podiatric curriculum is consistent with that of the majority of other medical schools and places significant demands on student time. Furthermore, because course examinations are routinely scheduled on Monday mornings, the precise time (over the weekend) when students needed faculty support the most was the time when they were most inaccessible.
During the spring semester of 2000, full time faculty in the department of basic medical sciences were given the opportunity to support their respective courses with a variety of asynchronous learning networks in an effort to enhance faculty-student collaboration. In addition to a discussion group, in which subjectbased interaction between faculty and students and between students could take place, faculty were able to establish discussion groups to share mnemonics or to make course-related postings and administrative changes in a Notices forum. With the exception of the author (Graham Shaw), this was the first exposure of faculty to asynchronous learning environments in support of medical education. Adoption of this technology was not made compulsory.
All courses were delivered using a traditional lecture or lecture/laboratory paradigm. The asynchronous learning environments were designed to run alongside lectures and laboratories and to promote interpersonal interactions between students and between faculty and students independent of place and time. Student participation in the various asynchronous learning environments was not compulsory initially; though, in an attempt to stimulate participation in the Biochemistry environment, credit was given for posting an assignment summary toward the end of the semester.
At the start of the semester the students were given an introduction to the asynchronous learning environment that included an online demonstration of the technology and clear written guidelines as to its use. In order to use the learning environment, students were required to register as a new user by completing a short online registration form and thereafter access was by username and password. Within the Biochemistry conference students were able to post a message, reply, delete, or 'post a new topic'. The Biochemistry conference was a closed learning environment visible only to the students in the course and to the course tutor. Student postings to the conference were not screened. The other discussion boards were open, allowing all registered students to view and post messages.
The conference starts with a 'Welcome' message posted by the Biochemistry moderator. In this message the moderator adopts a friendly, personal tone as advocated by Mason, encouraging participation [3]. Since this is the students' first interaction with ALNs, the moderator uses a 'light hearted' exercise to start and invites student contributions on the topic of Christmas holidays.
75% of students responded to the questionnaire; their academic backgrounds are shown in Table 1 [14]. Only Biochemistry, Human Anatomy and Lower Extremity Anatomy courses were supported by distinct asynchronous learning networks; faculty responsible for neuroscience and physiology courses chose not to take advantage of the asynchronous learning opportunity.
Within the questionnaire, in addition to basic demographic data such as gender and grade point average (GPA) on entry, students were asked to categorise their familiarity with using computers and the Internet and how many messages they contributed to each discussion board. They were also given a list of things that might have encouraged or prevented their participation, drawn from the themes identified in the literature, and asked to say how important each of these had been in influencing their engagement with the discussion. Students were also asked to rate how important each board had been in helping them to understand the topic, and to improve their scores for the course. Finally, students were invited to give feedback on the course.

D. Content analysis
In addition to the survey, one cohort's postings to an asynchronous discussion forum were analysed. The issue of how best to analyse such postings is widely debated; however, content analysis is frequently advocated as being suitable for such tasks [15]. This study was influenced by the work of Pilkington and others on dialogue, which concentrates on the interaction of particular types of 'moves' in conversations [7]. In practice, the structure of discussions that arose gave little opportunity for sustained conversations to be analyzed in this way; consequently, the data were classified into fairly broad categories (discussions of the subject and the course, or discussions of other issues, including technical problems) and then subclassified according to their status as conversational moves. This involved deciding whether a posting was a question, a self-contained statement, or a response. Responses were further sub-divided into 'response: answer,' 'response: additional question,' 'response: answer and additional question,' and 'response: statement.' Those responses which ended with a question invited further contributions to that particular conversational thread; those which were classified as answers attempted to provide a 'correct' response that would end the thread. 'Response: statements' offered opinions which were not integrated into the discussion; they were effectively asides from the conversation, and often simply indicated a person's opinion on a topic in a way that did not invite response.
This classification enabled a more detailed investigation of how the students acted in an online discussion, and a judgement of whether or not this engagement was particularly constructive. It also allowed issues of the reliability of students' responses to be investigated.
While the technology used to support each asynchronous learning environment remained the same, the topics and pedagogies varied. The following asynchronous learning networks provide the data for this study: Biochemistry, Lower Extremity Anatomy Discussion, Lower Extremity Anatomy Notices, Human Anatomy Notices, Human Anatomy Discussion, and Human Anatomy Mnemonics.  These levels of response fall slightly below those observed by Morris and others, who observed approximately 3 replies to every original message [16]. In this study, the level is around 2 replies per original message.

A. Students' attitudes towards the context for discussion
A primary aim for this study was to examine what students felt was important in encouraging them to contribute to the discussion, or what prevented their engagement. For each of these, students were offered five statements, modelled on issues in the literature, and asked to rate their importance using a 5-point Likert scale, with 1 as the weakest agreement and 5 as the strongest. There were 60 responses to the questionnaire (90%). Of these responses, 43 (71%) were male, and the average GPA score was 3.36, suggesting that the respondents were representative of the class as a whole. The results are summarized in Table 3.

Incentives Range Median Mode
The tutor was really enthusiastic about people using discussion areas 1-5 3 5 I valued the option to post contributions anonymously 1-5 2 1 Contributions were assessed 1-5 3 1 I think that engaging in debate and discussion with other students helps me to learn more

1-5 2 1
The group provided another opportunity for me to ask questions of the course tutor at any time 1-5 3 1 The discussion forum was an integral part of the course, rather than an optional extra  This descriptive analysis suggests that few of these issues are particularly important in influencing students' contributions to discussions. Although a few items had a median of three, only one had a mode above 1. Thus the only item that particularly stands out concerns the tutor's enthusiasm.
Other items seem to have had some small impact include students' priorities: that they felt they didn't have enough time and didn't think this would be particularly helpful, that this represented another avenue for questioning the tutor, and that contribution was assessed. The only noteworthy disincentive appears to be a general lack of time to participate. Rather than taking lack of time at face value, however, it may be more appropriate to interpret this as signifying that the non-compulsory nature of participation may have led students to allocate a low priority to participation (cf. Morris and others [16]).
Analytically, there seemed to be no difference in terms of attitudes between genders (although there was a slight trend for men to be more reluctant than women to share their resources: Mann Whitney U, n=60, p<0.076). However, students whose GPA on entry was below the class average felt more likely to contribute if they had the option of doing so anonymously (Mann Whitney U: n=60, p<0.042), and were more likely to feel that they had nothing to contribute to the discussion (Mann Whitney U: n=60, p<0.011). There was also a trend suggesting that students with a lower GPA than the class average were more likely to perceive the discussion forum as a 'bolt on' rather than an integral part of the course (Mann Whitney U:, n=60, p<0.061), suggesting that they were less clear about its pedagogic role.
Others indicators of students' attitudes were drawn from the correlation of students' self-reported number of postings on each of the four bulletin boards to which they had posting rights, and how helpful they felt that each board had been in terms of both improving their understanding and improving their scores. Perhaps unsurprisingly, there was a significant correlation with each of the sets of data between the extent to which students felt the board improved their understanding, and the extent to which it helped them improve their scores (Pearson: 1-tailed, n=60, p<0.01). Slightly less clear-cut, but perhaps more interesting, are the relationships between students' self-reported contribution to a board and its perceived usefulness. The following significant correlations emerged  For the bulletin board that forms the focus for this study, Biochemistry, there was no relationship between whether or not students contributed messages and any perceived improvement in understanding or scores. This provides some support for the claim that lurkers are not necessarily disadvantaged by their behavior. However, the picture seems markedly different for the other two courses, Human Anatomy and Lower Extremity Anatomy, raising the question of how use of these discussion areas differed. Understanding seemed to be linked to contribution for the Lower Extremity Anatomy courses, and scores appear to be linked to participation in Human Anatomy. This suggests that the influence here lay in the tutors' pedagogic use of the boards, rather than the technology, which remained the same).

B. Requests for further resources
In addition to the closed items on the questionnaires, students were invited to say what further resources might be useful to support their learning. Most responses concerned information management issues, such as: • Links to other sites on this topic (6) • Downloadable copies of slides and handouts (4) • Copies of past assessments (2) • Formative self-assessment exercises (4) • A course syllabus (2) • Pictures, illustrations and videos used in the course (6) • Information on administrative issues for the course (1) • A study guide (1) However, one student requested a social discussion forum for students on the course, and another took this opportunity to complain that the bulletin board was unusable because the volume of messages was overwhelming.

C. The influence of the tutor
It is interesting to consider the characteristics of each tutor and how the ALN was integrated into each course. The Biochemistry discussion was moderated by a tutor with previous e-moderating experience, considerable enthusiasm for the media and an appreciation of the learning benefits of ALNs. Within the Biochemistry course, continual reference was made to the discussion group throughout the program in class time, and students were encouraged to take an active/interactive role in their own learning. In addition, the class was directed to the asynchronous discussion forum in order to take advantage of learning resources such as websites posted by their peers. These resources were not available elsewhere and gave the discussion group some added value. It was hoped that this would stimulate student interest in the media. Toward the end of the Biochemistry program as interest in the discussion board was waning, the tutor suggested to the class that additional credit could be obtained for posting a paper summary to the discussion board in an effort to promote interaction with the medium.
Lower Extremity Anatomy was taught by an experienced traditional tutor with no previous experience of ALNs. This tutor was prepared to embrace the technology and was receptive to the advantages of the media. Given the intensity of the Podiatric program and the high proportion of 'in class' hours this tutor was prepared to utilize ALNs to facilitate 'anywhere-anytime' learning providing that it was student-led. The tutor made announcements regarding the existence of the asynchronous discussion forum in class time and informed the students of the need for them to take an active role in its use. The technology is in place and if the students want to use it to explore more in-depth aspects of the course outside of class time I am happy to respond, though these interactions should be initiated by the student. Human Anatomy was taught by a very experienced traditional tutor with no prior experience of ALNs.
He was very enthusiastic about the potential of the technology to enhance student-faculty interaction and referred to the existence of the discussion board repeatedly during the early stages of the course, occasionally posting messages during this period.
Two faculty in the department of Basic Medical Sciences did not utilize ALN, citing workload and contentment with operations currently in place as inhibiting factors. Shifter reports that non-participating faculty most commonly cite lack of technical support provided by the institution and concern about the quality of courses before faculty workload as inhibiting factors [17]. Interestingly both these faculty rated their enthusiasm for the technology as moderate, on a Likert scale from very high to very low.

IV. DISCUSSION
The findings summarised above suggest that the forum was not strongly interactive, in spite of the fact that all common recommendations for encouraging debate had been followed. Indeed, a high proportion of the postings were made by the tutor. This is far from a unique phenomenon.
The online CHAT was a strongly tutor-led discussion. From a total of 9831 words and 664 turns over eight analysed seminars the tutor produced 4487 words and 248 turns. This represents 46% of words and 37% of turns. One full-time International Student and two Home students accounted for another 3638 words and 284 turns between them -leaving just 17% of the word total (15% of turns) to the remaining students [7]. The only contextual element that appeared to influence engagement in any important way was the tutor. Comparisons across courses suggest that different pedagogic applications of this technology influenced the link between contribution to discussion areas and learning (Table 4). Although one possible interpretation is that the other boards were more effective in using contributions to foster learning, it is equally feasible and more credible to interpret this result as indicating that only the more able students contributed to the discussion boards on other courses, as they often do in the classroom, whereas a broader range of students contributed on the Biochemistry board. This suggests that the techniques used in the Biochemistry board were relatively effective in encouraging student participation, even though the students were not particularly aware of the influence of these techniques on their contributions (Table 3).
One distinctive feature of the pattern of postings was that students were 'playing the game' of assessment, making the posting that earned them marks but rarely contributing otherwise. The single biggest category of postings, the 'statements' from named students, was almost exclusively comprised of the project descriptions requested by the tutor as a for-credit part of the assessment process. This reinforces the observations of Jones [12], where assessment led to students consciously engaging in behaviors designed to maximise their performance. It also resembles the finding of Morris and others [16] that "most coursecontent-related conference postings were assignment focused." The other distinctive feature of this course, identified through a discussion of the role of the tutors and the feedback of the students, was the enthusiasm and expertise of the tutor. This suggests that it may be this personal, affective element of the students' context that fostered engagement. Mason's work supports this observation, reporting that the commitment, passion, involvement and absorption of the tutor in the online discussion is even more import than e-moderating skill in influencing the success of the conference [3]. Similarly, Kerr notes that a lack of adequate leadership is a major factor in the failure of such conferences, since a clear agenda for participants to follow and goals to aim for are needed if anything is to happen [18].
Another important feature, not directly highlighted by the findings, concerns the introduction of this technology. All students involved in this study had already completed one semester of medical education. As found in the very similar context of the study by Morris and others [16], the addition of new media to the course confounds students' expectations, and can lead to low levels of use, particularly if the students' first engagement with this new technology is frustrating [19].

V. CONCLUSION
The study described in this paper has demonstrated the feasibility of supporting courses in medical education using ALNs. However, more importantly, the study has called into question a number of assumptions that are frequently made about the reasons why such use of ALNs are successful.
The use of assessment to encourage student discourse appears to be only a superficial success. The findings here that students perform to criteria without necessarily engaging in worthwhile dialogue are supported by others [12]. Thus unless more sophisticated assessment criteria can be developed, it seems likely that giving credit for postings will change behavior without necessarily improving learning. There was even less evidence that the other pedagogic techniques used here benefit learners. As Jones says, "collaboration is shown to be in some sense the problem for networked learning not an outcome of the new technology or its associated pedagogy" (emphasis added) [20]. While many papers draw conclusions about techniques that influence students' engagement, the affective role of the tutor has remained relatively unexamined. This study suggests that whether or not the recommended techniques influence learning, the tutor has an important role to play. Attention to techniques alone is insufficient; tutors are not some confounding variable modifying the effectiveness of particular pedagogic approaches, but may instead be the root cause of engagement in discussions.
Thus, ironically, attention to both technology and pedagogy in studies of this type may be misplaced, or at the least, inadequate. What is required is an increased focus on the affective role of the tutor, in recognition of the realisation in the wider field of research into Higher Education that their enthusiasm stimulates students, communicating a love of the subject, medium and of learning [21]. and thank Jared Remmers, CCPM class of 2003 for his organizational skills and dedication to research at CCPM.