Implementing a Large On-Campus ALN : Faculty Perspective

This case study describes the implementation and continued operation of a large on-campus ALN for a 500-student course in introductory physics. The ALN was used to modify and complement the original course and thus represents an evolution rather than a revolution. A highly positive impact on student success rates was achieved and continues. Factors that increased faculty satisfaction and instances of dissatisfaction are presented. The potential increase in the latter with technology is of some concern.


I. INTRODUCTION
Approximately 500 science and engineering students enter the calculus-based Introductory Physics two-semester course, PHY183-184.For most of the students in the class, this course is a requirement each semester.The course is part of the established curriculum.Its goals and standards are well defined by the textbooks commonly used across various institutions.In its present form, it is an attempt to combine the best features of the face-to-face interactions and lectures with the use of network tools for anytime/anywhere interaction.The goal has been to establish and maintain high standards while providing students with the means and opportunity to succeed.This ALN was initiated in fall 1995 and has been continued since that time [1].

II. RATIONALE
In fall 1995, we implemented our first ALN with support of the Sloan Foundation.This was in large part as a consequence of discussion with colleagues at UIUC who had embraced the ALN concept and already had encouraging initial results [2].Our ALN followed three years of computerized assignments in introductory physics and chemistry courses using the Internet.This showed the added dimension that networked technology could add to a course.The prospect of improved student performance and satisfaction was also a driving factor, especially as we had observed that students were spending considerably more time in the course.For the first time, we were seeing student effort at a level that justified our own instructing efforts.This was indeed a source of satisfaction!Other members of the department taught this course during the past two years (Table 1).While using the ALN was not required, they were encouraged to do so to take advantage of the savings that were generated by the reduced teaching staff used in the ALN, i.e., about two-thirds of that in the traditional course 2 .The comments of these instructors are included in this case study.

A. Technology and Infrastructure
The principal ALN tool used was CAPA, a computer system developed at MSU over the past seven years.In the initial experience with its use, a 90-student science class, student reaction was very positive and has since been replicated in numerous other disciplines and at other institutions [3][4][5][6][7][8][9].This integrated software system has been used to ■ Prepare, deliver, and grade personalized homework, quizzes and examinations.
■ Provide feedback to students and instructors.
■ Communicate with students in a class and provide a discussion forum for students.
■ Provide links for student help via the Internet.
■ Facilitate course management -Table 2 summarizes some of the CAPA features for the instructor.The system was originally developed for Physics and Chemistry courses [3,5] and was used to develop a broad variety of conceptual questions adapted to the technology.Students are given several tries to get correct solutions and are given full credit when they succeed before the due date.

Course management and statistical analysis tools.
Can send semi-personalized E-mail to students based on performance Allows instructor to grade subjective answers such as essays Provides answers for individual student's assignments for hand-grading

An additional grading tool that supplements self-grading by students
Allows a problem to be excused for an individual student, section, or class While hints are available when an incorrect answer is given, the system does not attempt to give the students feedback on the particular error made (other than a formatting error).We believe that a key aspect of problem solving lies in the ability to both detect and correct one's mistake.
The strong emphasis on conceptual problems within CAPA makes it is a useful tool for many fields [7,9,10].Figure 1 shows an example of a conceptual problem.It clearly goes beyond a traditional one-out-of-five multiple-choice question.To solve this problem, students need a reasonable understanding of Archimede's Principle.
[1pt] A fisherman and his young daughter are in a boat on a small pond.Both are wearing life jackets.The daughter is holding a large floating helium filled balloon by a string.

Consider each action below independently, and indicate whether the level of the water in the pond R-Rises, F-Falls, S-Stays the Same, C-Can't tell. (If in the first the level Rises, and in the second it Falls, and for the rest One Cannot Tell, enter RFCCC.)
A) The daughter pops the balloon.B) The fisherman knocks the tackle box overboard and it sinks to the bottom.C) The fisherman lowers himself in the water and floats on his back.D) The fisherman fills a glass with water from the pond and drinks it.E) The daughter gets in the water, looses her grip on the string, letting the balloon escape upwards.

Figure 1. Example of a Conceptual Question in CAPA
The hint will read: "Think Archimede's Principle.How does the volume of fluid displaced by a body that `floats' differ from that for a body that sinks?"An additional strength of CAPA is the variations of a problem among students.A simple example is shown in Figure 2. Two versions of the same problem are shown for different students.This encourages collaboration without simply copying the solutions.Again, each individual student has to understand his own problem.It is also possible to give the students different selections; however, one should be careful that all the same concepts are presented to all students.

[1pt]
John is listening to a horn.He knows the frequency of the horn is 300 Hz when both he and the horn are at rest.If he hears a pitch of 270 Hz, there are clearly several possibilities.

[1pt]
Paul is listening to a horn.He knows the frequency of the horn is 300 Hz when both he and the horn are at rest.If he hears a pitch of 330 Hz, there are clearly several possibilities.
(Give ALL correct answers, i.e., B, AC, BCD...) A) Both can be moving and have the same speed.B) Both can be moving, in opposite directions.C) The distance between John and the horn is decreasing with time.D) John is moving away from the horn at rest.E) Both can be moving and have different speeds.F) Both cannot be moving in the same direction.
A) Both can be moving, in opposite directions.B) Both can be moving and have the same speed.C) Both can be moving and have different speeds.D) Paul is moving towards the horn at rest.E) The distance between Paul and the horn is increasing with time.F) Both cannot be moving in the same direction.Conceptual problems are an important part for understanding.There is a large correlation between understanding the concepts and the ability to solve numerical questions as shown in Figure 3.The most recent addition is a direct link to a discussion forum implemented in CAPA when accessing from the Web.This allows each student to participate directly in a discussion with other students and/or the teaching assistant (TA) while working on a specific problem.This feature simplifies the use of discussion groups significantly because the additional log-in and password required for an external discussion forum is not necessary.In this moderated discussion forum, the TAs make sure that the postings are hints and not just the posting of formulas.This control mechanism does not exist for Websites initiated by students that have recently appeared.On these sites solutions are posted and even problems having typical randomized variables can have their solutions posted.Fortunately, these student endeavors can be overcome.Figure 4 shows an example we have used.As can be seen it is virtually impossible to communicate the solution of this problem over the Web without discussing the physics involved (Kirchoff's Rules).That is exactly the purpose of collaborationexplaining and understanding the problems.It becomes easier to learn how to do the problem than to subvert the system.

B. Content Delivery
Table 3 shows the various components of the course.As in most physics courses, demonstrations are a large component of lecture time.Traditional lecture time is, however, significantly reduced, allowing for large segments of time on interactive lecture exercises and unannounced short quizzes.These quizzes have had a positive impact in improving class attendance even though they serve to assign only a small proportion of the student's grade (5%).The quizzes are also useful in identifying misconceptions very early.
Homework assignments are personalized.The conceptual problems are designed to encourage collaboration among students as they can benefit from the additional practice of working on somewhat different versions of the same problems.Numerical story problems have variables that inhibit rote copying among students.Recently we have added new techniques that have strongly inhibited the sharing of formulas where one can just plug in one's variables and get the correct answer without really understanding the problems.Each weekly assignment has a firm due date.This insures students do not fall behind.Students are given full credit if they get the correct answers before the due date.If a solution entered is incorrect, the student may work to find the errors.A number of attempts are allowed to obtain the solution.Thus, most students are able to receive very good grades on the assignments, and this has proved to be highly motivating.Note that these high grades do not lead to grade inflation, as a higher absolute scale is used to assign the course grade.Collaboration is also encouraged by a small component of teamwork in assignments.For example, groups of up to five students working together can submit a subjective essay discussing the observation of an experiment shown by video during class.This also has a big efficiency factor for the instructor grading these essays.
Assignments have a significant component of challenging problems.Students can obtain help at any time via the discussion forum established for the class.Posted questions are sure to be addressed within 24 hours or less by TAs or other students.In addition, help is available at scheduled periods throughout the week in a Physics Learning Center.There teaching assistants, who are assigned to the class and who have themselves solved their own personalized assignments, use the Socratic method in helping students with any difficulties.Students needing help can be there several hours while those who do not feel such a need are not required to be there at all.This face-toface help is an important part of the class, especially for less prepared students.It is also helpful to the instructor as it provides complimentary feedback to the detailed on-line feedback available from the student performance as recorded by the system.The Learning Center is also the way we have countered the impersonal nature of instruction in this large course.It provides many opportunities for one-on-one interactions between students and teaching staff.
A concept test given during the first week and again near the end of the class helps compare the class to others at similar institutions and represents one of the measures of learning.Mid-term and final exams are the main assessment tools.In the case of the mid-terms, we allow students to earn partial credit by correcting any part of the exams on which they have not done well.
Following the proctored examinations, students can pick up a copy of a different version of the same exam.They then can solve the problems in that version within the next three days and enter answers via the Internet.They can consult with fellow students, the teaching staff, etc. to get help.By this process, the mid-term grade is the original grade plus 30% of the difference between the corrections and original exams, i.e., they recover "30 cents on the dollar" of every missed point.More than 99% of students avail themselves of this correction opportunity.It is a very popular option!For the instructor, it represents an effective and efficient way to encourage students to review all the material on the exam and improve their understanding.(A common comment by students finding out how to solve a problem missed on the exam is, "I should have gotten it!It wasn't that hard.") The 99% of the students who work on the corrections actually solve all exam questions.In contrast, in traditional exams the students typically do not even look at their wrong answers and do not try to understand what they did wrong.This extra effort on the students' part comes at almost no extra effort by the instructor.

A. Effectiveness
Before we discuss faculty satisfaction, we need to mention briefly the effectiveness results obtained from our oncampus ALN approach.These can be found in the Journal of Engineering Education [1], where we demonstrated a strong, positive impact on student success in large classes while maintaining high standards.In retrospect, the improvement in student achievement observed should not have been surprising as our approach has been to retain the best established practices of more traditional teaching and to use ALN to fix the weak areas where it clearly can have a major impact.
The important aspects of using the technology which appear to have a significant positive impact on student achievement include ■ Interaction between the students and the computer using materials and problems well adapted to the technology enables students to receive instant feedback [11,12] on their understanding of concepts and ability to carry out calculations properly.Concurrently, it provides access to specific help provided by an instructor or provided via links.
■ Interaction between the instructor and the computer provides the instructor with on-line feedback on student misconceptions and misunderstanding so that they can be addressed in a timely manner, usually before the work is due.It provides early, comprehensive information about individual students who are having difficulties.This information is essential to properly advising these students.
■ Asynchronous interactions among students and between students and instructors via the network provide the opportunity for questions, answers, discussions, and elucidation of difficult concepts within the context of anytime/anywhere.
In addition, the efficiencies generated by the use of technology allow us to devote increased teaching staff time to Socratic one-on-one interactions with students.

B. Saving Paper
One issue in implementing the technology for large on-campus ALN course involves the printing of personalized assignments.Since the assigned work is on the Web, should students be given printed copies?Our reason for printing and handing out assignments has been to promote an environment where students collaborate and discuss their work.Such collaboration has been shown to have positive impact on performance [13].Would such collaborations be less likely if they had to work at individual computers?We have recently collected some data on the printing issue.In fall 1998, assignments for a large student course (not using CAPA) were provided on the Web.When polled, most of those students said they printed assignments from the Web all or most of the time.
We then polled the students in PHY183 (480 students using CAPA) during the fourth week of the semester to see if they would agree to have the following week's assignment not printed.The vote was overwhelmingly against, 420 to 6.In addition, 89 students commented by E-mailone was willing, 88 were not.The reasons given in those 88 Emails are shown in Table 4.

Why students want printed assignments
Harder to work in front of screen

56%
Printed HW more accessible

43%
Trouble printing from the Web, and excellent quality of LaTeX printout

27%
Web access required We believe that in our situation, the small effort by the instructor to prepare printed worksheets is well repaid by the large saving in student time and increased student interaction level.Note that because of a highly efficient format, our printing of the assignments prepared with Latex requires from four to nine times less paper than when students print from the Web.

C. Faculty Satisfaction
To put the faculty satisfaction issues in perspective, we have interviewed faculty, including some who have not used ALN in their disciplines and looked at previous studies of issues that affect faculty satisfaction [14][15][16][17][18][19].Faculty satisfaction is complex.The principal factors, which emerge from the literature, from interviews with our colleagues in this study, and from our own experience, include collegiality, workload, and autonomy.An interesting observation concerns the role conflict that occurs at the intersection between faculty and administrative domains of responsibility.While it does not appear to affect general faculty satisfaction, it can be a source of disaffection and dissatisfaction.
Our experience has been that the implementation of ALN technology on a large scale in teaching has greatly increased the domain where administrative and academic responsibility and control intersect.Thus, it is not surprising that we have experienced numerous situations that engendered faculty dissatisfaction, ranging from not so important to instances that, in our perception, are critical factors in how we want to do our task.In this area of collective decision-making and responsibility, we have encountered a spectrum of administrative attitudes across the administrative ladder.In the four situations briefly described below, we should keep in mind that the descriptions are from the faculty and that the perception of the administrator(s) involved may be considerably at odds.The following paraphrase professors either using or using and developing ALN: ■ Case A -While teaching a large (700 students) introductory physics course, I came across a software program that displayed physics demonstrations.This software was priced around $150.I had already been displaying demonstrations on a screen and thought this software would fit nicely with the existing format of the class.In response to my E-mail requesting authorization to purchase that software, the department chair responded that I should E-mail the person in charge of the academic affairs committee, which I did.This person responded that I should first get a demonstration version of the program, which I did.I received the demonstration version of that program but that version was inadequate.By this time, it was near the end of the term.I was frustrated with the process and discontinued my attempt to incorporate this program into my class.
■ Case B -In implementing the large ALN on campus, I decided there was a need to provide students an opportunity for a face-to-face interaction and help with teaching staff above and beyond what was provided through the network.A learning center was established with furnishings consisting of tables and chairs and computers obtained from salvage.The computer needs in this spartan environment were quite modest, vt100 terminal emulation, as our ALN predated the use of the sophisticated Web browsers that are ubiquitously in use today.
As the use of the ALN concept spread to a greater number of students, this initial setup soon became insufficient for the large demand and new capabilities.Requests that the area be upgraded and improved in several aspects eventually received administrative approval, at which time I informed my students to bear with us for a bit as significant changes were soon forthcoming in improving that component of the learning environment.These changes were then canceled and not implemented until more that a year later.I felt angry and frustrated.
■ Case C -Discussion with the Dean and Chair established the need for centralized support for faculty using the technology.The yearly combination of contributions from the two most highly involved departments and the College with miscellaneous funds (education research grant, etc.) has provided a salary for a coordinator to support faculty across campus in the use of the CAPA as ALN tool.Administration has clearly expressed support of a more permanent arrangement, which would eliminate my task to see that the support for the position is there each year.In spite of the 6000+ students involved each semester, such an arrangement is still not in place five years after the position was first filled.
■ Case D -When a key developer of the CAPA system left the University, I made repeated requests for support to continue development and essentially met no action, either positive or negative.I then addressed the request directly to the highest administrative level.This action triggered a significant dissatisfaction at lower administrative levels.Still, the outcome was that the request was fulfilled, with the University continuing to support the system development.
For the present survey we explicitly interviewed six physics professor who taught with CAPA recently.We also have close contact with most of the instructors using CAPA for about 20 courses in several disciplines on campus.The four cases above represent the major conflicts brought to our attention.Note that they are one-sided views in areas of shared responsibility and could be described in quite different terms from an administrative perspective.In the implementation of ALN technology, such areas have grown significantly, and thus, increase potential conflicts as faculty and administrators carry out their tasks.
With the increased use of technology in education, administrators have an increasing and important role in an area where they have limited knowledge and in which faculty also are not often experts.A quote (with a large dose of sarcasm) from a colleague asked to comment on sources of dissatisfaction in his work, "... too much administrative interference, decisions based on ignorance."We do not want to leave the impression that conflicts dominate our interaction with administrators.On balance, they have been facilitators and helped to establish the conditions in which we have obtained highly positive and encouraging results [1].
The level of satisfaction with the on-campus ALN implementation is high across many disciplines and faculty who have implemented all or part of its functionality.This satisfaction comes in spite of the universal agreement among faculty that work is increased, especially initially.Technical support is rated good to "… wonderful."Positive interactions with satisfied students, by far the majority, is a big factor, as is the interactions with colleagues doing ALN with whom one can share a remarkable variety of wonderful stories.The on-campus aspect has strong appeal.One faculty who volunteered that he is now ".. a convert to this technology" added that he liked that he was "still teaching the normal way."Another commented, "This was the first time I had the ability to really see how students were doing in such a large course and could review that information before meeting with them." There has been a redistribution of responsibility in the large courses.There is some loss of faculty control as courses depend more on the support of system administrators and on the proper functioning of the technology infrastructure.More of the course administration and management has become part of the lecturer's work.Instructors now have detailed knowledge of student performance, and so do the students.There is far greater interaction with students, via E-mail in particular, which is yet another factor increasing the time spent on a course.Those students who are having a difficult time, or who believe that getting some work excused is equivalent to having done the work, take up a far greater proportion of the instructor's time.Some students seek assistance when it is clear that very little studying has occurred.This additional work is a source of dissatisfaction for many faculty.Others who perceive the added work are not interested in adopting the new technology.
For faculty excited by the new opportunities, there have been many rewards.These include ■ Increased collegiality with colleagues in other departments and disciplines.
■ A perception that they can influence outcome.
■ Improved relations with students who are benefiting and who view the instructor as mentor rather than judge.
■ Positive feedback from graduate assistants whose work has been moved from grading and record keeping to Socratic interactions with students.

IV. SUMMARY
We believe that at MSU, ALN can and soon will be a significant part of the educational experience for a majority of students.This will be helped considerably if, as the number and variety of more sophisticated technical tools become available, faculty are assisted in becoming skilled in their use and the increased workload is kept in check.Broad implementation will also be assisted if faculty and administrators develop means to deal with the increasing number of conflict situations where their functions overlap.Time allocation shifted very significantly from mostly lecture to less than half lecture.More time was spent with quizzes, exercises, and on-line discussion and less time with recitation.Due to the automation provided by CAPA, exam-grading time was reduced to zero, a significant factor in cost reduction.

Yihjia Tsai is currently an
The bottom line is that student learning improved.The paradigm of retaking tests and reworking homework appears to work well.However, it is a bit shocking that the improvements in what the students could do did not improve their grades, since the absolute scales used to assign grades were raised.So, even though the students learned more, they did not get better grades, it seems.
The investigators found that students printed out assignments rather than use the assignments on the screen.Students preferred printed material to on-screen materials.The survey conducted indicated that most found it hard to work in front of the screen.It appears that we need to know why this is and what human factors need to be changed in order to provide a paperless environment.
The four cases reporting on faculty satisfaction focus primarily on dissatisfaction.The authors indicate that the viewpoints presented are largely one-sided.It is striking, however, that things that have been accomplished appear to have been done in spite of rather than because of administrative input.If, in fact, faculty satisfaction is so strongly linked to administrative meddling, what lessons might we learn from this paper's observations?How can progress in ALN be better linked to administrative understanding of ALN?
Several issues are important: ■ Learning Center Impact -The addition of a face-to-face learning center was one of the innovations described.One wonders how much the learning center impacted the satisfaction of the students and faculty.
If the learning center had not been implemented, what would have happened?Would any of the findings be different?
■ Sustaining Tools -CAPA, as with any homegrown system, has problems in sustainability in the sense that it is not a commercial tool.What indeed, happens to a tool created at a university when the creators leave?
■ Grade Inflation -One of the most disturbing parts of this article is the assertion that grade inflation does not occur because the grading scale changes as the students do better (always hold out the carrot and keep running).It is worth considering the impact of going to an absolute-grading scale.
■ Administrative/Faculty Conflicts -It appears due to the strongly negative comments about administrators, that training of administrative officers should be a priority for future ALN work.Perhaps a workshop on the impact of ALN should be considered.
The cost reduction and ease of use of course management systems for implementing ALN appear to have significant meaning for the teaching community.The lesson learned is that faculty are more satisfied when it is easy to move to an on-line environment.This finding translates into the idea that more fully developed course management packages result in more satisfied faculty, i.e., faculty can do things that they want to do more easily and quickly.
Students learn better in an on-line environment that includes immediate feedback.However, we imagine that this satisfaction will not persist if the faculty make it harder and harder for students to improve grades.Criteria-based grading would be an answer to this problem.
As for the administrative problems, it seems that the Sloan Foundation might consider creating a workshop for administrators to bring them up to speed on ALN.An on-line workshop and/or face-to-face workshop could be created.

Figure 2 .
Figure 2. Two Different Versions of the Same Problem The selections are automatically randomized.

Figure 3 .
Figure 3. Correlation Between Conceptual and Numerical Problems

Figure 4 .
Figure 4.Additional Randomization of Labels with CAPA
Prepares materials in three formats: ASCII, HTML, LATEX Each student receives unique questions and problems Over 170 pre-coded templates to facilitate creation of questions Allows printing of text and graphics in a compact, efficient manner Due dates can be set for individual sections independently Includes a timed entry option for use with take-home quizzes and exams A simple transformation allows conversion from homework style to exam style Provides the range of answers for a question across all students in a class

Table 4 . E-mail Responses of 88 Students
Assistant Professor of Computer Science and Information Engineering at Tamkang University in Taiwan, and has been a visiting scholar at MSU for several periods from 1998-2000.He received a B.S. in Mechanical Engineering from the Taiwan National University in 1985.Following a period of work in industry, he earned a M.S. (1995) and Ph.D. (1997) from Michigan State University.He was a member of the initial CAPA development team, was the principal programmer from its inception until 1998, and is actively participating in its current development.His research interests include fundamental aspects of communications in computers, as well as the use of computer technology in education.He is also co-awardee of the ASEE Benjamin J. Dasher Award ('98) and the William Elgin Wickenden Award ('99).Contact: Computer Science and Information Engineering, Tamkang University, Taiwan; E-mail: tsai@cs.tku.edu.tw.The teaching staff at MSU was reduced to two-thirds of that in traditional courses, largely because of the automated grading with CAPA.It is important that the authors observe that the number of teaching assistants needed for grading was reduced to zero.This finding has important implications for cost effectiveness of ALN.Another point is that the authors indicate in another on-line paper that the favorite aspect of ALN courses taught using CAPA is the use of automated grading and immediate feedback.Thus, the use of this system resulted in cost reduction coupled with improved student satisfactiona very good thing, indeed!One-on-one training of faculty resulted in rapid productivity.The Sloan grant permitted rapid training of faculty.The one-on-one model was shown to be quite good; however, is this model scalable?Some course management systems require almost no training, others significant amounts.Was CAPA unusual in the amount of training required compared to other systems?
John Bourne is Professor of Electrical Engineering, Computer Engineering and Biomedical Engineering at Vanderbilt University.He is Editor of the Journal of Asynchronous Learning Networks (JALN) and runs the ALN Web.He has been on the faculty at Vanderbilt since 1969 and is a Fellow of the IEEE and a Fellow of the AIMBE.Contact: John R. Bourne, The Center for Asynchronous Learning Networks, Vanderbilt University, Box 1595, Station B, Nashville, Tennessee 37235; Telephone: 615-322-2118; Fax: 615-343-6449; E-mail: john.bourne@vanderbilt.edu.