Measuring the Importance of Collaborative Learning for the Effectiveness of ALN: A Multi-Measure, Multi-Method Approach

Starr Roxanne Hiltz, Nancy Coppola, Naomi Rotter, Murray Turoff, Raquel Benbunan-Fich


Are there any differences in outcomes between traditional classroom-based university courses and courses delivered via ALN, which feature extensive on-line interaction among students? Under what conditions are ALN courses most effective? What can be done to improve the publishability of ALN evaluations, and counter the attacks of critics?
After providing background on the New Jersey Institute of Technology (NJIT) Virtual Classroom (VC) projects, this paper describes three studies that address the issue of the importance of collaborative learning strategies to the success of ALN for students. A three-year longitudinal field study of 26 courses that are part of an undergraduate degree in Information Systems compared the process and outcomes of learning using an on-line anytime/anywhere environment to those for comparison sections taught in the traditional classroom. An embedded field experiment looked at the separate and joint effects of working on-line versus in the classroom and of working individually versus in groups. Semi-structured interviews with experienced ALN faculty probed their pedagogy and their perceptions of whether or not students learned, on the average, more, less, or about the same as in their traditional sections. The results support the premise that when students are actively involved in collaborative (group) learning on-line, the outcomes can be as good as or better than those for traditional classes, but when individuals are simply receiving posted material and sending back individual work, the results are poorer than in traditional


Collaborative Learning,ALN,Effectiveness

Full Text:



Alavi, M. Computer-mediated collaborative learning: An empirical evaluation. MIS Quarterly, Vol. 18, No. 2, pp. 150-174, June, 1994.

Benbunan-Fich, R. Effects of computer-mediated communication systems on learning. Performance and Satisfaction: A Comparison of Groups and Individuals Solving Ethical Case Scenarios. Ph.D. diss., Rutgers University/NJIT Joint Program in the Management of Computer Systems, Newark NJ, 1997.

Benbunan-Fich, R. Guidelines for using case scenarios to teach computer ethics. ACM SIGCAS Bulletin, pp. 20-24, September, 1998.

Benbunan-Fich, R., and Hiltz, S. R. Impacts of asynchronous learning networks on individual and group problem solving: A field experiment. J. Group Decision and Negotiations, 1999.

Bouton, C., and Garth, R. Y. Learning in Groups, San Francisco: Jossey-Bass, Inc., 1983.

Brown, G., and Wack, M. The difference frenzy and matching buckshot with buckshot. Technology Source, Horizon, 1999.

Coppola, N., Hiltz, S. R., and Rotter, N. Becoming a virtual professor: Preliminary results of semi-structured interviews. Presentation at the Fourth International Conference on Asynchronous Learning Networks, New York, November, 1998.

Coppola, N., Hiltz, S. R., and Rotter, N. Becoming a virtual professor: Pedagogical changes and ALN. Presentation scheduled for the Fifth International Conference on Asynchronous Learning Networks, University of Maryland, October, 1999.

Daft, R. L., and Lengel, R. H. Organizational information requirements, media richness, and structural design. Management Science, Vol. 32, No. 5, pp. 554-571, 1986.

Dennis, A. R., and Valacich, J. S. Rethinking media richness: Towards a theory of media synchronicity. Proceedings, 32nd Hawaii International Conference on System Sciences, 1999.

Dennis, A., and Valacich, J. Doing experimental research on collaboration technology. Tutorial presented at the Hawaii International Conference on System Sciences, Maui, HI, January, 1999.

Dillenbourg, P., and Schneider, D. Collaborative learning in the Internet. Proceedings, Fourth Int. Conference on Computer Assisted Instruction, Taiwan, S10-6 to S10-13, 1994.

Duffy, T. M., and Cunningham, D. J. 1996. Constructivism: Implications for the design and delivery of instruction. Handbook of Research for Educational Communications and Technology, New York: Macmillan.

Glasser, R., and Bassok, M. Learning theory and the study of instruction. Annual Review of Psychology, No. 40, pp. 631-666, 1989.

Harasim, L. Ed. On-Line Education: Perspectives on a New Medium, New York: Praeger/Greenwood, 1990.

Harasim, L., Hiltz, S. R., Teles, L., and Turoff, M. Learning Networks: A Field Guide to Teaching and Learning On-line, Cambridge MA: MIT Press, 1995.

Hill, G. W. Group versus individual performance: Are N+1 heads better than one? Psychological Bulletin, Vol. 91, No. 3, pp. 517-539, 1982.

Hiltz, S. R. The virtual classroom: Using computer-mediated communication for university teaching. Journal of Communication, Vol. 36, No. 2, pp. 95-104, 1986.

Hiltz, S.R. Productivity enhancement from computer-mediated communication: A systems contingency approach. Communications of the ACM, Vol. 31, No. 12, pp. 1438-1454, December, 1988.

Hiltz, S. R. The Virtual Classroom: Learning Without Limits via Computer Networks, Norwood NJ: Ablex Publishing Corp., Human-Computer Interaction Series, 1994.

Hiltz, S. R., and Benbunan-Fich, R. The Importance of Collaborative Learning in Asynchronous Learning Networks. Manuscript submitted for publication, 1999.

Hiltz, S. R., and Turoff, M. The Network Nation: Human Communication via Computer, Revised edition, Cambridge MA: MIT Press, 1978/1993.

Johnson, D. W. Student-student interaction: The neglected variable in education. Educational Research, Vol. 10, No. 1, pp. 5-10, 1981.

Leidner, D., and Jarvenpaa, S. The use of information technology to enhance management school education: A theoretical view. MIS Quarterly, pp. 265-291, September, 1995.

Leuthold, J. Is computer-based learning right for everyone? IEEE Computer Society Press, Proc. 32nd Hawaii Int. Conf. On System Sciences, 1999, CD ROM.

McGrath, J. E., and Hollingshead, A. B. Groups Interacting with Technology, Thousand Oaks, CA: Sage Publications, 1994.

Mead, G. H. Mind, Self and Society, Chicago, University of Chicago Press, 1934.

Nunamaker, J., Dennis, A., Valacich, J., Vogel, D., and George, J. Electronic meeting systems to support group work. Communications of the ACM, Vol. 34, No. 7, pp. 41-61, 1991.

Ocker, R., Hiltz, S. R., Turoff, M., and Fjermestad, J. The effects of distributed group support and process structuring on software requirements development teams: results on creativity and quality. Journal of Management Information Systems, Vol. 12, No. 3, pp. 127-153, Winter, 1995.

Peek, L. E., Peek, G. S., and Horras, M. Enhancing Arthur Andersen business ethics vignettes: Group discussions using cooperative/collaborative learning techniques. Journal of Business Ethics, Vol. 13, pp. 189-196, 1994.

Phipps, R., and Merisotis, J. 1999. What’s the difference? A review of contemporary research on the effectiveness of distance learning in higher education. A report from the Institute for Higher Education Policy.

Poole, M. S., and DeSanctis, G. Understanding the use of group decision support systems: The theory of adaptive structuration, in Fulk, J. and Steinfield, C. (Eds.), Organizations and Communication Technology, Sage, Newbury Park, CA, 1990.

Rice, R. E. The New Media, Beverly Hills, Sage, 1984.

Rice, R. E. Task analyzability, use of new media, and effectiveness; A multi-site exploration of media richness. Organization Science, Vol. 3, No. 4, pp. 475-500, November, 1992.

Steiner, I. Group Process and Productivity, New York, Academic Press, 1972.

Turoff, M., and Hiltz, S. R. Computer support for group versus individual decisions. IEEE Transactions on Communications, Vol. 30, No. 1, pp. 82-91, January, 1982.

Webb, N. M. Student interaction and learning in small groups. Review of Educational Research, Vol. 52, No. 3, pp. 421-445, 1982.

Webster, J., and Hackley, P. Teaching effectiveness in technology-mediated distance learning. Academy of Management Journal, Vol. 40, No. 6, pp. 1282-1309, 1997.