Quality Assurance

This paper examines the social cost of ALN. This social cost is considered from several vantage points. First, student time is identified as the primary input in instruction. This time investment is not forthcoming automatically - several incentive problems must be resolved to elicit it. The resolution of these incentive problems entails additional cost. Second, the relationship between ALN and economies of scale in instruction is discussed. One channel for increasing returns is the better matching of instructor attributes with the demands of the course. This happens because the matching occurs in a global "market" rather than a market internal to the educational institution. Third, since distance ALN requires no brick and mortar investment, the paper considers how institutional commitment to quality instruction can be had without the brick and mortar. If there are institutional-specific investments associated with learning how to run a distance-ALN program, then institutional reputation is the likely mechanism. Otherwise, the labor market will require external verification, such as having students in ALN degree programs pass a series of standardized exams, to value such an ALN degree.


I. INTRODUCTION
The emergence of ALN has provided much fodder for futurists.One stark view was recently articulated by Dr. Ralph Gomory, President of the Alfred P. Sloan Foundation, at the Second International ALN Conference held in New York City.In his keynote address, Dr. Gomory likened the potential impact of ALN to Edison's invention of the lightbulb, the generation of cheap electricity engendered by that invention, and the revolutionary impact that this cheap electricity had on the quality of life thereafter.While it may not be astute for a Sloan grantee to take issue with Dr. Gomory, I'd nonetheless like to venture an alternative view in this essay.
At the heart of my argument is the belief that, when viewed from an economics perspective, education is fundamentally different from a commodity such as electricity.With commodities, it is a safe practice to distinguish the consumers from the producers, associating the benefits from consumption with the former group and the costs of production with the latter.With education, however, the student is both the primary consumer and the primary input supplier, the input being the time the student spends in learning 1 .The cost of the student's time is often ignored in thinking about the costs of education, because there is no market transaction involved.Nonetheless, there is indeed a social cost to this time: the time could be allocated to some other valuable use.
For the sake of illustration, I present a simple calculation to impute this cost on a per student basis for my Economics 300 course.This is an undergraduate, 3-credit hour course.A "normal" load for students is 15 credits per semester.The semester is 15 weeks long and I will use the recommended out-of-class time requirement of two hours per hour of in-class meeting time.This amounts to 9 hours per week for 15 weeks.The "art" in imputing these costs is coming up with a suitable wage.I will use a conservative figure, $6/hour.This is what I pay my undergraduate TAs. 2 With these assumptions, we get a cost of $810 for my course and a cost of $4050 per semester for a student taking a normal load.
It is not my goal to defend the precision of these estimates, but I do think it important to consider calculations of this sort when thinking about the costs of education.This is all the more so when considering adult learners who quite likely have a much higher opportunity cost of time than my undergrads.The message should be clear.Education is an expensive activity, even before accounting for those costs that tuition is supposed to cover.For a given quality of education, the total cost, inclusive of these imputed time costs, should be minimized.It is not obvious, however, indeed as I will argue below it is likely not true, that full cost minimization necessitates minimizing the expenditure on those inputs that are complementary to the student's time.
This essay is primarily devoted to making this argument in more detail.In the process we will encounter a host of issues that ALN programs must confront, particularly those programs where the students never, or only rarely, come to campus.We will also consider some arguments on the demand side for why the quality of education is likely to ratchet upwards, with a concomitant impact on the cost of education.
In the next section we confront the student time allocation problem head on.Section III takes on the economies-of-scale/appropriate-input-mix issue.Section IV considers possible mechanisms for assuring institutional commitment to a distance ALN program.The focus is on how the market gets to know the quality of such a program.Quality assurance is itself a costly activity.A brief conclusion is offered in section V. Throughout it will be useful to keep the following question in mind.Is the "right" ALN approach one with self-directed learning, using electronic materials prepared by the instructor and/or educational institution; an instructor-mediated approach, where the teacher spends much time interacting with the students; or some hybrid?Resolving this question is crucial for imputing the cost of an ALN-degree program.
One caveat should be added here.The focus in this essay is on programs that lead to degrees (or certificates) which have value in the labor market.It should be readily apparent that there already is a substantial amount of the self-directed type of ALN but which does not lead to a degree and which is not mediated by an institution of higher education--the transaction occurs directly between the publisher and the user of the software.It seems a safe prediction that more and more of this type of learning will occur in the future.But in this essay it will be ignored.Certification of the student is a crucial component to the ALNs which are the focus here.

II. STUDENT INCENTIVE ISSUES
This section considers how students are induced to make the requisite time investment.It might appear from a straightforward cost-benefit calculation that if there is an anticipated net benefit from pursuing a program of study then the time investment will be forthcoming automatically.This, however, is overly simplistic.The investment is not one-shot; it occurs piecemeal over an extended period.This raises the economic question about the "time consistency" of the decision: after the student is part way through the course of study and has already invested some time, is there still a net benefit from making the additional required time investment?To sharpen the argument I break this time consistency problem up into three separate issues.

A. The Best-Laid-Plans Problem
From my own experience and conversations I've had with many faculty, both SCALE affiliated and others, it is apparent that students often do not make the full time investment which is requisite for gaining understanding of the subject matter.In some cases, there never was any intent by the student to make such an investment.But in most instances where the investment falls short of the mark, lack of intent is probably not the right explanation.To better understand why we see underinvestment, it is helpful to conceive of learning as a highly risky activity.
There are many dimensions to this risk.One is in the alternative value of time.If competing demands for the student's time place strong pressure on the student, the student will be forced to re-allocate time away from his studies towards these other ends.Another is in the "match value" of the student and the degree program.One can think of education as in part "experimental consumption."(See [1] and [2].)While participating in the role of student, the student learns whether that role is right for him much in the same way that a diner at a new restaurant learns whether it's worth making a repeat visit.(For a view of the labor market as a matching process see [3].)Both of these stories suggest that the underinvestment in time is efficient.The rational student allocates time to its best use, given the student's current understanding of what that best use is.There is reason to believe, however, that much of the underinvestment is inefficient.That is, much of what passes for studying, doing homework, writing a term paper, or other like activities, is really pure time dissipation.Several reasons can be given for this waste.Among them are: 1) fear of failure, 2) lack of focus, and 3) poor work habits.An economist would describe the situation as one of "moral hazard," effort directed towards learning creates a disutility in the student.In this instance, however, that description does an injustice to the situation.These same students are quite happy when actively engaged in some learning task.
The problem seems to be in getting started.For whatever reason, if the student is idling there is a tendency to continue idling in the near future rather than to start in on productive work.This idle time should not be confused with recreation.The phrases a) procrastination, b) writer's block, and c) wheel spinning, are among those that the students themselves would use to describe the situation.These expressions conjure up frustration, not enjoyment.It is true that some idle time is necessary, even for the most fertile minds!However, much of the idle time we instructors witness in our students is not needed and this time wasting increases the chances that the real work never gets done.
Good instruction aims, in part, at minimizing this inefficient use of the student's time.To the extent that fear is the primary explanation of this inefficiency, mileage can be gotten by creating an environment in class where mistakes are tolerated, even welcomed, in the spirit that its all part of the learning process.More generally, the problem is attenuated by creating a stake for the student in the course.
In the traditional setting, this stake comes from attendance in lecture.Quite apart from the lecture's role as a vehicle for disseminating information, the instructor can use the lecture to monitor the student's commitment.From the student's perspective, why go to class if you aren't going to do the work out of class?It is true that very bright and highly motivated students may be able to do the work on their own at their own convenience.However, it is also true that class attendance serves as an indicator to the instructor about how serious the student is--the bulk of the students who don't come to class aren't putting in the time.
ALN offers flexibility--anytime and anyplace learning.But that flexibility is at odds with creating a stake in the course.An entirely self-paced curriculum, for example, offers maximum flexibility but no stake whatsoever.The challenge seems to be in creating such a stake while still offering a fair degree of flexibility to the student.
The best results I have seen in this regard are by making use of cohort effects, namely, peer-to-peer learning.I have witnessed the beneficial effects of interaction among a given student cohort in my own ALN teaching.And I have heard others relate similar experiences.On-line interaction among students can create a bond between them that is very strong.But for this to work, the communication between these students must be timely.That puts a damper on the amount of flexibility which can be permitted.

B. The Junk Mail Problem
The fact that electronic communication is both so fast and so cheap creates the following dilemma.There is so much information to view that the individual must filter the bulk of it--either not look at it at all or skim it very quickly.What type of information might get through the filter for a more careful read?If the identity of the sender is known, that will be used as the indicator whether to filter or not.What if the identity is unknown?Before answering this, turn the question around.If a sender of an electronic communication wants the recipient to read the message carefully, but the reader doesn't know the sender as of yet, what steps should the sender take so that the message makes its way through the reader's filters?The answer is to put a "stamp" on the message, of value large enough to notify the reader that this was a deliberate, costly act.The reader will take this act as a signal: "this is a message worth reading, don't toss it into the trash unread."The stamp serves a signaling function only; it has nothing to do with the cost of transmitting the message itself.
At the start of a course, the instructor's relationship to her students is very much like that of the sender who is unknown to her readers.It is true that students who are grade conscious will have incentive to look at some of the course materials, but only those components which are "going to be on the exam."In thinking about their subject matter, however, instructors don't envision this artifical compartmentalizing of the material.The instructors want their students to think of the subject holistically, just as they do.For this reason, a good instructor will take actions to differentiate the entire body of materials that she distributes from the mass of information that the student should rightly filter.In considering ALN teaching practices, one should look for the metaphorical stamp that the instructor affixes to her materials.The instructor's credentials may serve as such a stamp, to the extent that the students can verify these credentials and identify what they signify. 3Even impressive credentials may fail to do the trick, however, because the acquisition of these credentials is not directly associated with the materials that are being transmitted.A more effective stamp is the instructor time used in communicating with the students 4 .This is readily verified and is easily identified by the students as being germane to the course.
If this analogy is appropriate, then as I have argued elsewhere (see [4]) it makes sense for the instructor to devote a disproportionate amount of time to the course at the outset.Once the students have identified the instructor as a sender of high quality messages, there is no more reason to use on-line discussions to convince the students to take the course materials seriously.Consequently, the instructor can cut back on the amount of time spent conferencing.

C. There's Work and There's Work
Even when actively engaged in learning out of class, students may take an approach to their studies that is less productive than optimal.In my course, for example, I find the students trying to memorize the textbook.But since the abstract concepts have little meaning in themselves, it would be a far better use of the students' time to try to apply these concepts to concrete situations.Yet the students often won't do this on their own, my chiding notwithstanding.To again use the jargon of economics, the students view this chiding as "cheap talk." If an instructor is to exercise some control over the activities of students out of class, the instructor must send a message to which the students will pay attention.The most effective way to do this is by embedding the message in an assignment which will be evaluated for course credit.If there is enough out-of-class work to be done, the student must allocate his scarce time to doing that rather than to some other activity that he'd be inclined to undertake otherwise.
Recent advances in web technology have been a real boon in this regard.It is now possible to give assignments on line which can be accessed from wherever a student can connect to the Internet and which are evaluated almost immediately after they are submitted.Moreover, the software is "smart" in that it provides useful suggestions to the student on how to proceed after a mistake has been made.At the University of Illinois, experience with CyberProf © and Mallard © indicate both the effectiveness of this teaching approach and the popularity of it with the students.
Yet interactive software should not be viewed as a panacea.It works by having the student make a mouse click on the right selection or by entering a short answer.With these capabilities, it can be used effectively for getting the students familiar with the material and in teaching the basics.However, it is incapable of teaching "higher order" thinking, (e.g., how to frame a question properly, how to successfully use a methodology in a context other than the one in which it was presented, how to judge whether an argument is tight or full of holes).It is these higher order skills which are at the heart of "learning to learn." If there is to be evaluation on assignments that test these higher order skills, the evaluation must be done by people, not by computers.These evaluations can be transmitted electronically, just like the assignments, but the process cannot be completely automated.
As many authors have pointed out (for example see [5]), employers really seek in their new hires the ability to do further learning on the job--it is not the knowledge that students have when they come out of school per se.Moreover, employers are after general skills -the ability to communicate well (in writing and in speech), the ability to work well in groups, and the ability to take initiative.Yet when used in pure selfstudy mode, interactive software simply is incapable of teaching these skills.Interaction with people is needed here.Perhaps much of that can be peer-to-peer, but some degree of instructor mediation is likely needed in addition; both to provide the appropriate incentives, as discussed in the previous subsections, and to provide written evaluation.If higher education is to be responsive to the demands of the labor market, a pure self-study approach is not the solution.

III. INCREASING RETURNS AND INPUT SUBSTITUTION
Before turning to the topic of this section, let's briefly discuss the time elicitation problem for faculty.
There are two issues here.One is in regard to the time put into developing materials.Call this investment time.The other is in regard to the time spent communicating with students.Call this performance time.
Since monitoring these efforts is imperfect at best, direct compensation in the form of "piece rates" is untenable.For investment time, the solution is to have the faculty member own the returns to the investment. 5For performance time, the solution is to have the faculty member have a stake in the job, just as we established the need for the student to have a stake in the course.How that stake is created will depend on who is doing the instruction.This is the issue to which we turn next.To start, we'll look at this from the view of the traditional on-campus institution.Then we'll consider the impact of distance ALN.
In some cases, the assignment of instructor to course is a trivial matter.Teaching the course requires a research expertise and the topic lies within a sub-discipline in which few faculty have the requisite knowledge.There are many courses, however, which lie well within the frontier of knowledge (but do provide new knowledge from the student's perspective).For these more basic courses, there is a larger pool of potential instructors from which to draw.The salient question: how are the qualifications of those in this pool used to efficiently select the "right" instructor?Note that these qualifications will directly relate to the time value of the potential instructor.
To make this more concrete, think about the possibility of using tenured or tenure track faculty in instruction, on the one hand, or graduate student TAs, on the other.Consider the following hypothetical case.We have a course with 60 students.The course could be taught by one faculty member.Or, for roughly the same expenditure on teaching personnel, we could divide the course into three separate sections and have each section taught by a TA.Which is the superior alternative in terms of educational quality?This is an example of the "input substitution" issue.
Let's move from hypothetical to fact as we make the issue more complex by considering economies of scale.For the most recent semester for which the data are available, spring 1996, there were 10 courses with enrollments over 800 at the University of Illinois at Urbana-Champaign, with the largest course having an enrollment that exceeded 1700. 6All of these courses are at the introductory level--the bulk of the audience being first-year students.These courses are in the humanities, the sciences, and the social sciences.All make extensive use of TAs, though the organizational structure varies substantially.Some have the TAs teach stand-alone sections while others have the TAs run quiz sections that accompany large lectures.Some have a tenured faculty member delivering these lectures and coordinating the TAs, others have an adjunct faculty member doing the analogous job.
In terms of personnel expenditure per student, these large classes are much less costly than the median undergraduate course.Roughly speaking, personnel costs per student go up by about 200%, when going from the large course to the median course.
Is the cost saving in the large courses an indicator of increasing returns to instruction or an indicator of quality degradation in the large classes?If a mixture of the two, can we parse out the share of the one from the other?
The source of the increasing returns, if there are indeed increasing returns, is having a larger pool of potential instructors and hence a better fit of instructor qualifications to the needs of the course.It is important to note that the efficient pairing of instructors to courses is not intrinsic.Clearly teaching productivity matters.But so does the instructor's time value and that depends on the market.
This discussion relates to the currently observed increase in the use of adjuncts in instruction.It may be that this is an efficient response to decreases in line money for instruction.But let's not rule out the other possibility.It may be an inefficient degradation of quality that occurs because administrators are reluctant to pass on the cost that is borne by the students, when adjuncts are hired, to the tenured or tenure-track faculty, who would be re-allocated to the high demand courses if the adjuncts weren't hired.
Focusing on TAs rather than adjuncts, the quality degradation, if it occurs, is the result of two factors.First, the TAs may have insufficient qualifications.Here it is important to make a further distinction.The TAs today are the professors of the future.Thus, part of being a TA is being an apprentice instructor.
Poor teaching may result from lack of experience.But the apprentice function must be filled to preserve the stock of instructors, in total.If TAs don't teach, then new assistant professors will be apprentice teachers.The students bear the cost of this apprenticeship, to be sure, but socially it is a necessity.The key point is to realize that apprenticeship can't be foregone, though it can be passed on from one group of instructors to another.The alternative reason we may feel that the TA has inadequate teaching qualifications is that the TA doesn't have sufficient expertise in the subject matter.They either teach things that are outright wrong, or they present material in an insufficiently rich context.
The second possible source of quality degradation with large classes arises from an increased student/faculty ratio.The consequence being that higher order thinking skills are de-emphasized in such courses.But it may be that mastery of the basics is all that is reasonably expected out of such introductory courses, irrespective of class size.Indeed, if one conceives of the question at the level of the curriculum, then it must be determined whether the best approach is to have a constant mix of basic and higher order thinking throughout or whether this mix should include more emphasis on higher order thinking as the student moves through the curriculum.
Let's turn to the implications that distance ALN has for this discussion.Absent the constraints imposed by place, we can now envision a global pool of potential instructors from which to draw.In effect, we can replace the internal labor market with an external one.By moving to this external market, we may see a move to the unbundling of teaching services from other services that have traditionally been provided along with teaching--research by tenured and tenure-track faculty and pursuit of their studies by TAs.This unbundling should provide more efficient allocation of personnel in instruction and that should lead to cost savings.Moreover, because the demand will be global as well, there will be additional subject areas that will benefit from having this larger pool of potential instructors.This will happen if the traditional institutional setting had insufficient demand to rationalize having a dedicated faculty line in that area.
There are, of course, other avenues through which ALN matters in this discussion.The most obvious is through the use of interactive software, which may substitute for personnel as an input in instruction.To the extent that this can happen and yet enhance (or at least not degrade) the quality of instruction, this itself is a source of increasing returns.The distributional cost of such software is minimal as compared to the development cost, which is independent of the number of users.
Another avenue is through having a "helpline" staffed by instructional personnel to provide near round-theclock, almost-synchronous response to students who are stuck while working on an assignment.This would be prohibitively costly to do in small classes.In large classes, it is possible to achieve at a reasonable price.We have found this to be quite useful in courses that emphasize numerical problem solving, where the responses can be brief and to the point.Moreover, since the questions and responses are addressed to a public conference, other students can benefit from them.

IV. INSTITUTIONAL COMMITMENT
This section deals with how the market comes to understand the quality of product offered by a distance ALN program. 7When the next best alternative is on-campus instruction, there is a large potential cost saving in distance ALNs from economizing on brick and mortar.This is at least part of the impetus behind the Western Governor's University and the Florida Gulf Coast University.Brick and mortar serve the role of housing instruction.That function may no longer be needed, as instructional technology using the internet might be an adequate substitute.But brick and mortar also serves the role of making the institution commit to good instruction for its students and to the rewards it has promised its faculty and staff.This commitment function is necessary, even if the housing function is not.In this section we'll discuss how commitment may be achieved in the absence of brick and mortar.
Part of the story is accreditation, which from our view can be thought of as third-party verification of the inputs in instruction.Part of the story may be standardized testing, which can be thought of as third-party verification of the outputs in instruction.And part of the story may be reputation, which can be thought of as an informal market mechanism for assuring both input and output quality.(See Klein and Leffler theory [6].) We can think of reputation as embedded in brands--the name of the institution is readily recognized and it is easy to associate quality level to that name.Since standard exam performance is also easy to recognize and interpret, the rest of this section will be devoted to contrasting branding versus standardized tests as alternative means for assuring the quality of distance ALNs.
In Klein and Leffler's theory, reputation is viewed as a non-tangible asset.The value of that asset derives from future sales of the product at premium prices.In contrast, if the seller makes little or no return on the sale of its product, there is an incentive to "cash in" on the reputation--make sales of cheaper, lower quality output at premium prices.This, of course, can't be sustained indefinitely.Buyers won't tolerate it.The cashing in works because the buyers are temporarily fooled--they expect high quality.
The brand name itself can be a source of increasing returns.The increasing returns can lead to a degree of monopoly and that in turn can rationalize the price premiums.Under this scenario, we might expect a few large players in the distance ALN market.They'll offer good education.But one should expect markups, because of the market power.
One might reasonably expect that the brands for distance ALN emerge from those institutions that already have strong reputations in the on-campus market.The familiarity of the brand name notwithstanding, this is probably best viewed as entry into a new market, if, as might be expected, the off-campus market differs demographically from the on-campus market in characteristics other than location.
The economic theory of entry (for example see [7]) suggests that early entrants will make above normal returns if they have some asset which is valuable in production and can't be recovered if production ceases. 8If this is the right scenario, the reputational advantage in the on-campus market can be parlayed into a major presence in the distance ALN market.But for it to be the right scenario, there must be such a non-recoverable asset.
There has been much recent discussion about entering the distance ALN market now, for fear of being passed by other competitors.Implicit in this discussion is that the non-recoverable asset is the knowledge of how to run an ALN program.That would be the correct implication if this knowledge were embedded in the institution itself.But if that knowledge really lies in the individuals who run these programs and these individuals can readily move from one institution to the next, then it is not clear that there is an advantage to getting into this business very early.
Absent the existence of such a non-recoverable asset, the right theory is that of "Contestable Markets."(See [8].)In the contestable market case, early entrants and potential competitors are on an equal footing.Hence, there is only a normal rate of return, even for a first-in firm.The outcome in the contestable market case is at odds with the predictions of a model that assumes the existence of a strong reputation for high quality.Consequently, if this contestable market view is the right scenario, institutions with strong reputations for their on-campus performance should be very careful of expanding their base of operation onto the less familiar terrain of distance education.
If the distance ALN market is contestable, one needs an alternative mechanism for quality assurance, and standardized testing seems a natural to play this role.In fields of study where standardized tests already exist, one might reasonably predict rapid entry into the distance ALN market, if, apart from the quality verification issue, the benefits from ALN exceed the costs of supplying it.And if the volume in this market is substantial, one should see an increase in the price of the standardized exam.If there currently is no standardized exam in a field of study, however, the situation is much more complex.Standardized testing, by its nature, is itself an increasing returns activity, because the primary cost is in the design of the test itself.This cost is independent of the number of students who ultimately end up taking the exam.
This leads to a chicken and egg problem.Construction of a test in a particular field of study won't occur unless there is sufficient demand to warrant it.Yet if that demand is to come from distance ALNs, the magnitude of the demand will depend crucially on whether the standardized test is already in place.
Another problem might emerge even if it is evident that demand is large enough: there could very well be competition for the standard.And until a standard is commonly agreed upon, entry into the distance ALN market will be hampered.This is especially so if the education is geared toward preparation for the test, for fear that the education will end up not having much value should an alternative standard be arrived upon eventually.

V. CONCLUSION
Let us briefly return to the key question raised in the introduction, about the right form of instruction--selfdirected or instructor-mediated.If a full instructor-mediated approach is right, then even major advances in Internet technology are likely to translate, at best, into modest cost reductions in instruction.Dr. Gomory's view, I believe, relies on the self-study approach being right.Dramatic cost reductions in instruction would indeed result from significant improvements in the technology, if that were the case.At root then in my disagreement with Dr. Gomory is the degree to which instructor mediation is indispensable, no matter how the technology improves.I do not believe that there can be a purely technological solution to the incentive issues I discussed in section II of the paper.And the greater the extent of instructor mediation, the more costly it is to assure institutional commitment to quality, as discussed in the previous section, especially if that assurance is provide by the reputation of the institution.
If there is little room for input substitution does this mean that ALN will have only minor impact?Quite the contrary, in my view.For demographic reasons, mainly the aging of the population and the increasing frequency with which people change jobs, the demand for higher education will grow faster than the rate of population growth.This rapid growth in demand will be fueled by adult learners, who will constitute an ever increasing share of the market.This demand growth will tend to pull up costs, by raising the prices of instructional inputs.
It is my view that ALN will insulate higher education from much of the effects of this demand pull and that in the next twenty years or so we will not experience the rapid real increases in the cost of higher education that we have witnessed over the last twenty years, the demand growth notwithstanding.I base this on the belief that ALN will allow a better fit between the pool of instructors and the demands for their services, as discussed in section III, and that will go a long way towards cost containment.
The demand for education by adult learners, particularly among those who have already attained the baccalaureate, raises another point.This concerns the durability of the education that is being received.It can't possibly be cost effective for adult students to be enrolled in a degree program in perpetuity.Most of the lifelong education that these adult learners seek must indeed be self-directed study, but of an informal kind.The infrequent stints in formal settings must be solid preparation for the more regular self-study.
The adult learners will demand their formal education be durable.This demand will put continuing pressure to improve educational quality as the market expands to meet the demands of adult learners.