Visions of Quality Assurance in Online MBA Programs

Online MBA programs have undergone significant growth in recent years. However, quality assurance measures have not kept pace with this growth. The purpose of this study was to identify and prioritize aspects of quality assurance specific to Association to Advance College Schools of Business (AACSB)-accredited online MBA programs. The Delphi methodology was used to facilitate a group conversation among administrators, faculty members, and instructional designers around the topic of quality assurance for online Master of Business Administration (MBA) programs over the next 3-5 years. This paper reports the results of this study and how the results will help to direct the efforts of those involved in the delivery of a quality online MBA program.


Visions of Quality Assurance in Online MBA Programs
Fully online MBA programs accredited by the Association to Advance College Schools of Business (AACSB) have grown dramatically in recent years.In 1989, there were no AACSBaccredited online MBA programs; by 2015, there were 192 fully online AACSB-accredited online MBA programs (Nelson, 2016).Quality assurance measures and accreditation standards, however, have not kept up with this growth.Accreditation is the traditional quality assurance mechanism used by institutions of higher education (IHE).AACSB, the leading accrediting body for business colleges, uses a set of standards to promote excellence and continuous improvement in business colleges but these standards do not focus specifically on how courses are delivered.While the AACSB first acknowledged the growth of distance learning in 1999 and again in 2007 (AACSB, 2007), AACSB standards did not specifically address the quality of online learning until 2015 and even then, only on a limited basis (AACSB, 2013;AACSB, 2015a).Due to the AACSB's historical lack of specific standards related to online learning, most business colleges have developed their own internal quality assurance frameworks, often using popular external quality assurance models and frameworks (e.g., California State University, Illinois Online Network, Quality Matters, Online Learning Consortium).Moving forward, though, the question remains as to whether these frameworks offer the best approach for quality assurance in online MBA programs.
Despite the strengths of existing quality assurance frameworks, they were not designed specifically for MBA courses and programs.Research suggests that different disciplines have their own situational factors and unique issues that influence how courses are taught (Arbaugh, 2005;Arbaugh, Bangert, & Cleveland-Innes, 2010).Given this, the purpose of this study was to investigate what stakeholders involved with online MBA programs think about the future of quality assurance in AACSB-accredited online MBA programs.

Quality
A high-quality education provides students with the knowledge, skills, and abilities necessary for success.Quality assurance allows stakeholders to have confidence in the quality and value of the education provided to their students (European Commission, 2016).Not only must institutions of higher education meet basic quality criteria set forth by accrediting agencies in order for their students to even be eligible for federal financial aid, but students' perceptions of the quality of a program can also influence an institution's enrollments and, in turn, their bottom line.
Quality in online learning has its own unique considerations.For example, online programs often face higher levels of scrutiny than traditional face-to-face programs (OnlineCollege.org, 2011).T he reputation of online learning has been compromised to some degree by negative press related to the proliferation of online diploma mills (Pina, 2010), financial aid fraud in online programs (Federal Student Aid, 2011) , and investigations of online for-profit schools (Associated Press, 2007).Placing an additional focus on quality and quality assurance will help online programs to overcome negative perceptions related to this delivery modality.Many factors influence quality in both traditional and online education (Mariasingam & Inglis, 2012).We contend that online education in particular requires its own distinct quality metrics that are not always fully addressed by all accrediting agencies and quality assurance frameworks.

Accreditation
The United States, unlike many other countries, does not have a centralized federal authority (e.g., a Ministry of Education) controlling the quality of higher education (U.S.Department of Education, 2016).Instead, the practice of accreditation has evolved over time as a way to ensure that the education provided by institutions of higher education meets a basic level of quality.Accreditation is voluntary but important, as students are eligible to receive federal financial student aid only if they attend an institution accredited by an approved accreditor.The functions of accreditation are to (a) assess the quality of academic programs, (b) create a culture of continuous improvement designed to raise standards, (c) involve faculty members and staff in processes, and (d) establish criteria for professional criteria and licensure.Accreditors monitor and periodically evaluate institutions to verify that they continue to meet some pre-established standards.There are two basic types of accreditation: institutional and programmatic.Institutional accreditation is administered by regional and national accreditors and applies to the entire institution.Specialized or programmatic accreditation applies to a specific program, department, or school and is typically supplemental to institutional accreditation.It is important to note that while accreditors develop quality standards, they have no legal control over an institution (U.S.Department of Education, 2016).

Accreditation for Online Learning
Fully accredited online programs are recognized by the same regional or national accrediting bodies that recognize traditional on-campus programs (OnlineCollege.org,2016).These accrediting bodies address online learning to variable degrees within their overall standards (CRAC, 2011).Online programs may also be accredited by other institutional, programmatic, or specialized, accrediting agencies.The Distance Education Accrediting Commission (DEAC) ( 2016) is one recognized specialized accreditor of distance education institutions and programs.DEAC also offers Approved Quality Curriculum (AQC) (2016) as an external and peer review system.

AACSB Accreditation
The AACSB is the leading accrediting body for business colleges and is an example of programmatic accreditation.Like other accrediting agencies, the goal of AACSB accreditation is to ensure that the education provided by its members meets acceptable levels of quality.However, in addition to meeting basic requirements, AACSB standards are designed to promote excellence and continuous improvement.Until 2015, the AACSB standards did not specifically address the quality of online learning, and then did so only on a limited basis (AACSB, 2013(AACSB, , 2015a)).As a result, business colleges need to rely on internal quality assurance measures, external quality assurance models offered by organizations such as California State University, Illinois Online Network, Quality Matters, and the Online Learning Consortium (Chico, 2016;ION, 2015;Maryland Online, 2014;OLC, 2014), or a specialized accrediting agency such as DEAC (2016) if they choose to assess and ensure the quality of their online courses and programs.

Quality Assurance Frameworks
A number of quality assurance frameworks have been developed specifically to evaluate and improve the quality of online courses and programs.These programs can support an official external review process leading to certification or they can be used in an informal internal review process.For instance, the Quality Matters (QM) rubric and peer review process is one popular quality assurance framework focused on quality online course design.The Online Learning Consortium's Five Pillars of Quality Online Education and the corresponding OLC Scorecard for Online Learning (OLC, 2014, 2016) is another popular quality assurance framework.
The AACSB (2007) first developed guidelines to address quality issues in distance education in 1999 and revised the guidelines in 2007.Gaytan (2013) subsequently developed a quality framework, by analyzing the 2007 guidelines, as an aid for business school faculty members, administrators, and online educators.However, neither the 2007 guidelines nor Gayton's quality framework has been formally adopted into the AACSB accreditation standards.While some online MBA programs may use a supplemental quality assurance program to guide the planning and delivery of online instruction, no summary of such information is found in the literature.The goal of national and regional accrediting bodies is to ensure a basic level of quality.Specialized accrediting bodies such as the AACSB strive for excellence and aim to accomplish this through continuous quality improvement.The application of quality assurance measures and metrics designed specifically to improve the quality of online courses and programs can further strengthen and grow the online MBA programs already certified by the AACSB.

Methods
The Delphi Method (Delbecq et al., 1975;Linstone & Turoff, 2002;Shelton & Pedersen, 2015;Shelton & Creghan, 2015) was used to investigate stakeholder perspectives of the future of quality assurance measures for AACSB-accredited online MBA programs.While four key features define a Delphi study including anonymity, iteration, controlled feedback, and the statistical aggregation of group response, there are a wide variety of ways in which these features may be applied (Rowe & Wright, 1999).This study varies from the original or classical use of Delphi in three ways (Dalkey & Helmer, 1963).First, data were collected, and communications were delivered electronically via the Internet.Cole, Donohue, and Stellefson (2013) found the use of the Internet to be a "best-fit" for the needs of a Delphi study.Also, in contrast to the classical Delphi, the purpose of this study was not to reach consensus among participants.Rather, it had multiple objectives consistent with those outlined by Delbecq et al. (1975): 1. To determine or develop a range of possible program alternatives; 2. To explore or expose underlying assumptions or information leading to different judgments; 3. To seek out information which may generate a consensus on the part of the respondent group; 4. To correlate informed judgments on a topic spanning a wide range of disciplines; and 5. To educate the respondent group as to the diverse and interrelated aspects of the topic (p.11).
And lastly, the opinions of three distinct groups of experts were sought with comparisons made both among and within groups to help determine where consensus existed and where it did not.Quality assurance for online learning requires coordination among all involved and efforts may be compromised if there is a lack of agreement on best practices.The following research questions guided this study: 1. How should quality be assured for online MBA programs within the next 3-5 years? 2. Does the quality assurance vision differ between various stakeholder groups including program administrators, faculty, and instructional designers?3. What are the potential implications of stakeholder views on the implementation of quality assurance programs and future direction of AACSB standards?
For this study, a non-random, purposive sample of expert participants with at least five years of experience in online learning were recruited from AACSB-accredited online MBA programs.The expert panel participants were identified through a combination of methods with final selection ultimately relying on the judgment of the primary investigator.Administrative and faculty experts were identified through nomination by administrators in a Midwestern college of business and AACSB administrative staff using the membership roster of the MBA Round Table (MBA Roundtable, 2012) and the AACSB membership listing (AACSB, 2015b).Each "nominator" was contacted personally by the investigator and asked to either suggest participants for the panel or to provide the names of those who could suggest others as expert participants (Delbecq et al., 1975).Additional faculty and administrative experts were identified through their presentations at online learning conferences or through related peer-reviewed publications.Expert instructional designers were identified through recommendation of administrators or faculty at a business college, or through their publications or presentations at online learning conferences.All potential participants underwent a screening process that involved a review of the website of their college and personal communication as needed to confirm their involvement with a fully online AACSB-accredited MBA program.Due to the specific inclusion criteria, the pool of potential expert participants for this study was relatively small.In 2015-2016, only 228 institutions reported offering an MBA program through internet-based distance education courses (Brooks & Morse, 2016).Additionally, less than 5% of business colleges attain AACSB accreditation (AACSB, 2016).A number of potential panelists did not meet the inclusion criteria as their AACSBaccredited institution had not delivered a fully online MBA program for more than 5 years.
Participants were placed in one of three groups: (a) MBA program administrators at the program director level or above, (b) faculty with at least five years of experience teaching online MBA courses, and (c) instructional designers with at least five years of experience designing online course content currently working in an online MBA program.Twenty-two panelists met the inclusion criteria and completed the first round; eighteen panelists completed all three rounds of the study (Table 1).Table 2 and Table 3 illustrate demographic and other information about the participants.The data collection process consisted of three rounds of questionnaires, administered sequentially over 10 weeks, with controlled feedback delivered to participants between rounds through a summary of the previous results (Dalkey, 1969).The concept of controlled feedback is an important aspect of Delphi research with the investigator determining how aggregated data are shared with participants (von der Gracht, 2012).Data analysis for each round was performed by the investigator immediately after each survey was closed so that aggregated results could be reported back to panelists within two weeks and in conjunction with the delivery of the subsequent survey.The qualitative Round One responses were used to craft the quantitative surveys delivered in Round Two and Round Three.The Round Two and Round Three surveys were identical but in Round Three panelists were encouraged to review the Round Two group statistics and to use this data to inform their Round Three responses (von der Gracht, 2012).Descriptive statistics (i.e., mean, median, mode) and measures of dispersion (i.e., standard deviation, interquartile range) for each item statement in Round Two were shared via the study Website and were included following each item statement in the Round Three questionnaire.
Responses were analyzed, and results shared with participants following each round; results were also used to inform the subsequent questionnaire.The qualitative data was coded using multiple processes to help ensure the validity and trustworthiness of the data and to minimize bias including peer debriefing, member checking, and bracketing.Descriptive statistics, including mean, median, mode as well as standard deviation and interquartile range, were used to analyze quantitative data.Mean ratings of individual statements among groups were analyzed using the Kruskal-Wallis H test with multiple comparisons made on all statements (Laerd Statistics, 2013).

Round One
In Round One, the traditional Delphi approach was used with a broad, open-ended question to help establish the variables of interest for subsequent rounds (Cole et al., 2013;Hsu & Sandford, 2007).The initial survey asked panelists to respond to the following question: "How should quality be assured for online MBA programs within the next three to five years?"Participants had the ability to complete five different textboxes with answers in Qualtrics.Participants were limited to five different answers in order to direct their responses although the size of each text box was moderately large.Participants were informed that they did not have to provide five different answers, and the boxes were expandable, allowing unlimited space to present answers.The panelists were not directly asked to indicate the quality assurance frameworks currently used by their institutions.Any frameworks noted in their responses were spontaneously generated.
Round One data were qualitatively analyzed by the investigator and a peer debriefing process was used to improve the validity of inferences made from the data and to add credibility to the study (Creswell & Miller, 2000;Cohen & Crabtree, 2008;Lincoln & Guba, 1985).The open-ended responses generated in the Round One Qualtrics survey were entered into an Excel spreadsheet for analysis.In the first stage of the analysis, the primary investigator divided the essay responses of participants into individual statements, grouped similar statements together, consolidated these statements when possible, and generated item statements that were then categorized into major themes.Next, this initial coding was reviewed by two peers in the business college; changes made to the initial coding were discussed, negotiated, and incorporated where appropriate.Finally, data were cross checked and revised with the assistance of an experienced Delphi researcher to further reduce threats to internal validity.Participants meeting the inclusion criteria generated seventy-two responses in all (or an average of 3.27 responses per panelist).When generating the final item statements for the next round, the original words of the panelists were used whenever possible.
As part of the member checking process, a document summarizing the aggregated Round One data was created for each major theme and shared with study participants in Round Two.These summary documents included the individual item statements generated in the coding process, along with the text responses that supported each item statement.This illustrated to the panelists the interpretations of the investigator related to the creation of the final item statements.This member-checking process gave the panelists an opportunity to react to the data and provide additional open-ended comments that could then be incorporated into the final narrative (Creswell & Miller, 2000).

Round Two
The item statements and major themes generated from the open-ended question in Round One were used to create the Round Two questionnaire.In this questionnaire, each major theme was presented with its associated item statements.Panelists were asked to rate the statements presented under each theme for perceived importance using a 5-point Likert scale with 1 indicating "not at all important" and 5 indicating "absolutely critical" importance; they were also given the opportunity to provide additional comments, rationale, or clarification if they desired.Participants were also asked to rank each of the major themes in order of importance from 1-7 with 1 indicating the most important theme.The Round Two process helped to identify items requiring clarification or areas of consensus and disagreement and helped to open a dialogue among participants (Delbecq et al., 1975;Ludwig, 1997).

Round Three
The Round Three process allowed participants the opportunity to better understand other panelists' positions and offer more accurate judgments regarding the issues under discussion (Delbecq et al., 1975).This final questionnaire was important as it provided closure for the study, suggested areas where diversity exists while still allowing for aggregation of opinions and offered direction about future research and planning (Delbecq et al., 1975).The Round Three questionnaire was identical to the Round Two questionnaire except for the addition of descriptive statistics and measures of dispersion added to the associated item statement.The summary results of Round Two, including analysis of statement ratings using descriptive statistics, rankings, and open-ended responses, were also provided to panelists.The summary of the responses of their peers made panelists aware of the range of opinions and gave them an opportunity to reflect upon their own original responses (Delbecq et al., 1975;Franklin & Hart, 2007;Ludwig, 1997).
Panelists were asked to review this information and were again given an opportunity to further clarify their own opinions and revise their own responses based on the new information they received regarding the opinions of their peers.

Results and Discussion
In Round One, 22 expert panelists, classified in one of the three subgroups of administrator (n=9), faculty member (n=5), or instructional designer (n=8), responded to the question, "How should quality be assured for online MBA programs within the next 3-5 years?"There were 72 essay responses to this question.The 72 responses were collated, compressed, and combined to generate 46 item statements that were then categorized into seven major themes or subscales.Table 4 depicts the means and standard deviations of these themes by role in rank order.Each theme will be discussed in more detail below.

Theme 1: Academic Integrity and Rigor
Although the theme Academic Integrity and Rigor received the highest ranking by panelists, only one associated statement was among those most highly rated overall and only two associated statements (of four) were rated as "very important" or higher (Table 5).Delivering a highly demanding curriculum with rigorous grading standards was ranked third overall and implementing rigorous systems to ensure the academic integrity of quizzes, exams and assignments was rated as very important.
Panelists ranked academic integrity and rigor as the most important aspect of quality in online MBA programs and supported a highly demanding online curriculum, rigorous grading standards, and rigorous systems to support academic integrity.While the AACSB accreditation standards require business schools to set policies and procedures that support ethical behavior and mechanisms that address breaches of such policies, the standards do not specify requirements defining academic integrity or rigor for programs (AACSB, 2016).Instead, academic rigor requirements are phrased more broadly as determining the level of student performance that "triggers curricular interventions to address deficiencies" with challenging but attainable goals set as internal benchmarks (AACSB, 2013, p. 13).
The results also suggest that faculty members and instructional designers disagree on the emphasis of academic integrity and rigor in terms of program quality.While faculty members ranked academic integrity and rigor higher than instructional designers, this does not mean that instructional designers don't value these things; rather, it likely suggests that faculty members are simply more directly involved with assuring quality in this area.All 8 items within the theme of course content, design, and delivery were rated as "important" to "very important" or higher (> 3.50) (Table 6).This theme placed second in final rankings (first in Round Two) and had four associated statements among the most highly rated overall with a mean of 4.25 or higher.In this theme (as well as across the entire study), panelists rated provide resources and support for ongoing course design, development, delivery, and technology as the most important.O ther organizations and accrediting bodies have recognized the importance of resource allocation.For instance, the Online Learning Consortium (OLC) Quality Scorecard (2014) requires institutions to develop a process for planning and allocating resources for online programs.Panelists also reported that providing high quality, relevant, and practical course content and using the same learning objectives in both online and face-to-face courses are important components of a quality program.In addition, they reported that student interaction, engagement, and collaboration are important parts of a quality online MBA program and that online courses should be based on a common course template but one that allowed adequate freedom for instructors to change as they wish.Provide relevant and practical course content that can be applied directly to the workplace.

Use technology appropriately.
Design courses that promote student engagement and collaboration.
Provide quality content with the same learning objectives in both online and face-to-face classes.
Establish a common course template, structure, or architecture that also provides adequate freedom for an instructor to teach as s/he wishes.
Utilize both formative and summative assessments in course design.
Use innovative approaches to curriculum design and delivery of instruction.Panelists generated the most comments-and three of the most highly rated statements overall-related to faculty qualifications, development, and support (Table 7).This emphasis highlights the need to train and support faculty members as they move from a face-to-face to an online delivery mode.Panelists thought it was important to support faculty participation and growth in online learning.They also thought that uniform standards need to be established for faculty qualifications and credentials.Training and support specific to the LMS was the highest rated item under the Faculty Qualifications, Development, and Support theme.4.17 1.10

5-9
Create a course design partnership between faculty and instructional designers.

5-7
Faculty are required to attend training before teaching an online 3.72 1.07 course.

5-2
Require MBA instructors to be proficient in a uniform set of skills related to online teaching and learning.

5-6
Faculty are required to participate in training/professional development for research-based, best practices of online course design and delivery.

5-4
Online faculty should be part of the existing university culture and should also teach in the face-to-face classroom.

5-8
Provide a certification training program for faculty interested in teaching online at the graduate level.

1.31
Theme 4: Quality Frameworks Instructional designers rated the theme of Quality Frameworks higher-although not statistically significantly higher-than faculty members or administrators (see Table 8).But all groups thought it was important to have consistent and universal quality standards for online course design to encourage and maintain the quality of online learning.However, they placed less importance on using external or peer reviews to aid in this process.Develop processes and systems that encourage and maintain 4.28 0.67 quality.

7-3
Establish quality standards for online course delivery (teaching).4.28 0.75 7-4 Implement a structured internal review process adhering to 4.11 0.83 accepted quality standards for online courses and programs.

7-2
Standardize and clearly define online course design expectations 3.83 1.10 based on consistent and universal standards.
3.50 0.92 7-5 Implement external reviews of online courses and programs.
3.17 1.04 Visions of Quality Assurance in Online MBA Programs

Theme 5: Accreditation
Panelists supported AACSB accreditation of online MBA programs and assessment of Assurance of Learning (AoL) standards across all MBA programs regardless of delivery method (Table 9).The current AACSB accreditation standards address select aspects of online learning but embed these within the overall standards and do not differentiate between online and face-toface delivery models (AACSB, 2016).Panelists support tailoring Assurance of Learning Standards (AoL) to the objectives of individual disciplines.No statements or comments were generated related to accreditation by other higher education agencies.

2-2
Online programs should be accredited exactly like any 4.11 1.02 residential, part-time, or executive MBA program.

2-4
Assurance of Learning must be tailored to the learning objectives 3.67 1.24 of individual disciplines, and not from some top-down vision of how MBA programs can be remade to be more appealing to the masses.

2-5
Assurance of Learning should be the same in all modes of 3.39 1.14 instruction.

Theme 6: Learner Support
The fewest responses were offered in the area of learner support.Even though learner support is one of the common themes in published standards for online learning (Chico, 2016;CRAC, 2011;OLC, 2014), panelists ranked the overall category of learner support only sixth in importance.they did consider the statement, providing online student support services, to be of "very high importance" (Table 10).Of the seven themes, evaluation was ranked as the least important.This is interesting because evaluation is typically considered an integral part of quality assurance.Despite the low ranking, five of the eight items were ranked "very important" indicating that panelists did value evaluation (Table 11).The following two items were rated the lowest, and thus may have influenced the overall low ranking of this theme: (a) assess online MBA programs separately and, (2) integrate student evaluations into the quality assurance process.On the other hand, Provide the same level of quality in both online and on campus classes was one of the most highly rated items overall.Panelists also reported that courses and programs should be evaluated on an ongoing basis but considered student evaluations to be of less importance when it came to evaluating courses and programs.

Comparison of Individual Items from Rounds Two and Three
The same questionnaire was used for Rounds Two and Three.The Round Two aggregated data were presented to participants to make them aware of the range of opinions and to give them the opportunity to reflect upon the views of their peers as they formulated their Round Three responses.Round Three data were then compared to Round Two to determine whether providing participants access to the responses of other experts converge in consensus between rounds (Jairath & Weinstein, 1994).In Delphi studies, it is common to use the interquartile range (IQR) and standard deviation to measure consensus (von der Gracht, 2012).Standard deviation values decreased for 31 (67%) of the item statements, increased for 14 (30%) and stayed the same for one statement from Round Two to Round Three.Raskin (1994) and Rayens and Hahn (2000) determined that an IQR of 1 or less was a suitable consensus indicator for 4-or 5-unit scales such as the 5-point Likert scale used in this study.IQR values remained the same for 35% (n=16) of the items, decreased for 46% (n= 21), and increased for 20% (n=9) of the items between rounds.Twenty-nine of the forty-six items (63%) in the final Round Three questionnaire had an IQR value of 1.0 or less, indicating consensus on these items.

Subgroup analysis
One purpose of the study was to determine whether differences existed in how three groups of stakeholders-administrators, faculty members, and instructional designers-involved in the delivery of online MBA programs viewed quality assurance.Kruskal-Wallis was run on all 46 statements in the final Round Three questionnaire to determine whether there were significant differences in the importance scores assigned to questionnaire items between the participants in three groups: "administrators," "faculty," and "instructional designers."Results showed statistically significant different distributions of scores between subgroups for items 1-4 and 6-2 as well as significantly different distributions of ranks between groups for Theme 1 (Table 12).Post hoc analysis was completed using Dunn's (1964) procedure with a Bonferroni adjustment (Laerd Statistics, 2013).Adjusted p-values are presented, and values are mean ranks unless otherwise stated.Items were rated in importance on a Likert scale of 1 to 5 with 5 indicating absolutely critical importance.Post hoc analysis (see Table 13) revealed statistically significant differences in item 1-4 scores between the instructional designers (4.92) and faculty members (12.60) (p = .038)with faculty members scoring "structure an admission process that focuses on quality of students" higher in importance.Item 6-2 ("offer opportunity for electives") also scored significantly higher in importance by faculty members (13.50) than by instructional designers (4.92)(p = .018).These findings are not surprising as instructional designers typically have less direct involvement in the offering of electives or in the admission process.Seven themes were ranked in order of importance from 1 to 7 with 1 indicating the most important theme compared to the others.Theme 1 ("academic integrity and rigor") was ranked statistically significantly higher by faculty (4.60) than it was by instructional designers (12.70) (p = .024).This suggests that while instructional designers are involved in creating online courses that support academic integrity and rigor, there are likely other aspects of their role that they feel more directly impact quality.No statistically significant differences were found in any other group combinations for ratings of item importance or ranking of themes.

Limitations
The sample size of this study was small, thus limiting generalizability.The inclusion criteria were relatively stringent, and the study looked at a small group of stakeholders in a relatively small sampling of AACSB-accredited, fully online MBA programs in the United States.The results cannot be generalized to any other population, including other online programs.One of the most significant challenges in administering a Delphi study is the time commitment required by participants who must respond to multiple iterations of a survey questionnaire (Cole, Donohoe, & Stellefson, 2013;Linstone & Turoff, 2011).The small sample sizes of administrators (n=7), faculty members (n=5), and instructional designers (n=6) may decrease the power of the statistical analysis related to among-groups differences.However, the size of a Delphi panel can vary widely.The optimal number of Delphi participants is variable, cannot be a statistical decision, and never reaches a consensus in the literature (Hsu & Sandford, 2007;Ziglio, 1996).

Conclusions
Administrators, faculty members, and instructional designers have separate and distinct roles and responsibilities related to the delivery of a quality online MBA program.As a result, each group offers a different perspective when it comes to ensuring quality online education.Despite these differences, all panelists valued AACSB accreditation and overall reported that programs should be accredited in the same way regardless of delivery method and thus, online programs should not be assessed separately.
The results of this study provide insight into what aspects of quality assurance are most important for AACSB-accredited online MBA programs from the perspective of three groups of stakeholders who are integral to the delivery of such programs-administrators, faculty members, and instructional designers.The following recommendations are supported by the literature and by the findings from this study.
• !Identify and implement systems and processes to ensure the academic integrity of online courses and programs.Train faculty members and instructional designers to address academic integrity in the online environment, in the context of both course design and delivery.
• !Structure courses and programs to have equivalent quality standards, admission requirements, learning objectives, course content, and academic rigor regardless of delivery modality while carefully considering how to successfully translate the faceto-face curriculum for online delivery.While the results from this study should not be generalized to represent everyone involved in online MBA programs, they do provide a snapshot of how different groups of stakeholders think about quality assurance in online MBA programs.
support for ongoing course design, development, delivery, and technology.
• !Allocate adequate and ongoing resources for online course design, delivery, and maintenance.• !Develop a common online course template specific to the program that can be customized by the individual faculty member.• !Implement academic technologies that support the objectives and assessment strategies of the program and minimize superfluous use of technology.• !Implement a continuous quality improvement program with ongoing evaluation of online courses and programs.• !Provide faculty development, training and support related to technology, the learning management system, and online course development and teaching.• !Establish uniform standards for faculty qualifications and credentials for online teaching and assign faculty who are comfortable with technology and willing to teach online.• !Explore expansion of the AACSB standards to more completely encompass online learning.• !Provide comprehensive online student support services and clearly communicate the demands and expectations of online learning.

Table 1
Summary of Expert Panel Participation for Each Round

Table 4
Means and Standard Deviations for Themes by Role, in Rank Order

Table 5
Participant Ratings of Items in the Academic Integrity and Rigor Theme

Table 6
Participant Ratings of Items in the Course Content, Design, and Delivery

Table 7
Participant Ratings of Items in the Faculty Qualifications, Development, and Support Theme

Table 8
Participant Ratings of Items in the Quality Frameworks Theme

Table 9
Participant Ratings of Items in the Accreditation Theme