Accountability for Students in K-12 Online Learning : Perspectives from Michigan Stakeholders and Beyond

Policy surrounding K-12 online learning continues to evolve as the field grows exponentially. In Michigan, Section 21f of the State School Aid Act enacted in 2013 strengthened parents’ and students’ ability to request online courses: “A student enrolled in a district in any of grades 6 to 12 is eligible to enroll in an online course as provided for in this section.” The passing of 21f raised concerns around accountability in a choice environment. Examples of such concerns included a pervasive belief about the lack of rigor or quality in online courses, an aversion to another district educating a student for one or two courses yet remaining responsible for that student’s growth, and uncertainty about how mentors and teachers would be evaluated on their online students. Consequently, a legislative directive was issued to the Michigan Virtual Learning Research Institute, the research arm of Michigan Virtual University that centered on accountability. In response to that directive, Michigan stakeholders, as well as experts from other course access states and national organizations, were interviewed to better understand the conversations surrounding accountability in K-12 online learning in Michigan and beyond and to make key recommendations for moving the field forward in an informed way. Data were analyzed using thematic analysis. Implications for research, policy, and practice are shared. Accountability for Students in K-12 Online Learning: Perspectives from Michigan Stakeholders and Beyond Archambault, L., Freidhoff, J., & Kennedy K. (2016). Accountability for Students in K-12 Online Learning: Perspectives from Michigan Stakeholders and Beyond. Online Learning 20 (3), 126 139. Introduction Current educational policy continues to struggle to keep up with the ever-changing landscape, particularly when it comes to online education. Across the nation, states are grappling with how existing policies aligning with traditional education apply to learning in an online environment. Some states have implemented course choice options to enable parents and students to have a greater say in taking courses online. As a result of increased options, one significant area is accountability and how it plays out in K-12 online learning. According to Michigan’s K-12 Virtual Learning Effectiveness Report 2014-15, there has been concern about lower numbers of students completing online courses and higher attrition rates that these courses experience (Freidhoff, 2016). Accountability has been a recent issue in the state of Michigan, so much so that a directive from the state was issued to Michigan Virtual Learning Research Institute (MVLRI), the research arm of Michigan Virtual University, to explore the topic in more detail (MCL Section 388.1698, 2015). In part, the issue of accountability was raised in response to an addition to Section 21f of the Michigan’s State School Aid Act in 2013 that strengthened parents’ and students’ ability to enroll in online courses. The law states: “A student enrolled in a district in any of grades 6 to 12 is eligible to enroll in an online course as provided for in this section.” With this language being added to statute, there were a number of concerns about accountability. A recent statewide analysis (Freidhoff, 2016) revealed almost 446,000 virtual enrollments for the 2014-15 school year. Students who did not take a single virtual course had a pass rate of 91% for their face-to-face courses; in contrast, those who took at least one virtual course (of which there were over 91,000 students) passed their face-to-face courses only 73% of the time. Students’ pass rate dropped to 60% for the virtual courses taken. Ultimately, students who participated in virtual learning in Michigan tended to fare worse (-18%) in their face-to-face courses than non-virtual learning students, and their performance in their virtual courses was also worse (-13%) than their performance in their face-to-face courses (Freidhoff, 2016). As a result, the current study explores the issue of accountability in the K-12 online setting through a policy analysis. Researchers interviewed key experts from course access states and national organizations, in addition to Michigan’s educational leaders, including representatives from the Michigan Association of School Administrators (MASA), the Michigan Association of Secondary School Principals (MASSP), the Michigan Education Association (MEA), the Michigan Department of Education (MDE), and the Michigan Elementary and Middle School Principals Association (MEMSPA). Through this interview and analysis process, the researchers sought to better understand the conversations surrounding accountability in K-12 online learning and, based on those conversations, make key recommendations to Michigan specifically and to the field at large. Related Literature Given the relative newness of course access as a policy/program, there is little existing literature on the topic. What does exist is mostly descriptive in nature, providing an overview and justification for such policy. The following section describes course access, its definition, implementation, and justification. Gemin, Pape, Vashaw, and Watson (2015) define a course access program as those “which allow students to take one or more online courses from a provider other than the student’s district of enrollment and have their funding flow to the provider” (p. 107). Such programs extended school choice policy to the course level. According to Bailey, Martin, Taylor, Leichty, and Palmer (2014), “Course Access programs Accountability for Students in K-12 Online Learning: Perspectives from Michigan Stakeholders and Beyond Archambault, L., Freidhoff, J., & Kennedy K. (2016). Accountability for Students in K-12 Online Learning: Perspectives from Michigan Stakeholders and Beyond. Online Learning 20 (3), 126 139. are intended to provide students with expanded access to educational opportunities by supplementing existing curricular options offered by students’ schools with courses offered by third-party providers in online, blended, and/or face-to-face formats (depending on the state)” (p. 7). States with course access programs authorized by law and implemented include Michigan, Utah, Oklahoma, Texas, Louisiana, Minnesota, Wisconsin, Virginia, Georgia, and Florida (Bailey et al., 2014). States such as these often have a centralized web-based hub that provides families with information on online course choice. With state-level course access programs, there is significant involvement on the part of the state. States with school choice programs typically have a state-approved list of providers from which families can select. While districts should be familiar with online courses as an option for students and communicate this information to families, in actual practice, the degree to which this communication occurs on a regular basis is unknown. Funding typically follows a student when the student takes an online course. However, depending on the state, the amount of funding may be a portion of the regular full-time equivalent (FTE) amount for traditional, face-to-face courses. Often, funding to the online course provider is contingent upon students’ successful completion of the course. The purpose of school choice programs is to increase the ability of students to take classes beyond their local schools. It allows families the authority to enroll in digital learning courses and prevents districts from blocking these enrollments, although districts may impose specific conditions, such as credit limits, ensuring students have not taken the course previously, rigor of course(s), etc. In addition, states commonly ensure the quality of courses by implementing a state-level review of courses and/or providers (Bailey et al., 2014). This is one way in which states with school choice programs or policy have attempted to grapple with issues of accountability. The rationale for providing course access is rooted in efforts to ensure that all students are able to take courses for college and career readiness. Not all students have access to courses intended to prepare them for college or career options. For example, only 50% of high schools in the United States offer calculus to students, while 63% offer physics. In addition, 10-25% of high schools only offer one course in the standard sequence of math (such as algebra I and II, geometry) and science (biology and chemistry) (U.S. Department of Education, 2014). Students of minority backgrounds are more often affected. In fact, 25% of high schools with the largest number of students who are African American and Latino do not offer algebra II, while 33% do not offer chemistry (U.S. Department of Education, 2014). As a result, course access is seen as a way to provide equity in course offerings no matter the students’ socioeconomic status, ethnic group, or locale. The push involves offering the ability to take courses that will prepare students for college as well as career opportunities. These include courses that may be difficult for all districts/schools to offer, including Advanced Placement, International Baccalaureate, world languages, electives such as computer programming or specialized areas such as sustainability and/or other science, technology, engineering, or mathematics (STEM) classes. Students and their families may wish to access online courses for a number of reasons, either to recover credit, advance their studies, graduate on time, or add flexibility to schedules due to external commitments such as work, sports, or family obligations (Worthen & Patrick, 2014). According to a brief issued by the International Association for K-12 Online Learning (iNACOL), “Course Access provides public schools with expanded course offerings across learning environments from diverse, accountable providers” (Worthen & Patrick, 2014, p. 4, emphasis added). However, it is the issue of accountability that has yet to be syst


Introduction
Current educational policy continues to struggle to keep up with the ever-changing landscape, particularly when it comes to online education.Across the nation, states are grappling with how existing policies aligning with traditional education apply to learning in an online environment.Some states have implemented course choice options to enable parents and students to have a greater say in taking courses online.As a result of increased options, one significant area is accountability and how it plays out in K-12 online learning.According to Michigan's K-12 Virtual Learning Effectiveness Report 2014-15, there has been concern about lower numbers of students completing online courses and higher attrition rates that these courses experience (Freidhoff, 2016).
Accountability has been a recent issue in the state of Michigan, so much so that a directive from the state was issued to Michigan Virtual Learning Research Institute (MVLRI), the research arm of Michigan Virtual University, to explore the topic in more detail (MCL Section 388.1698, 2015).In part, the issue of accountability was raised in response to an addition to Section 21f of the Michigan's State School Aid Act in 2013 that strengthened parents' and students' ability to enroll in online courses.The law states: "A student enrolled in a district in any of grades 6 to 12 is eligible to enroll in an online course as provided for in this section." With this language being added to statute, there were a number of concerns about accountability.A recent statewide analysis (Freidhoff, 2016) revealed almost 446,000 virtual enrollments for the 2014-15 school year.Students who did not take a single virtual course had a pass rate of 91% for their face-to-face courses; in contrast, those who took at least one virtual course (of which there were over 91,000 students) passed their face-to-face courses only 73% of the time.Students' pass rate dropped to 60% for the virtual courses taken.Ultimately, students who participated in virtual learning in Michigan tended to fare worse (-18%) in their face-to-face courses than non-virtual learning students, and their performance in their virtual courses was also worse (-13%) than their performance in their face-to-face courses (Freidhoff, 2016).
As a result, the current study explores the issue of accountability in the K-12 online setting through a policy analysis.Researchers interviewed key experts from course access states and national organizations, in addition to Michigan's educational leaders, including representatives from the Michigan Association of School Administrators (MASA), the Michigan Association of Secondary School Principals (MASSP), the Michigan Education Association (MEA), the Michigan Department of Education (MDE), and the Michigan Elementary and Middle School Principals Association (MEMSPA).Through this interview and analysis process, the researchers sought to better understand the conversations surrounding accountability in K-12 online learning and, based on those conversations, make key recommendations to Michigan specifically and to the field at large.

Related Literature
Given the relative newness of course access as a policy/program, there is little existing literature on the topic.What does exist is mostly descriptive in nature, providing an overview and justification for such policy.The following section describes course access, its definition, implementation, and justification.Gemin, Pape, Vashaw, and Watson (2015) define a course access program as those "which allow students to take one or more online courses from a provider other than the student's district of enrollment and have their funding flow to the provider" (p.107).Such programs extended school choice policy to the course level.According to Bailey, Martin, Taylor, Leichty, and Palmer (2014), "Course Access programs are intended to provide students with expanded access to educational opportunities by supplementing existing curricular options offered by students' schools with courses offered by third-party providers in online, blended, and/or face-to-face formats (depending on the state)" (p. 7).States with course access programs authorized by law and implemented include Michigan, Utah, Oklahoma, Texas, Louisiana, Minnesota, Wisconsin, Virginia, Georgia, and Florida (Bailey et al., 2014).States such as these often have a centralized web-based hub that provides families with information on online course choice.With state-level course access programs, there is significant involvement on the part of the state.States with school choice programs typically have a state-approved list of providers from which families can select.
While districts should be familiar with online courses as an option for students and communicate this information to families, in actual practice, the degree to which this communication occurs on a regular basis is unknown.Funding typically follows a student when the student takes an online course.However, depending on the state, the amount of funding may be a portion of the regular full-time equivalent (FTE) amount for traditional, face-to-face courses.Often, funding to the online course provider is contingent upon students' successful completion of the course.The purpose of school choice programs is to increase the ability of students to take classes beyond their local schools.It allows families the authority to enroll in digital learning courses and prevents districts from blocking these enrollments, although districts may impose specific conditions, such as credit limits, ensuring students have not taken the course previously, rigor of course(s), etc.In addition, states commonly ensure the quality of courses by implementing a state-level review of courses and/or providers (Bailey et al., 2014).This is one way in which states with school choice programs or policy have attempted to grapple with issues of accountability.
The rationale for providing course access is rooted in efforts to ensure that all students are able to take courses for college and career readiness.Not all students have access to courses intended to prepare them for college or career options.For example, only 50% of high schools in the United States offer calculus to students, while 63% offer physics.In addition, 10-25% of high schools only offer one course in the standard sequence of math (such as algebra I and II, geometry) and science (biology and chemistry) (U.S.Department of Education, 2014).Students of minority backgrounds are more often affected.In fact, 25% of high schools with the largest number of students who are African American and Latino do not offer algebra II, while 33% do not offer chemistry (U.S.Department of Education, 2014).As a result, course access is seen as a way to provide equity in course offerings no matter the students' socioeconomic status, ethnic group, or locale.The push involves offering the ability to take courses that will prepare students for college as well as career opportunities.These include courses that may be difficult for all districts/schools to offer, including Advanced Placement, International Baccalaureate, world languages, electives such as computer programming or specialized areas such as sustainability and/or other science, technology, engineering, or mathematics (STEM) classes.Students and their families may wish to access online courses for a number of reasons, either to recover credit, advance their studies, graduate on time, or add flexibility to schedules due to external commitments such as work, sports, or family obligations (Worthen & Patrick, 2014).According to a brief issued by the International Association for K-12 Online Learning (iNACOL), "Course Access provides public schools with expanded course offerings across learning environments from diverse, accountable providers" (Worthen & Patrick, 2014, p. 4, emphasis added).However, it is the issue of accountability that has yet to be systematically explored and is in need of further inquiry.

Methodology
The current study used stratified purposeful selection of cases (Patton, 2002).Specifically, the researchers chose accountability practices of school choice programs from Louisiana, Minnesota, and Texas to provide examples as to how these states have dealt with issues of accountability.State-level representatives and stakeholders from all three states were interviewed to glean information and recommendations regarding how their school choice programs have dealt with aspects of ensuring student success and online course quality as part of accountability.Then, the study took a deeper dive into Michigan.To gather data, key state-level representatives were interviewed.These included individuals from the Michigan Association of School Administrators (MASA), the Michigan Association of Secondary School Principals (MASSP), the Michigan Education Association (MEA), the Michigan Department of Education (MDE), and the Michigan Elementary and Middle School Principals Association (MEMSPA).The interviews were semi-structured.The semi-structured nature of the interviews allowed for the use of prompts and follow-up questions if an answer required clarification.Each interview lasted between 45 and 90 minutes.Interview questions related to accountability in the K-12 online learning setting as well as to state level policy surrounding the topic (Appendix A).In addition to interview data, state level policies were reviewed by the researchers.As a supplement to sharing information about the conversations that were happening around the state amongst educators, interviewees provided background on the issue as well as their perspectives regarding efforts to ensure accountability in online settings.They also discussed the feasibility and impact of implementing accountability policy itself in their specific contexts (Karger & Stoesz, 2009).The analysis of interview data along with state level policy allowed the researchers to better understand accountability and make meaningful recommendations.
Open codes were assigned to each of the interviews throughout the duration of the study.The two lead researchers read the data and coded, focusing on excerpts that centered on accountability, related policy, its implementation, and impact.Open coding of the qualitative responses began with an initial read-through and theme identification for all responses.Then, the researchers discussed recurring themes until consensus was reached.Any initial disagreements were resolved through iterative dialogue and rechecking and re-confirming of data between the lead researchers.Subsequently, a final round of coding was conducted by the researchers in which all of the participants' responses were re-coded using the determined set of themes.The data were then reviewed to see what themes were consistent across cases, a process that provided the researchers a way to identify common themes within the interviews.The third researcher then reviewed themes and their application to check for overall consistency.
Validation of qualitative research relies more on trust and authenticity rather than relying on statistical measures (Denzin & Lincoln, 2005;Maxwell, 1998).The researchers in this study used member checks and feedback to help ensure transferability, dependability, and confirmability of the data.After data were collected, analyzed, and conclusions were written, the researchers asked stakeholders to review the findings.This allowed the researchers to safeguard the authenticity of the data (Guba & Lincoln, 1989;Miles & Huberman, 1994).In the next section, the data is presented in narrative form to provide a holistic look at accountability in the K-12 online context.

Analysis
Each of the state-level school choice programs are described below including Louisiana, Texas, and Minnesota, followed by an in-depth examination of the issue within the state of Michigan.

Louisiana
From interviews with Louisiana state officials, it became evident that Louisiana has a wide array of virtual, blended, and face-to-face course offerings through Louisiana's Supplemental Course Academy (SCA) program.High school as well as advanced middle school students can participate in the program to take academic and career and technical education (CTE) courses, including dual enrollment offerings that enable students to earn high school and college credit at the same time.Students can take online classes from a variety of providers approved by the state.However, the courses from which they can select are subject to approval.Courses must be (1) academically appropriate, (2) logistically feasible, and (3) keep students on track for graduation.Funding is distributed such that approved course providers receive only half of the funding upon enrollment and the other half upon successful student completion (Watson, Pape, Murin, Gemin, & Vashaw, 2014).Local Education Agencies (LEAs) in Louisiana now control how they expend their Course Choice / SCA funds as a result of changes to the original course choice law.Consequently, district administrators have the final decision-making authority on how funding is spent when selecting among approved courses.
Accountability efforts in Louisiana largely centered on student success.For supplemental online courses, accountability remains with the local district.Louisiana has taken a number of steps to assure course quality, the chief of which involved instituting an application process for approval of providers and courses.Within the Louisiana Department of Education, there is one dedicated staff member as well as additional part-time employees to monitor online provider quality, register students, and track student performance.The application review process evaluates course provider background credibility, capacity, and prospects for student success.Applicants who receive a favorable evaluation of their application then present the proposed course content and demonstrate their capability as a provider to a three-person interview team.In addition, there is an external expert review process, followed by a review conducted by the State Board of Education.
Approval as a course provider does not automatically ensure that individual courses are accepted.The provider, after approval, must then submit proposed courses for review.The Louisiana Department of Education then evaluates the quality and suitability of each course.Ultimately, the Course Choice Chief Academic Officer must approve each proposed course.Course providers may add courses in future years, especially if they have a positive track record and the courses meet quality standards.Louisiana retains the right to put course providers on probation and/or terminate them if it is determined that they are not performing based on the detailed course provider agreement.
Another measure of accountability in the state of Louisiana is the requirement that all students pass End of Course (EOC) exams in algebra I or geometry, English II or III, and biology or United States history in order to graduate.Students are required to earn at least a score of "Fair," indicating that the student has demonstrated the fundamental knowledge and skills needed for the next level of coursework.To be able to graduate, a score of "Fair" or higher must be achieved on three End-of-Course exams, one from each category.These exams are administered three times a year and comprise between 15% and 30% of the student's final grade, as determined by the local school district.

Texas
From discussions with officials representing the Texas Virtual School Network (TxVSN), the researchers learned that the network offers supplemental and fully online options for local districts.For supplemental courses, TxVSN provides a course catalog from which students can select up to three approved online courses per semester.In general, students must be younger than 21 years of age, must not have graduated from high school, and must be eligible to enroll in a Texas public school.Students can take online courses for credit recovery or initial credit including Advanced Placement and dual credit to be able to meet high school graduation requirements.They can choose to complete TxVSN courses during their regular school day or off campus at their own convenience.
To understand accountability in Texas, it is helpful to gain insights into the financial arrangements of TxVSN courses.For supplementary cases, the student's school district pays the TxVSN for the course, and then the TxVSN in turn pays the course provider.Assuming that the student successfully completes the courses, state funding can be generated for up to three TxVSN courses per semester.If the three TxVSN courses are taken as part of the student's normal course load, then the district pays the cost of the courses.If the student wants to take additional TxVSN courses, the district may decline the course cost.In this situation, families cover the cost of the course.
For fully online students in elementary grades (3-8), the instructional program must be structured so that the level of student progress justifies promotion to the next grade level.Once this requirement is met and the online school earns full average daily attendance, it gets full funding.However, no funding is provided if the student does not make enough progress to advance to the next grade level.For high school, successful completion is on a course-by-course basis.If a student successfully completes three or four TxVSN courses, half-time funding is awarded.If a student successfully takes five or more courses, full-time funding is provided.One TxVSN online course counts as 55 minutes per day per course of instruction.Half-time is equivalent for the funding of two hours of instruction per day, while full-time funding is considered the equivalent allowed for four hours of instruction per day.In either case, whether or not the student successfully completes the course is what matters.
With this model in place, tying funding to successful student achievement, the Commissioner of Education in the State of Texas ensures quality online programs.Similar to Louisiana, there is an approval process overseen by the Texas Education Agency (TEA) to meet eligibility requirements.The TEA also evaluates courses on the basis of three different sets of standards, including state curriculum standards, iNACOL Standards for Quality Online Courses, and TxVSN accessibility guidelines.
In addition to course approval, one of the distinctive elements in Texas is that there must be an agreement among the various stakeholders.If a provider wishes to offer courses, or a district wants to enroll students, each must sign an agreement that specifies their rights and responsibilities.For providers, they must certify that they have the legal right of ownership to be able to offer the course, they agree to submit their course for review, and they cannot offer the course until it is approved.They agree to specific class size limits and offer the course for four consecutive semesters.In addition, providers agree that teachers must be certified in the state of Texas and must have taken TxVSN-approved professional development in effective online teaching.School districts agree that they are responsible for mentoring/monitoring online students for the duration of the course as part of the agreement.Districts must also provide a site coordinator who registers students, makes sure prerequisites have been met, and mentors and monitors student progress.Districts also agree to accept the grade and transfer it to the student's transcript using local grading policy.Also similar to Louisiana, Texas uses EOC exams.While EOC exams can help to ensure accountability, one of the challenges in using them is their administration for online students.Although online learning provides flexibility, as a result, students may be taking courses out of sequence, such as for credit recovery or acceleration.Some students may be taking courses typically offered during the second semester within another semester of the school year (fall or summer).Consequently, EOC testing may be more difficult.There is also a challenge in proctoring state-level tests for fully online students.Students in outlying areas may need to test, and a local school may not wish to deal with having to give the assessment.Ultimately, TxVSN has the responsibility to ensure that the EOC is administered.
TxVSN continues to work to make improvements to ensure quality and accountability specific to online instruction across the state.In a recent needs assessment regarding online courses in Texas, three major issues were identified: 1) lack of awareness on the part of districts, parents, and students as to what online options were available, 2) questions about funding in terms of how to pay for online courses, and 3) fear of losing local control over curriculum/content/instruction on the part of districts.These issues are indicators of the hesitancy of districts when it comes to online education.
Of the approximately 1,200 districts in Texas, less than 300 have signed agreements with online providers.Signed agreements are not mandated, but they are required in practice.A district cannot enroll a student in an online course without signing the agreement.These concerns have also prompted recent changes.Local districts can now specify the TxVSN provider they approve for their students.In making this decision, TxVSN encourages districts to consider data from successful course completion rates by a specific provider, both overall as well as by specific individual courses, along with taking into account feedback rankings on courses from students and parents provided in the TxVSN course catalog.

Minnesota
One of the veteran states in school choice is Minnesota.Since 1985, Minnesota has offered its students school choice options including open enrollment, charter, and post-secondary concurrent enrollment.When these options became available, there was concern from districts as to what students were learning, the rigor and quality of the courses, the level of qualification of the teachers, and the levels of attendance within the courses.As a result, accountability has always been central to the discussion about school choice.
Minnesota was one of the first states to enact policy to allow students the choice to take an online course from various school-based providers.Students in Minnesota can take supplemental online courses from providers that are approved by the Minnesota Department of Education.Currently, there are a variety of supplemental providers, including charters and district-level providers.As far as the financial model goes, 88% of funding goes to the provider per course, while 12% stays at the local district.
According to state officials, accountability within the state is based on "inputs and outcomes."Online courses are required to meet state academic standards.When districts reported the perception that online courses lacked rigor, wording in the Minnesota legislation was changed.This made it so that the course meeting state academic standards must be "verified" rather than "affirmed."Every core course provider had to complete an extensive alignment document verifying the curriculum, how it was being taught, and what assessments were being used to qualify that the standards were being met.
At that point, accountability was focused more on inputs rather than student outcomes.There are now multiple measures to ensure quality, including a review process in which the provider application is reviewed by four reviewers, a team that includes individuals at the department level as well as those in the field.Outcome variables are used to measure how students are growing in their learning process.These variables include examining proficiency, student growth, and graduation rate.Although it varies due to a variety of factors, the successful completion rate is typically about 70%.This rate must be reported to the Minnesota Department of Education.
Minnesota also relies on state level assessments to ensure accountability.This can be difficult, however, because the state assessments in English/language arts and math are given only once a year.As a result, it can be difficult to separate out how specific online courses may or may not have had a direct impact on student performance.While EOC exams have been considered, this option has not taken hold for fear of the unintended consequences they might cause.One concern among districts is the disparity that may exist between providers and the difficulty in developing a common EOC exam, particularly for certain content areas such as liberal arts, social studies, and language arts.Another measure of accountability in Minnesota is the statute that the local district must designate a contact person to facilitate and monitor student progress in the online course as well as track his/her accumulated credits toward graduation.Typically, this role falls on the school counselor.Online teachers in Minnesota are hired by the online program and work with this contact person who is on-site at the local school.While schools are required to identify a contact person, they are not required to assign a mentor.
Each district looks at this contact person very differently, and unfortunately, some districts do not express a desire to become involved.
As with Texas, the awareness of online learning options is limited within the state.Despite this, because the courses are provided by local school districts and overseen by curriculum directors and school boards, schools are comfortable with online students.Even if a national vendor is offering a course, a district oversees the course through the adoption/review process.This oversight provides the opportunity for more local control, and the school districts have more "home-grown" content with local teachers teaching the courses.

Examining Online Learning Accountability in Michigan
After interviewing state officials from the major state educational organizations, a more in-depth understanding of how accountability affects school choice in Michigan was made evident.It was interesting to examine how this issue became significant in the state and to explore what caused a directive to be established to more closely investigate the issue.With the passing of Section 21f of the State School Aid Act in 2013, Michigan became a course access state.This measure allowed eligible students to take up to two courses per academic term if the courses were in compliance with a mandated set of criteria.The law requires districts to allow students to take online courses and for the courses to be paid for by the district.With the enactment of 21f, districts and teachers raised concerns specific to accountability.
The first concern was the question of who should be held accountable for the student when the student opted to take an online course from an outside entity, such as another district, an intermediate school district (ISD), or the Michigan Virtual School (MVS).The primary district was held accountable for the student's performance on standardized measures that factored into a district's accountability scorecard and the district's ability to make adequate yearly progress (AYP), even when the student was taking a course that was provided externally.Districts were concerned about the prospects of underperforming on these important measures, especially if underperformance was the result of an outside provider educating its students.
Another worry involved how online teachers would be evaluated.In Michigan, there is a mandated requirement that students taking online courses be assigned an on-site mentor.This on-site mentor is employed by the school district.His/her role is not to teach, but rather to mentor and support the online student during the course.In practice, however, many online students are enrolled in courses in which there is no online instructor.Another issue that happens is when an outside vendor provides the instructor, but the district does not have the personnel identification code (PIC) for this instructor.In such cases, a district often records the mentor's personnel identification code (PIC) both as the mentor of record as well as the teacher of record.Many have expressed concerns about the possible repercussions this may have on their teacher effectiveness scores, which could have a possible impact on promotion and retention.Some districts argued that for students taking online courses, accountability should be removed from the primary district and become the responsibility of the providing district.However, this position was weakened by key factors.First, the amount of money districts, ISDs, and MVS were charging for the online courses was substantially less than the pro-rated amount the district received from the state to educate each student.In addition, the primary district kept a portion of that money in-district.Second, the 21f legislation grants a primary district the ability to block an online enrollment request for a number of reasons, including the perception that the online course lacks rigor or quality.As a result, the primary district has the ability to help their students select online courses that are of perceived high quality and taught by high-quality teachers, alleviating some of the concern about student learning.Finally, the district also has the responsibility to provide the on-site mentoring to support and ensure that the student has a quality learning experience.Shared funding, course selection, and on-site mentoring are factors that create a strong case for shared accountability for online students.It is clear that districts have a responsibility to help students select quality and rigorous online courses.Available online course options in Michigan are published in a statewide catalog (https://www.micourses.org).Districts control what comes into the statewide catalog because all courses in the catalog must be approved by a public school district, ISD, or MVS in order to be included.Unlike several other course access states, such as the ones previous described, the state does not play a role in determining which online providers and courses are approved for inclusion.
As of 2015, ISDs offered about 50% of the courses in the catalog.The content for the courses offered by the ISDs was predominately from a third-party provider and used teachers also provided by the outside provider.About a third of the courses in the catalog were added by local districts.Similar to ISDs, over 90% of the courses used content provided by third-party providers.However, local districts assigned their own teachers to the courses for about two-thirds of the course titles.
To assist districts, parents, and students in choosing among multiple online course options, all online courses displayed in the statewide catalog contain a rating for each of the 52 iNACOL standards for quality online courses (iNACOL, 2011).Student performance data that includes the number of students who enrolled in the online course in each previous year, as well as the number of those enrollments that earned 60% or more of the total course points, is provided.In addition, information from syllabi is also included.
Unfortunately, there are currently no external measures provided in the catalog that offer an independent perspective on student competency, despite the fact that student performance data on how well students perform based on measures internal to each course is provided.For instance, Michigan does not currently implement end-of-course (EOC) assessments, as do some other school choice states.While there was an effort made by the Michigan Department of Education to create and implement EOC exams (referred to as Interim Assessments), the funding to support the effort was discontinued while the state was enmeshed in the Common Core State Standards adoption debate.Some districts are afraid that the final course grade may be misleading if an independent measure at the end of the course is lacking.The worry is that a lack of student performance will be discovered through standardized test scores administered by the state.At this stage, however, it would be too late to remedy the situation, and the school itself may be penalized based on the student's poor performance.Despite these concerns, stakeholders reported that there are those in the state who have embraced online learning, conveying, "We can do this, and kids should have an online experience before they graduate" and "We'll train teachers to teach online and help mentors."At the same time, those same stakeholders acknowledge the following: "You still have a majority [of districts] that want to hide [the online learning option] on the last page of their course description book and try to find reasons to deny the opportunity."

Implications and Recommendations
Given these varying cases, what options do stakeholders identify to begin addressing accountability in K-12 online learning?Based on the researchers' discussions with educational officials from school choice states, the following recommendations were gleaned: 1. Schools, districts, and states should work to clearly communicate expectations and promote awareness of online learning course choice options for K-12 students.
2. Districts should work to establish student support structures for online settings.
3. Specific guidelines for teacher-of-record and mentor-of-record roles should be established.4. States should consider changes to state reporting guidelines to improve available data for evaluation purposes.5. States should work to maintain an up-to-date course catalog with comprehensive information about online course options.6. Online providers should provide additional alignment information within course syllabi 7. Consider the use of end of course (EOC) Exams for consistent measures across online course providers.8.The course review process should be assessed annually and updated or revised as appropriate Each recommendation is shared and discussed below.

Clear Communication of Expectations and Awareness
Despite states having long histories with online learning policy and practice, stakeholders mentioned that many school districts and parents were not sufficiently aware of students' options to take online courses.A great need remains to communicate and promote awareness of course choice options in online learning for K-12 students.A practical starting point for increasing awareness is to include clear language and information about online learning options within district and school handbooks.States also need to share information with parents and districts about how to determine what quality online learning is and what it is not using common, clear language that is easy to understand.
In addition, officials believe there should be clear quality assurance measures for all online providers.They stressed the importance of these measures being communicated to the public.Having clear contracts that outline the responsibilities of each party to help students succeed would be extremely beneficial.This communication should include sharing principled practices and resources that are necessary to help students.Also, professional development for administrators, teachers, mentors, parents, students, and others should be provided.Resources should be provided so that all parties realize for what and how they are being held accountable.Mentors and counselors also need guidance as to which learning environment will best meet student needs.Consistent standards should be used to ensure quality.For example, Quality Matters (QM) can serve as a baseline, and QM training would be ideal for stakeholders, especially administrators, teachers, and mentors.Districts need to understand what qualities to look for in effectively designed online courses and also understand what support systems need to be implemented in order to provide the best support possible for students.Ultimately, empowering districts to work with parents and offering students the best choice for their learning path is less about data and high stakes accountability and more about transparency.
Based on the context above, key recommendations are listed below:  Stakeholders should work together to inform both districts and course providers about their responsibilities to online students.Additionally, a similar process to that employed by TxVSN-holding districts and online providers accountable via a clear contract that outlines each stakeholder's rights and responsibilities to the online student-might be considered. School districts should inform parents of their options to take online courses  School districts should require a signed agreement from parents that acknowledges their rights and responsibilities relative to their child's enrollment in an online course.
 School districts and course providers should work together to provide professional development specific to online learning for administrators, teachers, mentors, counselors, and students. Course providers should offer an online learning orientation for students to identify potential areas where students may need additional support. School districts should familiarize themselves with the iNACOL and Quality Matters standards specific to quality online programs, online course design, and online teaching and use these standards to evaluate online course provider options.

Consistency and Value of Student Support Structures
Local level student support structures are critical.Stakeholders felt that the primary district should be held accountable for providing students with supports they need to be successful in the online learning environment.As one of the stakeholders noted, Online learning is often most used as a remediation rather than acceleration.And when we're using it as a remediation, the kids aren't getting a lot of support.They've struggled before, and it's no different when they take an online course.So the districts should have some responsibilities like a mentor who can provide some support and reinforcement.
However, the stakeholder also acknowledged that these support structures come with an associated cost, together with an investment of time and infrastructure to make them work.As a result, one of the key recommendations is that school districts should invest in mentoring and student support programs for online students that include professional development for mentors.Also, districts need to document mentoring services to ensure that they are taking place and that they are continuously examined for their effectiveness.

Specific Guidelines for Teacher-of-Record and Mentor-of-Record Roles
Stakeholders believed it important that schools accurately report teacher-of-record and mentor-ofrecord information.The new language requires the sharing of needed information but does not guarantee that schools will change their reporting conventions.Ongoing awareness campaigns clarifying the difference between these two roles and stressing the importance of accurately capturing who serves in each capacity will be essential to changing reporting practices.In addition, it would help to document the number of online students and teachers, a statistic in the literature that is currently difficult to accurately depict.
Based on the context above, the key recommendations are listed below:  Continue defining and clarifying the roles of teachers-of-record and mentors-of-record.
Include in these roles the need for consistent monitoring and interaction to assist online students. Continue to spread awareness about the importance of accurate data regarding the mentor-ofrecord and the teacher-of-record. School districts should set up quality assurance measures and a training process around the importance of accurate data reporting for online students.

Changes to State Reporting Guidelines
Stakeholders expressed an overall need to improve the quality and comprehensiveness of the readily available data to evaluate the performance of both individual online courses as well as individual course providers.Given that districts are responsible for ensuring that the online courses their students select are rigorous and high quality, more information to inform such a decision is needed.In addition to being able to view data at the individual course level, districts would be better served if they were also able to look at aggregated performance data for an online provider.Thus, the following recommendations are offered:  When reporting student course information, it is recommended that schools be required to report the following: o The name of the entity offering the online course (e.g., district or third-party provider).o The name of the company, organization, or district that is providing the actual content for the online course.o The name of the company, organization, or district that is supplying the online instructor for the online course. Using the data collected above and elsewhere, the state should make transparent both the number of enrollments served by online course providers as well as student performance for each provider.Where possible, additional information, such as AP exam pass rates should accompany this information. In addition to having schools report data on online enrollments, explore ways to have online course providers provide similar data directly to the state.

Updates to Course Catalogs
As mentioned above, the idea of supplementing course performance data for individual courses with aggregated views of course providers were positively received.Stakeholders also expressed interest in capturing student and parent ratings, similar to that of TripAdvisor or Yelp, allowing students and parents to comment on specific courses.Such a feedback process is already in place in Texas and can be seen in the TxVSN catalog (see https://catalog.mytxvsn.org/).As states develop more comprehensive systems for tracking online providers, they may want to consider requiring that a link to the state data on each provider be part of the information available to parents and students when choosing online courses.States may also wish to incorporate a small set of course ratings for each course to be completed by students who take the course, as well as to collect feedback from their parents.

Additional Alignment Requirement of Course Syllabi
Another suggestion was to add alignment documents to online course syllabi.Stakeholders felt that alignment documents should be required because they provide critical information to districts that help them determine whether a course is of sufficient quality and rigor.It also helps districts distinguish the degree to which the topics covered in an online course best align with the topics presented in the district's face-to-face version of the same course.Michigan officials even mentioned that a course alignment document would facilitate the targeting of certain units or lessons to make up if students are repeating the course or have shown proficiency in other course topics.

Pilot the Use of Michigan Interim Assessments for Online Courses as End of Course (EOC) Exams
To target the need for consistency across courses, End of Course (EOC) exams could be used to measure what a student has learned.For many states, there are no independent, external measures of student competency to verify the accuracy of the course grade.While completion rates act as a minimum level of accountability, a common measure of course performance would allow districts and states to better benchmark student learning and provide important data for comparing the performance of various course providers.
One potential drawback for EOC exams is that they are typically not available for every course, and for cost purposes, may be limited to core courses required for graduation.The use of EOC exams is already in place in some states, such as Florida.In Florida, they are used as a mechanism to increase student achievement as well as to document readiness for college and career.Florida's EOC exam results are available on their Florida Department of Education website (see http://fcat.fldoe.org/resultsEOC/).Louisiana also has EOC exams in place that are offered three times a year.The recommendation is to have the EOC exams available at the end of every semester for students who take a particular subject and need to be assessed at the time they complete the course.Additionally, there is concern regarding universal acceptance of EOC exams based on whether or not each stakeholder is in agreement of the rigor and quality of the exam itself.This, coupled with the growing movement against excessive testing, may present additional challenges for using EOC exams as a mechanism for ensuring accountability in the K-12 online settings.

Updates to Course Review Process
Stakeholders believed that there was a desire to see greater consistency in how the courses were being reviewed, specifically as it pertained to a shared competency of course reviewers.Educational personnel who are course reviewers should be trained on how to conduct online course reviews.The shared training would help ensure that reviewers know what they are looking for, share a common vernacular, and employ a common process for considering quality assurance in online courses.In addition, a third party could provide a formal review that could yield a seal of approval.For example, Quality Matters (QM) is a national organization that conducts in-depth evaluations of courses and certifies high-performing courses with a recognized symbol.The idea of helping to differentiate courses based on an independent seal of approval was seen as advantageous for helping to prove that the online course is rigorous and of high quality.However, the expenses associated with engaging in such an intensive review process would be a general concern with this recommendation.

Conclusion
While this study highlighted many recommendations on accountability for K-12 online learning, it represents a start to the conversation.It is certainly a dialog that needs to continue.As one stakeholder mentioned, When thinking about accountability, the difficulty is not just online and blended; it's the collective advent of shared models and unique models of how students are educated, and what are we going to do differently accountability-wise to make sure the student is learning what he/she needs to learn.
Another stakeholder relayed the sentiments for the collective: Let's incentivize the behavior and support the practice rather than change the metric.If you change the compliance function and you don't change the behavior around it, then it's not going to actually result in a change for kids.If we're going to do anything about accountability, it is a piece of a bigger solution but it's not the solution.Changing that policy lever, we have to consider what other pieces have to change along with it and what behavior we want to incentivize.It's not more online learning; instead, it's more quality online learning in order to arrive at better outcomes for students.
Accountability should be communicated as dynamic in real-time, and should lead to continuous improvement.The focus should be on individual students and how educators are meeting the needs of each student in order to help them succeed in his/her educational endeavors.The legitimacy of online learning depends on implementing accountability measures, continuously monitoring and improving them, and acting to make course improvements where needed.The ultimate goal is to safeguard student learning and progress.With this in mind, the recommendations from this initial exploration are offered as a starting point to help states examine where realistic measures might be implemented as mechanisms to ensuring student success.