The Online Learning Consortium Quality Scorecard for K12 Blended Learning
DOI:
https://doi.org/10.24059/olj.v30i3.6441Keywords:
Quality K12 education, K12 blended learning, K12 digital learning, quality blended learning, Delphi method, blended learning program evaluationAbstract
As K12 blended learning continues to expand, a K12-specific framework for evaluating program quality was not found. While rubrics such as Quality Matters and iNACOL address course-level design, no comprehensive scorecard exists for the broader design, leadership, implementation, and administration of K12 blended programs. This study used a four-round Delphi methodology to build expert consensus for adapting the Online Learning Consortium's 2023 Quality Scorecard for Blended Programs, originally developed for higher education, for use in K12 settings. A panel of 20 U.S.-based experts in K12 blended teaching, administration, and research evaluated 65 quality indicators across nine categories, voting on relevance, proposing new indicators, and refining language for a K12 audience using a 6-point Likert scale with consensus defined as M ≥ 5.0 and 70% panel agreement. The resulting OLC K12 Blended Learning Scorecard organizes quality indicators into eight categories: School District Support, Technology Support, Course Development and Instructional Design, Course Structure, Teaching and Learning, Teacher Support, Student Support, and Evaluation and Assessment. Findings show quality blended learning in K12 settings depends on the combination of organizational leadership, infrastructure, instructional design, pedagogy, and ongoing evaluation, rather than an isolated intervention. This study contributes a practical, research-informed tool to guide K12 educators, administrators, and policymakers in designing, implementing, and evaluating blended learning programs, and offers a foundation for future research on scorecard implementation, blended teacher training, and blended learning program evaluation.
References
Adelstein, D., & Barbour, M. (2016). Building better courses: Examining the construct validity of the iNACOL National Standards for Quality Online Courses. Journal of Online Learning Research, 2(1), 41–73. https://doi.org/10.70725/019085jzqdar
Al-Busaidi, K. A. (2012). Learners’ perspective on critical factors to LMS success in blended learning: An empirical investigation. Communications of the Association for Information Systems, 30. https://doi.org/10.17705/1CAIS.03002
An, Y., Kaplan-Rakowski, R., Yang, J., Conan, J., Kinard, W., & Daughrity, L. (2021). Examining K-12 teachers’ feelings, experiences, and perspectives regarding online teaching during the early stage of the COVID-19 pandemic. Educational Technology Research and Development, 69(5), 2589–2613. https://doi.org/10.1007/s11423-021-10008-5
Angawi, M., & Tasir, Z. (2024). Blended learning research: A systematic review and identification of future research gaps. International Journal of Academic Research in Progressive Education and Development, 13(3). https://doi.org/10.6007/ijarped/v13-i3/22216
Archambault, L. & Kennedy, K. (2018). Teacher preparation for K-12 online and blended learning. In K. Kennedy & R. E. Ferdig (Eds.), Handbook of research on K–12 online and blended learning (2nd ed., pp. 221–245). Carnegie Mellon University, ETC Press.
Anthony, E. (2019). Blended learning: How traditional best teaching practices impact blended elementary classrooms. Journal of Online Learning Research, 5(1), 25-48. https://doi.org/10.70725/310262fgtpwl
Azukas, M. E., Garrett Dikkers, A., & Viano, S. (2026). Proximal and auxiliary support systems in K–12 digital learning. Journal of Online Learning Research, 12(1), 73–96. https://doi.org/10.70725/898687otxwsq
Baca, A., Creghan, C., & Maninger, R. M. (2019). Teacher perceptions of the effectiveness of a blended learning program at a public charter middle school (5th–8th Grade) in a large city in Texas. Journal of Multidisciplinary Graduate Research, 4(1). https://jmgr-ojs-shsu.tdl.org/jmgr/article/view/3
Barbour, M. K., Borup, J., & DeBruler, K. (2026). Mind the (digital) gap: Building a practitioner-centered research agenda for K–12 online and blended learning. Journal of Online Learning Research, 12(1), 1–13. https://doi.org/10.70725/988738sldkrs
Barbour, M. K., Clark, T., DeBruler, K., & Bruno, J. A. (2014). Evaluation and approval constructs for online and blended courses and providers. Michigan Virtual University. /research/publications/michigan-virtual-report-examines-online-learning-policies-and-practices/
Barrios, M., Guilera, G., Nuño, L., & Gómez-Benito, J. (2021). Consensus in the Delphi Method: What makes a decision change? Technological Forecasting and Social Change, 163, 120484. https://doi.org/10.1016/j.techfore.2020.120484
Basham, J. D., Smith, S. J., & Satter, A. L. (2016). Universal Design for Learning. Journal of Special Education Technology, 31(3), 147–155. https://doi.org/10.1177/0162643416660836
Belton, I., MacDonald, A., Wright, G., & Hamlin, I. (2019). Improving the practical application of the Delphi method in group-based judgment: A six-step prescription for a well-founded and defensible process. Technological Forecasting and Social Change, 147, 72–82. https://doi.org/10.1016/j.techfore.2019.07.002
Bergdahl, N., & Bond, M. (2021). Negotiating (dis-)engagement in K-12 blended learning. Education and Information Technologies, 27(2), 2635–2660. https://doi.org/10.1007/s10639-021-10714-w
Chanenson, J., Sloane, B., Rajan, N., Morril, A., Chee, J., Danny Yuxing Huang, D., & Chetty, M. (2023). Uncovering privacy and security challenges in K-12 schools. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. ACM, Hamburg Germany, 1–28.
Cheng, C., Kirby, C. K., Fu, J., Kelly, S., & Vandepol, N. (2023). Instructor challenges, strategies, and satisfaction in blended courses: A case study. TechTrends. https://doi.org/10.1007/s11528-023-00884-4
Consortium for School Networking [COSN]. (2025). Driving K-12 innovation. https://www.cosn.org/wp-content/uploads/2025/02/2025_CoSN-Driving-K12-Innovation-Report-V15.pdf
Dalkey, N., & Helmer, O. (1963). An experimental application of the DELPHI Method to the use of experts. Management Science, 9(3), 458–467. https://doi.org/10.1287/mnsc.9.3.458
Darling-Aduana, J., Heath, M. K., Steward, A. Viano, S., Asim, S., Garcia, A., Langran, E., Heinrich, C., & Woodley, X. (2025). Digital ethics and equity in K-12 blended and online Learning. In D.A. Christakis & L. Hale (Eds.) Handbook of children and screens. Springer. https://doi.org/10.1007/978-3-031-69362-5
Dawson, K., & Dana, N. F. (2018b). Professional development for K–12 online teachers. In K. Kennedy & R. E. Ferdig (Eds.), Handbook of research on K–12 online and blended learning (2nd ed., pp. 247–260). Carnegie Mellon University, ETC Press.
Diep, A.-N., Zhu, C., Struyven, K., & Blieck, Y. (2017). Who or what contributes to student satisfaction in different blended learning modalities? British Journal of Educational Technology, 48(2), 473-489. https://doi.org/10.1111/bjet.12431
Drexler, W. (2018). Personal learning environments in K–12. In K. Kennedy & R. E. Ferdig (Eds.), Handbook of research on K–12 online and blended learning (2nd ed., pp. 151–162). Carnegie Mellon University, ETC Press.
EdChoice. (2026). School moves survey: School switching in America [Report]. https://www.edchoice.org
Fox, E. (2024). Novice K-12 online teacher support. Journal of Online Learning Research, 10(1), 15–48. https://doi.org/10.70725/327969pefoha
Garrison, D. R., & Vaughan, N. D. (2008). Blended learning in higher education: Framework, principles, and guidelines. Jossey-Bass/Wiley.
Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105.
George, A. (2025). Digital equity in blended learning: Closing achievement gaps in underserved communities. Eduschool Academic Research and Publishers. https://doi.org/10.5281/ZENODO.17707403
Graham, C. R. (2013). Emerging practice and research in blended learning. In M. G. Moore (Ed.), Handbook of distance education (3rd ed., pp. 333–350). Routledge.
Halverson, L. R., Spring, K. J., Huyett, S., Henrie, C. R., & Graham, C. R. (2023). Blended learning research in higher education and K–12 settings. In J. M. Spector, B. B. Lockee, & M. D. Childress (Eds.), Learning, design, and technology (pp. 3107–3135). Springer International Publishing. https://doi.org/10.1007/978-3-319-17461-7_31
Hanny, C. N., Arnesen, K. T., Guo, Q., Hansen, J., & Graham, C. R. (2023). Barriers and enablers to K–12 blended teaching. Journal of Research on Technology in Education, 55(4), 568–589. https://doi.org/https://doi.org/10.1080/15391523.2021.1991865
Harrell, K. B., & Wendt, J. L. (2019). The impact of blended learning on Community of Inquiry and perceived learning among high school learners enrolled in a public charter school. Journal of Research on Technology in Education, 51(3), 259–272. https://doi.org/10.1080/15391523.2019.1590167
Hasson, F., Keeney, S., & McKenna, H. (2000). Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 32(4), 1008–1015.
Hathaway, D. & Mehdi, T. (2020). Exploring teachers’ perceptions of quality K-12 blended and online learning. In D. Schmidt-Crawford (Ed.), Proceedings of Society for Information Technology & Teacher Education International Conference (pp. 693-700). Association for the Advancement of Computing in Education (AACE).
Haynes, C. A., & Shelton, K. (2018). Delphi Method in a digital age. In Advances in Library and Information Science (pp. 132–151). IGI Global. https://doi.org/10.4018/978-1-5225-5164-5.ch009
Henríquez, V., & Hilliger, I. (2024). Blended learning in rural K‐12 education: Stakeholder dynamics and recommendations. Journal of Computer Assisted Learning, 40(4), 1463–1480. https://doi.org/10.1111/jcal.12963
Horn, M. B., & Staker, H. (2011). The rise of K–12 blended learning. Social Innovations Journal, (6). https://socialinnovationsjournal.com/index.php/sij/article/view/7985
Horn, M. B., & Staker, H. (2015). Blended: Using disruptive innovation to improve schools. Jossey Bass.
Huett, J. B., & Huett, K. C. (2012). Designing engaging virtual courses for K-12 students. In P. Resta (Ed.), Proceedings of Society for Information Technology & Teacher Education International Conference 2012 (pp. 432-440). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE).
Huett, K. C. (2018). Designing K–12 student-centered blended learning environment. Journal of Online Learning, 4(3), 295–326. https://doi.org/10.70725/976239xsvsvi
Huett, K. C., Huett, J. B., & Ringlaben, R. (2011). From bricks to clicks: Building quality K–12 online classes through an innovative course review project. Online Journal of Distance Learning Administration, 14(4). http://www.westga.edu/~distance/ojdla/winter144/huett_huett_ringlaben.htm
Jackson, B. L., Jones, W. M., Schad, M., & Strand, D. (2021). Assessing K12 online teachers’ knowledge of online student identities. Journal of Online Learning Research, 7(1), 35–36. https://doi.org/0.70725/063959cahlyf
Johnson, C. C., Walton, J. B., Strickler, L., & Elliott, J. B. (2023). Online teaching in K–12 education in the United States: A systematic review. Review of Educational Research, 93(3), 353–411. https://doi.org/10.3102/00346543221105550
Jorm, A. (2025). Expert consensus in science. Palgrave Macmillan. https://doi.org/10.1007/978-981-97-9222-1
Kennedy K., Tomaselli K., Stimson R. (2018). National standards for quality online courses (K–12) and QM K–12 secondary and K–12 publisher rubric revision: Literature review. Quality Matters. https://www.qualitymatters.org/sites/default/files/research-docs-pdfs/National-Standards-for-Quality-Online-Courses-Lit-Review-122818.pdf
Khodyakov, D., Grant, S., Kroger, J., & Bauman, M. (2023). RAND methodological guidance for conducting and critically appraising Delphi panels. RAND Corporation. https://www.rand.org/pubs/tools/TLA3082-1.html
Kömür, İ. A., Kılınç, H., & Okur, M. R. (2023). The rotation model in blended learning. Asian Journal of Distance Education, 18(2), 63–74.
Li, S., & Wang, W. (2022). Effect of blended learning on student performance in K–12 settings: A meta-analysis. Journal of Computer Assisted Learning, 38(5), 1254–1272. https://doi.org/10.1111/jcal.12696
Liao, H., Zhang, Q., Yang, L., & Fei, Y. (2023). Investigating relationships among regulated learning, teaching presence and student engagement in blended learning: An experience sampling analysis. Education and Information Technologies, 28, 12997–13025. https://doi.org/10.1007/s10639-023-11717-5
Linton, J. (2016). Electronic learning communities as a support for building relationships with students in a statewide virtual high school. Journal of Online Learning Research, 2(4), 419–445. https://doi.org/10.70725/292002tclusy
McCabe, C., & Francis, R. W. (2020). Effective instruction in blended learning environments. In T. I. Asino (Ed.) Learning in the digital age. https://open.library.okstate.edu/learninginthedigitalage/chapter/effective-instruction-in-blended-learning-environments/
McShane, M. Q. (2024). What do parents want? Information, choices, constraints. Ed Choice. https://files.eric.ed.gov/fulltext/ED662411.pdf
Means, B., Bakia, M., & Murphy, R. (2014). Learning online: What research tells us about whether, when and how. Routledge.
Min, W., & Yu, Z. (2023). A systematic review of critical success factors in blended learning. Education Sciences, 13(5), 469. https://doi.org/10.3390/educsci13050469
Molnar, A., Miron, G., Elgeberi, N., Barbour, M. K., Huerta, L., Shafer, S. R., & Rice, J. K. (2019). Virtual schools in the US 2019. National Education Policy Center. https://files.eric.ed.gov/fulltext/ED595244.pdf
Moore, M., Robinson, H., Sheffield, A. & Phillips, A. (2017). Mastering the blend: A professional development program for K–12 teachers. Journal of Online Learning Research, 3(2), 145–173. https://doi.org/10.70725/544886mqrfos
National School Choice Awareness Foundation. (2025). January 2025 survey. https://schoolchoiceawareness.org/january-2025-survey/
Nasa, P., Jain, R., & Juneja, D. (2021). Delphi methodology in healthcare research: How to decide its appropriateness. World Journal of Methodology, 11(4), 116–129. https://doi.org/10.5662/wjm.v11.i4.116
Newfield, T. (2025). Barriers to implementing blended learning in K–12 education: A quantitative analysis. International Journal of Educational Leadership Preparation, 20(1), 158–184.
Niederberger, M., & Spranger, J. (2020). Delphi technique in health sciences: A map. Frontiers in Public Health, 8, Article 457. https://doi.org/10.3389/fpubh.2020.00457
Palloff, R. M., & Pratt, K. (1999). Building learning communities in cyberspace. Jossey-Bass.
Palloff, R.M., & Pratt, K. (2005). Collaborating online: Learning together in community. Jossey-Bass.
Peirce, D. (2015, August 25). 3 ways mobile technology is transforming learning spaces. THE Journal. https://thejournal.com/articles/2015/08/25/3-ways-mobile-technology-is-transforming-learning-spaces.aspx
Poirier, M., Law, J. M., & Veispak, A. (2019). A spotlight on lack of evidence supporting the integration of blended learning in K–12 education. International Journal of Mobile and Blended Learning, 11(4), 1–14. https://doi.org/10.4018/ijmbl.2019100101
Pulham, E. B., & Graham, C. R. (2018). Comparing K–12 online and blended teaching competencies: A literature review. Distance Education, 39(3), 411–432. http://doi.org/10.1080/01587919.2018.1476840
Quiroz, R. E., Ritter, N. L., Li, Y., Newton, R. C., Palkar, T., Texas A&M University, & United States. (2016). Standards based design: Teaching K–12 educators to build quality online courses. Journal of Online Learning Research, 2(2), 123–144.
Reardon, M. R. (2010). Instructional leadership and blended learning. In Y. Inoue (Ed.), Cases on online and blended learning technologies in higher education: Concepts and practices (pp. 44–62). IGI Global. https://doi.org/10.4018/978-1-60566-880-2.ch003
Sarmento, T. S., Gomes, A. S., & Moreira, F. (2018). Classroom adaptations for blended learning practices. In F. J. García-Peñalvo (Ed.), Proceedings of 6th International Conference Technological Ecosystems for Enhancing Multiculturality (TEEM’18), Salamanca, Spain, October 2018). ACM. https://doi.org/10.1145/3284179.328429
Shand, K., & Glassett Farrelly, S. (2018). The art of blending: Benefits and challenges of a blended course for preservice teachers. Journal of Educators Online, 15(1). https://www.thejeo.com/archive/2018_15_1/shand_farrelly
Schmalz, U., Spinler, S., & Ringbeck, J. (2021). Lessons learned from a two-round Delphi-based scenario study. MethodsX, 8, Article 101179. https://doi.org/10.1016/j.mex.2020.101179
Schifano, J., & Niederberger, M. (2025). How Delphi studies in the health sciences find consensus: A scoping review. Systematic Reviews, 14, Article 7. https://doi.org/10.1186/s13643-024-02738-3
Shelton, K., & Creghan, K. A. (2015). Demystifying the Delphi Method. In V. C. Wang (Ed.). Handbook of research on scholarly publishing and research methods (pp. 375-395). IGI-Global Publishing. https://doi.org/10.4018/978-1-4666-7409-7.ch019
Short, C. R., Graham, C. R., & Sabey, E. (2021). K-12 blended teaching skills and abilities: An analysis of blended teaching artifacts. Journal of Online Learning Research, 7(1), 5–33. https://doi.org/10.70725/456263phwuix
Siko, J., Barbour, M., Wilson, V. & Adelstein, D. (2026). Assessing and improving digital learning readiness: A review and roadmap for future research. Journal of Online Learning Research, 12(1), 151–168. https://doi.org/10.70725/291317sfcqzq
Smith, S., & Stahl, W. (2016). Determining the accessibility of K–12 digital materials: Tools for educators. Journal of Special Education Leadership, 29, 70–79.
Staker, H., & Horn, M. B. (2012). Classifying K–12 blended learning. Innosight Institute. https://www.christenseninstitute.org/wp-content/uploads/2013/04/Classifying-K-12-blended-learning.pdf
Stevens, M., & Rice, M. (2016). Inquiring into presence as support for student learning in a blended learning classroom. Journal of Online Learning Research, 2(4), 447–473. https://doi.org/10.70725/492597rcbgmr
Sustanto, R., & Atabik, S. P. (2025). The impact of blended learning models on student achievement and engagement in secondary education: A systematic review. International Journal of Multidisciplinary Research and Analysis, 8(7), 3982–3993. https://doi.org/10.5281/zenodo.15879106
Tao, Y., Yu, L, Luo, L. & Zhang, H. (2024). Effect of blended teaching on college students’ EFL acquisition. Frontiers in Education, 9, 1264573. https://doi.org/10.3389/feduc.2024.1264573
Topping, K. J., Walter Douglas, Robertson, D., & North Lanarkshire Council. (2022). Effectiveness of online and blended learning from schools: A systematic review. Review of Education. https://doi.org/10.1002/rev3.3353
Tovine, G., Fleetwood, A., Shepherd, A., Tapoler, C. J., Hartshorne, R., & Pesce, R. (2019). Investigating student perceptions and the effectiveness of K–12 blended learning communities. In T. L. Heafner, R. Hartshorne, & R. Thripp (Eds.), Handbook of research on emerging practices and methods for K–12 online and blended learning (500–522). IGI Global. https://doi.org/10.4018/978-1-5225-8009-6.ch024
Valverde-Berrocoso, J., & Fernández-Sánchez, M. R. (2020). Instructional design in blended learning: Theoretical foundations and guidelines for practice. In A. Martín-García (Ed.) Blended learning: Convergence between technology and pedagogy (pp. 113–140). Springer. https://doi.org/10.1007/978-3-030-45781-5_6
Valverde-Berrocoso, J., & Fernández-Sánchez, M. R. (2020). Instructional design in blended learning: Theoretical foundations and guidelines for practice. In Lecture Notes in Networks and Systems (pp. 113–140). Springer International Publishing. https://doi.org/10.1007/978-3-030-45781-5_6
Vaughan, N. D., Cleveland-Innes, M., & Garrison, D. R. (2013). Teaching in blended learning environments: Creating and sustaining communities of inquiry. Athabasca University Press.
Villaneuva, J. A. R. (2021). Teaching presence in K–12 blended learning classes under the alternative delivery mode. International Journal on Open and Distance E-Learning, 7(1). https://ijodel.upou.edu.ph/ijodel/article/view/33
Villanueva, J.A.R., Redmond, P., Galligan, L., & Eacersall, D. (2024). Investigating blended learning interactions in Philippine schools through the community of inquiry framework. Asia Pacific Education Review, 25, 813–828. https://doi.org/10.1007/s12564-023-09826-4
Voelker, C., Gemin, B., Quality Matters, Virtual Learning Leadership Alliance, Powell, A., Oliver, W., Archambault, L., Borup, J., Boyarko, M., Cavanaugh, C., Copeland, R., Day, J., DeRosa-Davis, T., Deschaine, M., Graham, C., Hanson, E., Hernandez, B., Imbrunone, M., Jensen, J., . . . Quality Matters. (2019). National Standards for Quality Online Teaching (3rd ed). https://www.nsqol.org/wp-content/uploads/2019/02/National-Standards-for-Quality-Online-Teaching.pdf
Wang, M.-T., & Hofkens, T. L. (2019). Beyond classroom academics: A school-wide and multi-contextual perspective on student engagement in school. Adolescent Research Review, 5, 419–433. https://doi.org/10.1007/s40894-019-00115-z
Wang, M.-T., Fredricks, J., Ye, F., Hofkens, T., & Linn, J. S. (2017). Conceptualization and assessment of adolescents’ engagement and disengagement in school. European Journal of Psychological Assessment, 35(4), 592–606. https://doi.org/10.1027/1015-5759/a000431
Whiteside, A. L., Dikkers, A. G., & Baker III, F. W. (2019). Moving beyond trial and error: Exploring case studies of professional development models in K-12 blended learning. In T. L. Heafner, R. Hartshorne, & R. Thripp (Eds.), Handbook of research on emerging practices and methods for K–12 online and blended learning (pp. 248–270). IGI Global. https://doi.org/10.4018/978-1-5225-8009-6.ch012
Yu, Q., Yu, K., & Wang, J. (2025a). Unraveling the impact of blended learning vs. online learning on learners’ performance: Perspective of self-determination theory. Behavioral Sciences, 15(9), 1263. https://doi.org/10.3390/bs15091263
Yu, Q., Yu, K., Li, B., & Wang, Q. (2025b). Effectiveness of blended learning on students’ learning performance: A meta-analysis. Journal of Research on Technology in Education, 57(3), 499–520. https://doi.org/10.1080/15391523.2023.2264984
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Kaye Shelton
This work is licensed under a Creative Commons Attribution 4.0 International License.
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
