Moving Beyond Smile Sheets: A Case Study on the Evaluation and Iterative Improvement of an Online Faculty Development Program

Ken-Zen Chen, Patrick R. Lowenthal, Christine Bauer, Allan Heaps, Crystal Nielsen


Institutions of higher education are struggling to meet the growing demand for online courses and programs, partly because many faculty lack experience teaching online. The eCampus Quality Instruction Program (eQIP) is an online faculty development program developed to train faculty to design and teach fully online courses. The purpose of this article is to describe the eQIP (one institution’s multipronged approach to online faculty development), with a specific focus on how the overall success of the program is evaluated using surveys, analytics, and social network analysis. Reflections and implications for improving practice are discussed.


online education, faculty development, instructional consultation, online course design

Full Text:



Bacow, L. S., Bowen, W. G., Guthrie, K. M., Lack, K. A., & Long, M. P. (2012). Barriers to adoption of online learning systems in US higher education. New York, NY: ITHA. Retrieved from

Bailey, C. J., & Card, K. A. (2009). Effective pedagogical practices for online teaching: Perception of experienced instructors. Internet and Higher Education, 12, 152-155.

Baran, E., & Correia, A. (2014). A professional development framework for online teaching. TechTrends, 58(5), 96-102.

Baran, E., Correia, A., & Thompson, A. (2011). Transforming online teaching practice: Critical analysis of the literature on the roles and competencies of online teachers. Distance Education, 32, 421-439. doi:10.1080/01587919.2011.610293

Benton, C. J. (2011). Asynchronous learning and faculty development: Evolving college-level online instruction and empowered learning. International Journal of Information & Communication Technology Education, 7(1), 89-96. doi:10.4018/jicte.2011010109

Blignaut, A., & Trollip, S. R. (2003). Measuring faculty participation in asynchronous discussion forums. Journal of Education for Business, 78, 347-353.

Boettcher, J., & Conrad, R. (2004). Faculty guide for moving teaching and learning to the Web (2nd ed.). Phoenix, AZ: League for innovation in the Community College.

Bond, T. (Producer). (2011, August 17). Community building: Putting it all together - Web 2.0. [Video File] Retrieved from

Britto, M., Ford, C., & Wise, J., (2014). Three institutions, three approaches, one goal: Addressing quality assurance in online learning. Journal of Asynchronous Learning Network, 17, 11-24.

Brooks, C. F. (2010). Toward 'hybridised' faculty development for the twenty-first century: Blending online communities of practice and face-to-face meetings in instructional and professional support programmes. Innovations in Education & Teaching International, 47(3), 261-270. doi:10.1080/14703297.2010.498177

Carter-Cram, K. (2014). Game on! Teaching foreign language online. In M. Orleans (Ed.), Cases on critical and qualitative perspectives in online higher education (pp. 208-226). Hershey, PA: IGI Global. doi:10.4018/978-1-4666-5051-0.ch011

Carter-Cram, K., & Black, J. (2014, January). Professional development for effective online teaching. Poster presented at the 3rd Annual Great Ideas Symposium, Center for Teaching & Learning, Boise ID.

Chen, C. C., & Shaw, R. S. (2006). Online synchronous vs. asynchronous software training through the behavioral modeling approach: A longitudinal field experiment. International Journal of Distance Education Technologies, 4(4), 88-102.

Author1. (2014, March). Supporting faculty toward high-quality online course design: Boise State eQIP Program. Poster session presented at the conference of Diversity, Learning, and Student Success: Policy, Practice, and Privilege. Chicago, IL: Association of American Colleges and Universities.

Creswell, J.W., & Plano, V.L. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.

Dabbagh, N., & Bannan-Ritland, B. (2005). Online learning: Concepts, strategies, and application. Upper Saddle River, NJ: Prentice Hall.

Dailey-Hebert, A., Mandernach, B. J., Donnelli-Sallee, E., & Norris, V. R. (2014). Expectations, motivations, and barriers to professional development: Perspectives from adjunct instructors teaching online. Journal of Faculty Development, 28, 67-82.

Dawson, S., Bakaharia, A., & Heathcote, E. (2010). SNAPP: Realizing the affordances of real-time SNA within networked learning environments. Paper presented at the 7th International Conference on Networked Learning, Aalborg, Denmark.

Dobrovolny, J. & Author2, P. R. (2011). Using reflection surveys to improve teaching and learning. In P. R. Author2, D. Thomas, A. Thai, B. Yuhnke, M. Edwards, & C. Gasell. (Eds.). The CU Online Handbook, 2011 (pp. 97-113). Raleigh, NC: Lulu.

Dunlap, J. (2005). Workload reduction in online courses: Getting some shuteye. Performance Improvement, 44(5), 18-25.

Dunlap, J., Bose, D., Author2, P. R., York, C. S., Atkinson, M., & Murtagh, J. (in press). What sunshine is to flowers: A literature review on the use of emoticons to support online learning. To appear in Emotions, Design, Learning and Technology. Elsevier.

Dunlap, J.C., & Author2, P.R. (2014). The power of presence: Our quest for the right mix of social presence in online courses. In A. A. Piña & A. P. Mizell (Eds.), Real life distance education: Case studies in practice (pp. 41-66). Greenwich, CT: Information Age.

Fein, A. D., & Logan, M. C. (2003). Preparing instructors for online instruction. New Directions for Adult and Continuing Education, 100, 45-55.

Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education model. The Internet and Higher Education, 2(2-3), 87-105.

Haskell, C. (2013). Understanding quest-based learning. [white paper]. Retrieved from

Hislop, G. W., & Ellis, H. J. C. (2004). A study of faculty effort in online teaching. Internet and Higher Education, 7, 15-31.

Kukulska-Hulme, A. (2012). How should the higher education workforce adapt to advancements in technology for teaching and learning? Internet and Higher Education, 15, 247-254.

Author2, Author 3, & Author1 (2015). Student perceptions of online learning: An analysis of online course evaluations. American Journal of Distance Education, 29(2), 85-97.

Author2, P. R., & Leech, N. (2009). Mixed research and online learning: Strategies for improvement. In T. T. Kidd (Ed.), Online education and adult learning: New frontiers for teaching practices (pp. 202-211). Hershey, PA: IGI Global.

Author2, P. R., & Thomas, D. (2010a). Death to the digital dropbox: Rethinking student privacy and public performance. EDUCAUSE Quarterly, 33(3). Retrieved from

Author2, P. R., & Thomas, D. (2010b). Digital campfires: Innovations in helping faculty explore the online learning wildness. Journal of Online Learning and Teaching, 6, 665-672.

Author2, P. R., & White, J. W. (2009). Enterprise model. In P. Rogers, G. Berg, J. Boettcher, C. Howard, L. Justice, & K. Schenk (Eds.), Encyclopedia of distance and online learning (2nd ed., pp. 932-936). Hershey, PA: IGI Global.

Author2, P. R., Wray, M. L., Bates, B., Switzer, T., & Stevens, E. (2013). Examining faculty motivation to participate in faculty development. International Journal of University Teaching and Faculty Development, 3, 149-164.

Lucas, C. J., & Murry, J. W. (2011). New faculty: A practical guide for academic beginners. New York, NY: Palgrave Macmillan.

Marek, K. (2009). Learning to teach online: Creating a culture of support for faculty. Journal of Education for Library & Information Science, 50(4), 275-292.

MarylandOnline (2011). Quality MattersTM rubric workbook for higher education (2011-2013 Edition). Annapolis, MD: MarylandOnline.

Mason, J., Hickman, C., Dyer, A., Koproske, C., Fry, G., & Taha, M. (2010). Engaging faculty in online education: Rightsizing incentives and optimizing support. Washington, DC: Education Advisory Board.

McCormick, J. (2013). Visualizing Interaction: Pilot investigation of a discourse analytics tool for online discussion. Bulletin of the IEEE Technical Committee on Learning Technology, 15(2), 10-13.

McElrath, E., & McDowell, K. (2008). Pedagogical strategies for building community in graduate level distance education course. Journal of Online Learning and Teaching, 4(1), 117-127.

McQuiggan, C. A. (2011). Preparing to teaching online as transformative faculty development. Unpublished Doctoral Dissertation, Penn State University, State College, PA.

Merriam, S. B. (1998). Qualitative research and case study applications in education (2nd ed.). San Francisco, CA: Jossey-Bass.

Meyer, K. A., & Murrell, V. S. (2014). A national study of training content and activities for faculty development for online teaching. Journal of Asynchronous Learning Network, 18. Retrieved from

Maor, D. (2006). Using reflective diagrams in professional development with university lecturers: A developmental tool in online teaching. Internet and Higher Education, 9, 133-145.

Murdock, J. L., & Williams, A. M. (2011). Creating an online learning community: Is it possible? Innovative Higher Education, 36(5), 305-315.

Natriello, G. (2005). Modest changes, revolutionary possibilities: Distance learning and the future of education. Teacher College Record, 110, 105-138.

Pickett, A. (Producer). (2010, August 16). Social presence. [Video File] Retrieved from

Ragan, L. C. (1999). Good teaching is good teaching. An emerging set of guiding principles and practices for the design and development of distance education. Cause/Effect, 22(1), 20-24.

Ramsay, C. M., Aman, D. D., & Pursel, B. K. (2014). Blogging pragmatics and pedagogy: An adventure in faculty development. Education and Information Technologies, 19, 425-440.

Sally, M. J., & Louis, S. (2014). Principles for developing competency-based education programs. Change, 46(2), 12-19. DOI: 10.1080/00091383.2014.896705

Salmon, G. (2012). E-moderating: The key to online teaching and learning. New York, NY: Routledge.

Schrum, L., Burbank, M. D., Engle, J., Chambers, J. A., & Glassett, K. F. (2005). Post-secondary educators’ professional development: Investigation of an online approach to enhancing teaching and learning. Internet and Higher Education, 8, 279-289.

Sellani, R. J., & Harrington, W. (2002). Addressing administrator/faculty conflict in an academic online environment. Internet and Higher Education, 5, 131-145.

Sorinola, O. O., Thistlethwaite, J., Davies, D., & Peile, E. (2015). Faculty development for educators: A realist evaluation. Advances in Health Sciences Education: Theory and Practice, 20, 385-401.

Stake, R. (1995). The art of case study research. London, UK: SAGE.

Stavredes, T. (2011). Effective online teaching: Foundations and strategies for student success. San Francisco, CA: Jossey-Bass.

Stone, M. T., & Perumean-Chaney, S. (2011). The benefits of online teaching for traditional classroom pedagogy: A case study for improving face-to-face instruction. Journal of Online Learning and Teaching, 7, 393-400.

Stufflebeam, D. L., Shinkfield, A. J. (2007). Evaluation theory, models, and applications. San Francisco, CA: Jossey-Bass.

Tipple, R. (2010). Effective leadership of online adjunct faculty. Online Journal of Distance Learning Administration, 13(1). Retrieved from

Vaughan, N., & Garrison, D. R. (2005). Creating cognitive presence in a blended faculty development community. Internet and Higher Education, 8, 1-12.

Vesely, P., Bloom, L., & Sherlock, J. (2007). Key elements of building online community: Comparing faculty and student perceptions. Journal of Online Learning and Teaching, 3(3), 234-246.

Vries, S., Grift, W. J. C. M., &, Jansen, E. P. W. A. (2014). How teachers’ beliefs about learning and teaching relate to their continuing professional development. Teacher and Teaching: Theory and Practice, 20, 338-357.

Vu, P., Cao, V., & Cepero, J. (2014). Factors driving learner success in online professional development. The International Review of Research in Open and Distance Learning, 15(3), 120-139. Retrieved from

Wilcoxon, K. (2011, October 3). Building an online learning community. Learning Solutions. Retrieved from

Wilson, B. G., Ludwig-Hardman, S., Thornam, C. L., & Dunlap, J. (2004). Bounded community: Designing and facilitating learning communities in formal courses. The International Review of Research in Open and Distance Learning, 5(3). Retrieved from

Wolf, E. A. (2013). A study of motivation of online adjunct college undergraduate faculty (Doctoral dissertation, University of Phoenix).

Xu, H., & Morris, L. (2007). Collaborative course development for online courses. Innovative Higher Education, 32(1), 35-47. doi:10.1007/s10755-006-9033-5

Yin, R. K. (2009). Case study research: Design and methods (4th ed.). London: SAGE.