Developing Peer Review of Instruction in an Online Master Course Model
DOI:
https://doi.org/10.24059/olj.v25i3.2428Keywords:
peer observation, peer reviews, online learning, master courses,Abstract
In this study we looked at how participation in a peer-review process for online Statistics courses utilizing a master course model at a major research university affects instructor innovation and instructor presence. We used online, anonymous surveys to collect data from instructors who participated in the peer-review process, and we used descriptive statistics and qualitative analysis to analyze the data. Our findings indicate that space for personal pedagogical agency and innovation is perceived as limited because of the master course model. However, responses indicate that participating in the process was overall appreciated for the sense of community it helped to build. Results of the study highlight the blurred line between formative and summative assessment when using peer review of instruction, and they also suggest that innovation and presence are difficult to assess through short term observation and through a modified version of a tool (i.e., the Quality Matters rubric) intended for the evaluation of an online course rather than the instruction of that course. The findings also suggest that we may be on the cusp of a second stage for peer review in an online master course model, whether in-person or online. Our findings also affirm the need for creating a sense of community online for the online teaching faculty. The experiences of our faculty suggest that peer review can serve as an integral part of fostering a departmental culture that leads to a host of intangible benefits including trust, reciprocity, belonging, and, indeed, respect.
References
Baran, E., Correia, A. P., & Thompson, A. (2011). Transforming online teaching practice: Critical analysis of the literature on the roles and competencies of online teachers. Distance Education, 32(3), 421-439.
Barnard, A., Nash, R., McEvoy, K., Shannon, S., Waters, C., Rochester, S., & Bolt, S. (2015). LeaD-in: a cultural change model for peer review of teaching in higher education. Higher Education Research & Development, 34(1), 30-44.
Barnett, D. E. (2019). Full-range leadership as a predictor of extra effort in online higher Education: The mediating effect of job satisfaction. Journal of Leadership Education, 18(1).
Bennett, S., & Barp, D. (2008). Peer observation–a case for doing it online. Teaching in Higher Education, 13(5), 559-570.
Blackmore, J. A. (2005). A critical evaluation of peer review via teaching observation within higher education. International Journal of Educational Management, 19(3), 218-232.
Calsamiglia, C., & Loviglio, A. (2019). Grading on a curve: When having good peers is not good. Economics of Education Review, 73(C).
Cavanagh, R. R. (1996). Formative and summative evaluation in the faculty peer review of teaching. Innovative higher education, 20(4), 235-240.
Chism, N. V. N. (1999). Peer review of teaching. A sourcebook. Bolton, MA: Anker.
Drysdale, J. (2019). The Collaborative Mapping Model: Relationship-Centered Instructional Design for Higher Education. Online Learning, 23(3), 56-71.
Edkey, M. T. & Roehrich, H. (2013). A faculty observation model for online instructors: Observing faculty members in the online classroom. Online Journal of Distance Learning Administration, 16 (2). Retrieved from: http://www.westga.edu/~distance/ojdla/summer162/eskey_roehrich162.html
Edström, K., Levander, S., Engström, J., & Geschwind, L. (2019). Peer review of teaching merits in academic career systems: a comparative study. In Research in Engineering Education Symposium.
Esfijani, A. (2018). Measuring quality in online education: A meta-synthesis. American Journal of Distance Education, 32(1), 57-73.
Gosling, D. (2002). Models of Peer Observation of Teaching. Report. LTSN Generic Center. https://www.researchgate.net/profile/David_Gosling/publication/267687499_Models_of_Peer_Observation_of_Teaching/links/545b64810cf249070a7955d3.pdf
Graham, C., Cagiltay, K., Lim, B. R., Craner, J., & Duffy, T. M. (2001). Seven principles of effective teaching: A practical lens for evaluating online courses. The Technology Source, 30(5), 50.
Hanbing, Y., & Mingzhuo, L. (2012). Research on master-teachers’ management model in online course by integrating learning support. Journal of Distance Education, 5(10), 63-67.
Hutchings, P. (1996). Making Teaching Community Property: A Menu for Peer Collaboration and Peer Review. AAHE Teaching Initiative.
Hutchings, P. (1996). The peer review of teaching: Progress, issues and prospects. Innovative Higher Education, 20(4), 221-234.
Hyland, K. M., Dhaliwal, G., Goldberg, A. N., Chen, L. M., Land, K., & Wamsley, M. (2018). Peer Review of Teaching: Insights From a 10-Year Experience. Medical Science Educator, 28(4), 675-681.
Johnson, G., Rosenberger, J., & Chow, M. (October 2014) The importance of setting the stage: Maximizing the benefits of peer review of teaching.” eLearn Magazine, ACM, https://elearnmag.acm.org/archive.cfm?aid=2673801, October 2014.
Jones, M. H., & Gallen, A. M. (2016). Peer observation, feedback and reflection for development of practice in synchronous online teaching. Innovations in Education and Teaching International, 53(6), 616-626.
Keig, L. (2000). Formative peer review of teaching: Attitudes of faculty at liberal arts colleges toward colleague assessment. Journal of Personnel Evaluation in Education, 14(1), 67-87.
Keig, L. W., & Waggoner, M. D. (1995). Peer review of teaching: Improving college instruction through formative assessment. Journal on Excellence in College Teaching, 6(3), 51-83.
Kell, C., & Annetts, S. (2009). Peer review of teaching embedded practice or policy‐holding complacency?. Innovations in Education and Teaching International, 46(1), 61-70.
Knowles, E., & Kalata, K. (2007). A model for enhancing online course development. Innovate: Journal of Online Education, 4(2).
Lomas, L., & Nicholls, G. (2005). Enhancing teaching quality through peer review of teaching. Quality In higher education, 11(2), 137-149.
Mayes, R. (2011, March). Themes and strategies for transformative online instruction: A review of literature. In Global Learn (pp. 2121-2130). Association for the Advancement of Computing in Education (AACE).
McGahan, S. J., Jackson, C. M., & Premer, K. (2015). Online Course Quality Assurance: Development of a Quality Checklist. InSight: A Journal of Scholarly Teaching, 10, 126-140.
McManus, D. A. (2001). The two paradigms of education and the peer review of teaching. Journal of Geoscience Education, 49(5), 423-434.
Nilson, L. B. (2012). 14: Time to raise questions about student ratings. To improve the academy, 31(1), 213-227.
Nilson, L. B. (2013). 17: Measuring student learning to document faculty teaching effectiveness. To improve the academy, 32(1), 287-300.
Nogueira, I. C., Gonçalves, D., & Silva, C. V. (2016). Inducing Supervision Practices among peers in a Community of Practice. Journal for Educators, Teachers and Trainers, 7, 108-119.
Parscal, T., & Riemer, D. (2010). Assuring quality in large-scale online course development. Online Journal of Distance Learning Administration, 13(2).
Peel, D. (2005). Peer Observation as a Transformatory tool? Teaching in Higher Education, 10(4), 489 - 504.
Roy, R., & Ashburn, J. R. (2001). The perils of peer review. Nature, 414(6862), 393-394.
Schneider, G. (2013, March). Student evaluations, grade inflation and pluralistic teaching: Moving from customer satisfaction to student learning and critical thinking. In Forum for Social Economics 42(1),122-135.
Seldin, P. (1993). The use and abuse of student ratings of professors. Chronicle of Higher Education, 39(46), A40-A40.
Seldin, P., Miller, J. E., & Seldin, C. A. (2010). The teaching portfolio: A practical guide to improved performance and promotion/tenure decisions. John Wiley & Sons.
Sharma, M., & Ling, A. (2018). Peer review of teaching: What features matter? A case study within STEM faculties. Innovations in Education and Teaching International, 55(2), 190-200.ms: a comparative study. In Research in Engineering Education Symposium.
Shattuck, K., Zimmerman, W. A., & Adair, D. (2014). Continuous improvement of the QM Rubric and review processes: Scholarship of integration and application. Internet Learning Journal, 3(1).
Shelton, K. (2011). A review of paradigms for evaluating the quality of online education programs. Online Journal of Distance Learning Administration,4(1), 1-11.
Smith, S. L. (2014). Peer collaboration: Improving teaching through comprehensive peer review. To Improve the Academy, 33(1), 94-112.
Swinglehurst, D., Russell, J., & Greenhalgh, T. (2008). Peer observation of teaching in the online environment: an action research approach. Journal of Computer Assisted Learning, 24, 383-393.
Taylor, A. H. (2017). Intrinsic and extrinsic motivators that attract and retain part-time online teaching faculty at Penn State (Doctoral dissertation, The Pennsylvania State University).
Thomas, S., Chie, Q. T., Abraham, M., Jalarajan Raj, S., & Beh, L. S. (2014). A qualitative review of literature on peer review of teaching in higher education: An application of the SWOT framework. Review of educational Research, 84(1), 112-159.
Tobin, T. J., Mandernach, B. J., & Taylor, A. H. (2015). Evaluating online teaching: Implementing best practices. San Francisco, CA: John Wiley & Sons.
Trautmann, N. M. (2009). Designing Peer Review for Pedagogical Success. Journal of College Science Teaching, 38(4).
Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22-42.
Wood, D., & Friedel, M. (2009). Peer review of online learning and teaching: Harnessing collective intelligence to address emerging challenges. Australasian Journal of Educational Technology, 25(1).
Zipser, N., & Mincieli, L. (2018). Administrative and structural changes in student evaluations of teaching and their effects on overall instructor scores. Assessment & Evaluation in Higher Education, 43(6), 995-1008.
Downloads
Published
Issue
Section
License
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
