Presenting a Validated Mid-Semester Evaluation of College Teaching to Improve Online Teaching
DOI:
https://doi.org/10.24059/olj.v24i2.2126Keywords:
Online Teaching, Student Evaluation of Teaching, Formative Feedback, Formative Evaluation of Teaching, Classroom ClimateAbstract
Formative feedback from students can help college instructors improve their online teaching practices - especially instructors who are new to online teaching. Prior research indicates that mid-semester formative evaluations of college teaching are a promising, low-cost solution to providing online instructors with in-the-moment feedback. However, existing instruments suffer from issues of validity and bias, and fail to align with evidence-based strategies. In this paper, we present psychometric results from a pilot study of our research-based Mid-Semester Evaluation of College Teaching (MSECT) to assist online educators in gathering student input to improve their online teaching and classroom climate.
References
Authors. (2018). [Blinded Title]. Paper presented in a roundtable at American Educational Research Association annual meeting. New York, NY.
Authors. (2019a). [Blinded Title]. To Improve the Academy: A Journal of Educational Development, [BLIND]
Authors. (2019b). [Blinded Title]. Poster presented at Society for Research in Child Development annual meeting. Baltimore, MD.
Alexander, P. A., & Winne, P. H. (2006). Handbook of educational psychology. (Second ed.). Mahwah, NJ: Erlbaum.
Ambrose, S., Bridges, M., DiPietro, M., Lovett, M. & Norman, M. (2010). How learning works: Seven research-based principles for smart teaching. San Francisco, CA: Jossey-Bass.
Bangert, A. W. (2008). The development and validation of the student evaluation of online teaching effectiveness. Computers in the Schools, 25(1–2), 25–47. http://doi.org/10.1080/07380560802157717
Berridge, G. G., Penney, S., & Wells, J. (2012). eFACT: Formative assessment of classroom teaching for online classes. Turkish Online Journal of Distance Education, 13(April), 119–131.
Borup, J., & Evmenova, A. S. (2019). The effectiveness of professional development in overcoming obstacles to effective online instruction in a college of education. Online Learning Journal, 23(2), 1–20. http://doi.org/10.24059/olj.v23i2.1468
Byrne, D. (2017). Publishing your medical research (Second ed.). Philadelphia: Wolters Kluwer.
Carle, A. C. (2009). Evaluating college students’ evaluations of a professor’s teaching effectiveness across time and instruction mode (online vs. face-to-face) using a multilevel growth modeling approach. Computers and Education, 53(2), 429–435. http://doi.org/10.1016/j.compedu.2009.03.001
Coffman, S. J. (1998). Small group instructional evaluations across disciplines. College Teaching, 46(3), 106-111.
Cohen, P. A. (1980). Effectiveness of student-rating feedback for improving college instruction: A meta-analysis of findings. Research in Higher Education, 13(4), 321–41. https://doi.org/10.1007/BF00976252.
Costello, M. L., Weldon, A. & Brunner, P. (2002). Reaction cards as a formative evaluation tool: students’ perceptions of how their use impacted classes. Assessment and Evaluation in Higher Education, 27(1), 23–33.
Darby, F. (2019). How to be a better online teacher. The Chronicle of Higher Education. Retrieved from https://www.chronicle.com/interactives/advice-online-teaching
Gómez-Rey, P., Barbera, E., & Fernández-Navarro, F. (2016). Measuring teachers and learners’ perceptions of the quality of their online learning experience. Distance Education, 37(2), 146–163. http://doi.org/10.1080/01587919.2016.1184396
Hammer, J. (2016). Construct replicability calculator: A Microsoft Excel-based tool to calculate the Hancock and Mueller (2001) H index. Retrieved from http://DrJosephHammer.com/
Hammonds, F., Mariano, G. J., Ammons, G., & Chambers, S. (2017). Student evaluations of teaching: Improving teaching quality in higher education. Perspectives: Policy and Practice in Higher Education, 21(1), 26–33. http://doi.org/10.1080/13603108.2016.1227388
Hampton, S. E., & Reiser, R. A. (2004). Effects of a theory-based feedback and consultation process on instruction and learning in college classrooms. Research in Higher Education, 45(5), 497–527.
Hancock, G R. & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. Structural Equation Modeling: Present and future, 195-216.
Hattie, J. & Timperley, H. (2007). The power of feedback the meaning of feedback. Review of Educational Research, 77(1), 81–112. https://doi.org/10.3102/003465430298487.
Hill, P. (2012). Online educational delivery models: A descriptive view. EDUCAUSE Review, 47, 84–86. EDUCAUSE Review, 47, 84-86. Retrieved from https://er.educause.edu/articles/2012/11/online-educational-delivery-models--a-descriptive-view
Howard, T. C. (2001). Telling their side of the story: African-American students' perceptions of culturally relevant teaching. The Urban Review: Issues and Ideas in Public Education, 33(2), 131–49. doi:10.1023/A:1010393224120.
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
Kline, R. B. (2011). Principles and Practice of Structural Equation Modeling. (Third ed.). Methodology of the Social Sciences. New York: Guilford Press.
Knol, M. H, Dolan, C. V., Mellenbergh, G. J., & van der Maas, H. L. J. (2016). Measuring the quality of university lectures: Development and validation of the instructional skills questionnaire. Plos One, 11(2), 1-21.
Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press.
Liu, O. L. (2012). Student evaluation of instruction: In the new paradigm of distance education. Research in Higher Education, 53(4), 471–486.
http://doi.org/10.1007/s11162-011-9236-1
Marsh, H. W. (1987). Students’ evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11(3), 253–388. https://doi.org/10.1016/0883-0355(87)90001-2.
Marsh, H. W., & Roche, L. A. 2000. Effects of grading leniency and low workload on students’ evaluations of teaching: Popular myth, bias, validity, or innocent bystanders? Journal of Educational Psychology, 92(1), 202–28. http://10.0.4.13/0022-0663.92.1.202.
McKeachie, W. J. (1997). Student ratings: The validity of use. American Psychologist, 52(11), 1218–1225.
Meyer, K. (2014). An analysis of the cost and cost-effectiveness of faculty development for online teaching. Online Learning, 18(1). doi:http://dx.doi.org/10.24059/olj.v18i1.389
Meyer, K. A., & Murrell, V. S. (2014). A national study of training content and activities for faculty development for online teaching. Journal of Asynchronous Learning Networks, 18, 3–18.
Morin, A. J. S, Marsh, H. W., Nagengast, B., & Scalas, L. F. (2014). Doubly latent multilevel analyses of classroom climate: An illustration. The Journal of Experimental Education, 82(2), 143–67. doi:10.1080/00220973.2013.769412.
O’Neil-Hixon, K., Long, J., & Bock, M. (2017). The eSGID process: How to improve teaching and learning in online graduate courses, 17(2), 45–57.
Online Learning Consortium. (2019). OLC quality scorecard. Retrieved from https://onlinelearningconsortium.org/consult/olc-quality-scorecard-suite/
Overall, J. U., & Marsh, H. W. (1979). Midterm feedback from students: Its relationship to instructional improvement and students’ cognitive and affective outcomes. Journal of Educational Psychology, 71(6), 856–65. https://doi.org/10.1037/0022-0663.71.6.856.
Peterson, J. L. (2016). Formative Evaluations in Online Classes. Journal of Educators Online, 13(1).
Raykov, T., & Marcoulides, G. (2011). Classical item analysis using latent variable modeling: A note on a direct evaluation procedure. Structural Equation Modeling, 18(2), 315-324.
Spooren, P. (2010). On the credibility of the judge: A cross-classified multilevel analysis on students’ evaluation of teaching. Studies in Educational Evaluation, 36(4), 121–131. http://doi.org/10.1016/j.stueduc.2011.02.001
Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research, 83(4), 598–642. http://doi.org/10.3102/0034654313496870
Spooren, P., Vandermoere, F., Vanderstraeten, R., & Pepermans, K. (2017). Exploring high impact scholarship in research on student’s evaluation of teaching (SET). Educational Research Review, 22,129–41. https://doi.org/10.1016/j.edurev.2017.09.001.
Thomas, J., Graham, C., & Piña, A. (2018). Current practices of online instructor evaluation in higher education. Online Journal of Distance Learning Administration, 21(2), 1-13.
Thomas, J. E., & Graham, C. R. (2019). Online teaching competencies in observational rubrics: What are institutions evaluating? Distance Education, 40(1), 114–32. https://doi.org/10.1080/01587919.2018.1553564.
Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington, DC: American Psychological Association.
Ting, K. F. (2000). A multilevel perspective on student ratings of instruction: Lessons from the Chinese experience. Research in Higher Education, 41(5), 637–61. https://doi.org/10.1023/A:1007075516271.
Walker, S. L. (2005). Modifying formative evaluation techniques for distance education class evaluation. Turkish Online Journal of Distance Education, 6(4), 7-11.
Wass, R., Timmermans, J., Harland, T., & McLean, A. (2018). Annoyance and frustration: Emotional responses to being assessed in higher education. Active Learning in Higher Education, 1-13. doi:10.1177/1469787418762462.
Wentzel, K. R. & Brophy, J. E. (2014). Motivating Students to Learn. Fourth. New York: Routledge.
Wigfield, A. & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68–81. doi:10.1006/ceps.1999.1015.
Winchester, M. K., & Winchester, T. M. (2012). If you build it will they come?: Exploring the student perspective of weekly student evaluations of teaching. Assessment & Evaluation in Higher Education, 37(6), 671–682. Retrieved from http://10.0.4.56/02602938.2011.563278
Downloads
Published
Issue
Section
License
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
