Measuring Faculty Engagement in Online Formative or Whole-Person Education
A Revised Instrument and Item Response Theory Model
DOI:
https://doi.org/10.24059/olj.v27i4.4033Abstract
The COVID-19 pandemic reinforced the importance of supporting students’ comprehensive well-being when teaching online. One promising approach is formative or whole-person education, which emphasizes wholeness, purpose, and community. We created a scale using a
polytomous Item Response Theory modeling approach, measuring the extent to which postsecondary teachers engage in formative education online. To our knowledge, this is the first scale designed to measure this construct. The scale was developed within an exploratory sequential mixed methods study on formative education online that also included semi-structured interviews with 37 faculty members. Results from the qualitative analysis were used to develop initial items. This data-informed process increased the construct validity of the scale. We refined the original item pool through a pilot test using a sample of 308 instructors. This article presents psychometric results for the final, 10-item scale using a sample of 245 instructors. Evaluation of item fit statistics, item trace lines, and the total information curve indicate that the graded response model was appropriate for this scale. The Cronbach’s alpha and marginal reliability coefficients for the final scale were .90 and .91, indicating good reliability. Future research can explore how this scale might be adapted for in-person learning environments and other contexts.
References
Archibald, D. (2010). Fostering the development of cognitive presence: Initial findings using the community of inquiry survey instrument. Internet & Higher Education, 13(1/2), 73–74. https://doi.org/10.1016/j.iheduc.2009.10.001
Berry, S. (2017). Building community in online doctoral classrooms: Instructor practices that support community. Online Learning, 21(2), n2. http://dx.doi.org/10.24059/olj.v21i2.875
Berry, S. (2019). Teaching to connect: Community-building strategies for the virtual classroom. Online Learning, 23(1), 164-183. http://dx.doi.org/10.24059/olj.v23i1.1425
Borowiec, K., Kim, D., Wang, L., Kim, J., & Wortham, S. (2021). Supporting holistic student development through online community building. Online Learning, 25(4), 125-155. https://olj.onlinelearningconsortium.org/index.php/olj/article/view/2882
Boston College. (2007). The journey into adulthood: Understanding student formation. http://www.bc.edu/content/dam/files/offices/mission/pdf1/umm1.pdf
Castañeda, L., & Selwyn, N. (2018). More than tools? Making sense of the ongoing digitizations of higher education. International Journal of Educational Technology in Higher Education, 15(1), 22. https://doi.org/10.1186/s41239-018-0109-y
Chickering, A. W., & Reisser, L. (1993). Education and identity (2nd ed.). Jossey-Bass Inc.
Conklin, S. & Dikkers, A. G. (2021). Instructor social presence and connectedness in a quick shift from face-to-face to online instruction. Online Learning, 25(1), 135-150. https://doi.org/10.24059/olj.v25i1.2482
Creswell, J. W. (2015). A concise introduction to mixed methods research. Sage Publications.
Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Wadsworth Publishing.
De Ayala, R. J. (2009). The theory and practice of item response theory. Guilford Publications.
DeVellis, R. F. (2016). Scale development: Theory and applications. Sage Publications.
Ebel, R. L., & Frisbie, D. A. (1991). Essentials of educational measurement (5th ed). Prentice Hall.
Elharake, J. A., Akbar, F., Malik, A. A., Gilliam, W., & Omer, S. B. (2023). Mental health impact of COVID-19 among children and college students: A systematic review. Child Psychiatry & Human Development, 54, 913-925. https://doi.org/10.1007/s10578-021-01297-1
Freudenberg, N., Goldrick-Rab, S., & Poppendieck, J. (2019). College students and SNAP: The new face of food insecurity in the United States. American Journal of Public Health, 109(12), 1652-1658. https://ajph.aphapublications.org/doi/full/10.2105/AJPH.2019.305332
Garrison, D. R., & Akyol, Z. (2013). The community of inquiry theoretical framework. In M. G. Moore (Ed.), Handbook of distance education (3rd ed.) (pp. 104-119). Routledge.
Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. The Internet and Higher Education, 13(1), 5–9. https://doi.org/10.1016/j.iheduc.2009.10.003
Geger, F. B. T. (2014). Cura personalis: Some Ignatian inspirations. Jesuit Higher Education, 3(2), 6–20.
Goin Kono, K., & Taylor, S. (2021). Using an ethos of care to bridge the digital divide: Exploring faculty narratives during a global pandemic. Online Learning, 25(1), 151-165. https://doi.org/10.24059/olj.v25i1.2484
Goldrick-Rab, S., Coca, V., Kienzl, G., Welton, C. R., Dahl, S., & Magnelia, S. (2020). #RealCollege during the pandemic: New evidence on basic needs insecurity and student well-being. The Hope Center.
Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory (Vol. 2). Sage.
Hart, C.M.D., Xu, D., Hill, M., & Alonso, E. (2021). COVID-19 and community college instructional responses. Online Learning, 25(1), 41-69.
Healthy Minds Network, & American College Health Association. (2020). The impact of COVID-19 on college student well-being. https://healthymindsnetwork.org/wp-content/uploads/2020/07/Healthy_Minds_NCHA_COVID_Survey_Report_FINAL.pdf
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1-55. https://doi.org/10.1080/10705519909540118
IBM Corp. (2021). IBM SPSS Statistics for Windows (Version 28.0) [Computer software].
Kauffman, H. (2015). A review of predictive factors of student success in and satisfaction with online learning. Research in Learning Technology, 23, 26507. https://doi.org/10.3402/rlt.v23.26507
Kilgour, P., Reynaud, D., Northcote, M., McLoughlin, C., & Gosselin, K. P. (2019). Threshold concepts about online pedagogy for novice online teachers in higher education. Higher Education Research & Development, 38(7), 1417-1431. https://doi.org/10.19173/irrodl.v14i1.1338
Kim, D., Wortham, S., Borowiec, K., Yatsu, D. K., Ha, S., Carroll, S., Wang, L., & Kim, J. (2021). Formative education online: Teaching the whole person during the global COVID-19 pandemic. AERA Open, 7(1), 1-12. https://doi.org/10.1177/23328584211015229
Kolvenbach, P. H. (2007). Cura personalis [Care of the whole person]. Review of Ignatian Spirituality, 38(1), 9–17.
Kuh, G. (2018). Whither holistic student development: It matters more today than ever. Change: The Magazine of Higher Learning, 50(3-4), 52-57. https://doi.org/10.1080/00091383.2018.1509590
Lowenthal, P. R., & Dunlap, J. C. (2018). Investigating students’ perceptions of instructional strategies to establish social presence. Distance Education, 39(3), 281-298. https://doi.org/10.1080/01587919.2018.1476844
Martin, F., Budhrani, K., Kumar, S., & Ritzhaupt, A. (2019). Award-winning faculty online teaching practices: Roles and competencies. Online Learning, 23(1), 184–205. https://doi.org/10.24059/olj.v23i1.1329
Mathes, J. (2020, April 13). A defining moment for online learning. https://onlinelearningconsortium.org/a-defining-moment-for-online-learning/
McMillan, D. W., & Chavis, D. M. (1986). Sense of community: A definition and theory. Journal of Community Psychology, 14(1), 6-23. https://doi.org/10.1002/1520 6629(198601)14:1%3C6::AID-JCOP2290140103%3E3.0.CO;2-I
Miller, K.E. (2021). A light in students’ lives: K-12 teachers’ experiences (re)building caring relationships during remote learning. Online Learning, 25(1), 115-134. https://doi.org/10.24059/olj.v25i1.2486
Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
O’Malley, J. W. (2015). Jesuit schools of humanities: Yesterday and today. Seminar on Jesuit Spirituality, 47(1), 1–34. https://ejournals.bc.edu/index.php/jesuit/issue/view/641
Oswalt, S. B., Lederer, A. M., Chestnut-Steich, K., Day, C., Halbritter, A., & Ortiz, D. (2020). Trends in college students’ mental health diagnoses and utilization of services, 2009–2015. Journal of American College Health, 68(1), 41-51. https://doi.org/10.1080/07448481.2018.1515748
Patton, L. D., Renn, K. A., Guido, F. M., & Quaye, S. J. (2016). Student development in college: Theory, research, and practice. John Wiley & Sons.
Quezada, R. L., Talbot, C., & Quezada-Parker, K. B. (2020). From bricks and mortar to remote teaching: a teacher education programme’s response to COVID-19. Journal of Education for Teaching, 1-12.
Robinson, C. C., & Hullinger, H. (2008). New benchmarks in higher education: Student engagement in online learning. Journal of Education for Business, 84(2), 101–109. https://doi.org/10.3200/JOEB.84.2.101-109
Salmon, G. (2011). E-moderating: The key to teaching and learning online (3rd ed.). Routledge.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement. Psychometric Society.
Shea, P., & Bidjerano, T. (2009). Community of inquiry as a theoretical framework to foster “epistemic engagement” and “cognitive presence” in online education. Computers & Education, 52(3), 543-553. https://doi.org/10.1016/j.compedu.2008.10.007
Vector Psychometric Group. (2020). IRTPRO guide. https://vpgcentral.com/wp-content/uploads/2020/06/IRTPROGuide.pdf
Vector Psychometric Group. (2022). IRTPRO (Version 6.0) [Computer software].
Wortham, S., Love-Jones, R., Peters, W., Morris, S., & Garcia-Huidobro, J. C. (2020). Educating for comprehensive wellbeing. ECNU Review of Education, 3(3), 406–436. https://doi.org/10.1177/2096531120928448
Zhai, Y., & Du, X. (2020). Addressing collegiate mental health amid COVID-19 pandemic. Psychiatry Research, 288, 113003.
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Stanton Wortham, Katrina Borowiec, Deoksoon Kim
This work is licensed under a Creative Commons Attribution 4.0 International License.
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
