The Impact of Multimodal Communication on Learners’ Experience in a Synchronous Online Environment: A Mixed-Methods Study


  • Ying Cai The University of Texas at Austin
  • Zilong Pan Lehigh University
  • Songhee Han The University of Texas at Austin
  • Peixia Shao The University of Texas at Austin
  • Min Liu The University of Texas at Austin



Synchronous online learning, Social presence, Teaching presence, Multimodality


During the COVID-19 pandemic, from early 2020 onwards, the adoption of synchronous online learning increased rapidly. It offers students a unique learning experience, utilizing communication modes from both in-person and asynchronous online classes. This mixed-methods study examined the impact of modes of communication (visual, bodily behaviors, spoken language, and written language) found in synchronous online contexts on students’ learning experiences from the perspective of social presence and teaching presence, as well as their satisfaction with synchronous online learning experience. An online survey was distributed first to collect quantitative data. The survey results indicated that four different modes influenced students’ communication to a different extent, with written and spoken language being the most effective modes of online communication. These modes were also significantly positively correlated with social presence, teaching presence, and student satisfaction; however, only spoken language was a significant predictor of student satisfaction. In the qualitative phase, semi-structured interviews were conducted to examine students’ perceptions of how multimodality affects social presence, teaching presence, and satisfaction with online learning. This led to five major themes and highlighted how multiple modes of communication supports social presence, thereby helping teachers scaffold students. In addition, the online learning context impacts type of instruction, and the reduced distance between teachers and students improves teaching presence; however, the students felt a lack of affective belonging in their online classes. This study also provided implications for course instructors and designers to help them effectively adopt different modes in synchronous online environments and promote social and teaching presence.


Abuhassna, H., Al-Rahmi, W. M., Yahya, N., Zakaria, M. A. Z. M., Kosnin, A. B., & Darwish, M. (2020). Development of a new model on utilizing online learning platforms to improve students’ academic achievements and satisfaction. International Journal of Educational Technology in Higher Education, 17(1), 1-23.

Afolabi, R.O. (2016). Emphasizing instructor presence in distance learning environments. In S. D’Agustino (Ed.), Creating teacher immediacy in online learning environments (pp. 37-55). IGI Global.

Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer conferencing context. Journal of the Asynchronous Learning Network, 5(2), 1–17.

Angelone, L., Warner, Z., & Zydney, J. M. (2020). Optimizing the technological design of a blended synchronous learning environment. Online Learning, 24(3), 222-240.

Arbaugh, J. B., Cleveland-Innes, M., Diaz, S. R., Garrison, D. R., Ice, P., Richardson, J. C., & Swan, K. P. (2008). Developing a community of inquiry instrument: Testing a Community of Inquiry framework using a multi-institutional sample. The Internet and Higher Education, 11(3), 133–136.

Arbaugh, J. B., & Hwang, A. (2006). Does “teaching presence” exist in online MBA courses?. The Internet and Higher Education, 9(1), 9-21.

Astin, A.W. (1993). What matters in college? Four critical years revisited. Jossey-Bass.

Baker, C. (2010). The impact of instructor immediacy and presence for online student affective learning, cognition, and motivation. The Journal of Educators Online, 7(1).

Bezemer, J., & Kress, G. (2016). Multimodality, learning, and communication: A social semiotic frame. Routledge.

Bolliger, D. U., & Martindale, T. (2004). Key factors for determining student satisfaction in online courses. International Journal on E-learning, 3(1), 61-67.

Bozkaya, M. (2008). The relationship between teacher immediacy behaviours and distant learners' social presence perceptions in videoconferencing applications. The Turkish Online Journal of Distance Education, 9 (1), 180–192.

Bray, E., Aoki, K., & Dlugosh, L. (2008). Predictors of learning satisfaction in Japanese online distance learners. International Review of Research in Open and Distributed Learning, 9(3), 1-24.

Cheung, A. (2021). Synchronous online teaching, a blessing or a curse? Insights from EFL primary students’ interaction during online English lessons. System, 100, 102566.

Corbin, J. M., & Strauss, A. (1990). Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative Sociology, 13(1), 3-21.

Creswell, J. W., & Clark, V. L. P. (2017). Designing and conducting mixed methods research. Sage.

Creswell, J. W., & Poth, C. N. (2016). Qualitative inquiry and research design: Choosing among five approaches. Sage

Cunningham, U. (2014). Teaching the disembodied: Othering and activity systems in a blended synchronous learning situation. The International Review of Research in Open and Distributed Learning, 15(6), 33–51.

Debourgh, G. A. (1999). Technology is the tool, teaching is the task: Student satisfaction in distance learning. In J. D. Price, J. Willis, D. A. Willis, M. Jost, & S. Boger-Mehall (Eds.), Proceedings of the Society for Information Technology & Teacher Education International Conference (pp. 131-137). Association for the Advancement of Computing in Education (AACE).

Erfanian Mohammadi, J., Elahi Shirvan, M., & Akbari, O. (2019). Systemic functional multimodal discourse analysis of teaching students developing classroom materials. Teaching in Higher Education, 24(8), 964-986.

Forceville, C. (2020). Visual and multimodal communication: Applying the relevance principle. Oxford University Press.

Garcia, A. C., & Baker Jacobs, J. (1999). The eyes of the beholder: Understanding the turn-taking system in quasi-synchronous computer-mediated communication. Research on language and social interaction, 32(4), 337-367.

Garrison, D. R., & Arbaugh, J. B. (2007). Researching the community of inquiry framework: Review, issues, and future directions. The Internet and Higher Education, 10(3), 157–172.

Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2), 87–105.

Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7-23.

Garrison, D. R., & Cleveland-Innes, M. (2005). Facilitating cognitive presence in online learning: Interaction is not enough. The American Journal of Distance Education, 19(3), 133-148.

Garrison, R. (2009). Implications of online and blended learning for the conceptual development and practice of distance education. International Journal of E-Learning & Distance Education, 23(2), 93-104.

Glaser, B. G., & Strauss, A. L. (2017). Discovery of grounded theory: Strategies for qualitative research. Routledge.

Hawkes, T., & Hawkes, T. (1977). Structuralism & semiotics. London: Methuen.

Henriksen, D., Creely, E. & Henderson, M. (2020). Folk pedagogies for teacher transitions: Approaches to synchronous online learning in the wake of COVID-19. Journal of Technology and Teacher Education, 28(2), 201-209.

Hodges, C., Moore, S., Lockee, B., Trust, T., & Bond, A. (2020). The difference between emergency remote teaching and online learning. Educause Review, 27, 1-12.

Hoffman, E. B. (2018). Untangling the talk: A new multimodal discourse analysis method to investigate synchronous online learning. Journal of Digital Learning in Teacher Education, 34(3), 179–195.

Husserl, E. (1962). General introduction to pure phenomenology. Collier.

Jewitt, C. E. (2011). The Routledge handbook of multimodal analysis. Routledge/Taylor & Francis Group.

Kress, G., & Van Leeuwen, T. (2001). Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Arnold.

Kuo, Y. C., Walker, A. E., Schroder, K. E., & Belland, B. R. (2014). Interaction, internet self-efficacy, and self-regulated learning as predictors of student satisfaction in online education courses. The Internet and Higher Education, 20, 35-50.

Lambert, J. L., & Fisher, J. L. (2013). Community of inquiry framework: Establishing community in an online course. Journal of Interactive Online Learning, 12(1).

Landrum, B., Bannister, J., Garza, G., & Rhame, S. (2021). A class of one: Students’ satisfaction with online learning. Journal of Education for Business, 96(2), 82-88.

Lee, J., & Lee, Y. (2006). Personality types and learners’ interaction in web-based threaded discussion. Quarterly Review of Distance Education, 7(1), 83.

Liaw, M. L., & Ware, P. (2018). Multimodality and social presence in an intercultural exchange setting. In R. Kern & C. Develotte (Eds.), Screens and scenes (pp. 256-278). Routledge.

Lowenthal, P. R., West, R.E., Archambault, L., Borup, J., & Belt, E. S. (2021). Faculty perceptions of using synchronous video-based communication technology. Online Learning, 25(4), 74-103. DOI: 10.24059/olj.v25i4.2890

Lumsden, J., Miles, L. K., & Macrae, C. N. (2014). Sync or sink? Interpersonal synchrony impacts self-esteem. Frontiers in Psychology, 1064.

Malkawi, E., Bawaneh, A. K., & Bawa'aneh, M. S. (2020). Campus off, education on: UAEU Students’ satisfaction and attitudes towards e-learning and virtual classes during COVID-19 pandemic. Contemporary Educational Technology, 13(1), ep283.

Marsh, K. L., Richardson, M. J., & Schmidt, R. C. (2009). Social connection through joint action and interpersonal coordination. Topics in Cognitive Science, 1(2), 320-339.

Martin, F., Wu, T., Wan, L., & Xie, K. (2022). A Meta-analysis on the community of inquiry presences and learning outcomes in online and blended learning environments. Online Learning, 26(1).

McBrien, J. L., Cheng, R., & Jones, P. (2009). Virtual spaces: Employing a synchronous online classroom to facilitate student engagement in online learning. International Review of Research in Open and Distributed Learning, 10(3).

Merriam, S. B., & Tisdell, E. J. (2015). Qualitative research: A guide to design and implementation. John Wiley & Sons.

Meskill, C., & Anthony, N. (2010). Teaching language online. Multilingual Matters.

Miles, M. B., Huberman, A. M., & Saldana, J. (2013). Qualitative data analysis: A methods sourcebook (3rd ed.). SAGE.

Mood, T. A. (1995). Distance education: An annotated bibliography. Libraries Unlimited.

Peterson, A. T., Beymer, P. N., & Putnam, R. T. (2018). Synchronous and asynchronous discussions: Effects on cooperation, belonging, and affect. Online Learning, 22(4), 7-25.

Poquet, O., Kovanović, V., de Vries, P., Hennis, T., Joksimović, S., Gašević, D., & Dawson, S. (2018). Social presence in massive open online courses. International Review of Research in Open and Distributed Learning, 19(3).

Romero-Hall, E., & Vicentini, C. R. (2017). Examining distance learners in hybrid synchronous instruction: Successes and challenges. Online Learning Journal, 21(4).

Rosenman, R., Tennekoon, V., & Hill, L. G. (2011). Measuring bias in self-reported data. International Journal of Behavioural & Healthcare Research, 2(4), 320.

Satar, H. M. (2015). Sustaining multimodal language learner interactions online. CALICO Journal, 32(3), 480–507.

Satar, M. (2020). L1 for social presence in video-conferencing: A social semiotic account. Language Learning & Technology, 24(1), 129–153.

Silverman, K. (1983). The subject of semiotics. Oxford University Press.

Stewart, W. H., Baek, Y., & Lowenthal, P. R. (2022). From emergency remote teaching (ERT) to sustained remote teaching (SRT): A comparative semester analysis of exchange students’ experiences and perceptions of learning online during COVID-19. Online Learning, 26(2). DOI:

Swan, K., Shea, P., Richardson, J., Ice, P., Garrison, D. R., Cleveland-Innes, M., & Arbaugh, J. B. (2008). Validating a measurement tool of presence in online communities of inquiry. e-Mentor, 24(2).

Tashakkori, A., & Teddlie, C. (2003). Issues and dilemmas in teaching research methods courses in social and behavioural sciences: US perspective. International Journal of Social Research Methodology, 6(1), 61-77.

Tichavsky, L. P., Hunt, A. N., Driscoll, A., & Jicha, K. (2015). “It’s just nice having a real teacher”: Student perceptions of online versus face-to-face instruction. International Journal for the Scholarship of Teaching & Learning, 9(2), 1–8.

Toohey, K., Dagenais, D., Fodor, A., Hof, L., Nuñez, O., Singh, A., & Schulze, L. (2015). That sounds so cooool: Entanglements of children, digital tools, and literacy practices. Tesol Quarterly, 49(3), 461-485.

Turk, M., Müftüoglu, A. C., & Toraman, S. (2021). Teaching presence in online courses: Similar perceptions but different experiences from multiple instructor perspectives. Online Learning, 25(4), 156-177.

Van Leeuwen, T. (2005). Introducing social semiotics. Psychology Press.

Wang, Q., & Huang, C. (2018). Pedagogical, social and technical designs of a blended synchronous learning environment. British Journal of Educational Technology, 49(3), 451–462. http://doi:10.1111/bjet.12558

Watson, S. L., Watson, W. R., Richardson, J., & Loizzo, J. (2016). Instructor’s use of social presence, teaching presence, and attitudinal dissonance: A case study of an attitudinal change MOOC. International Review of Research in Open and Distributed Learning, 17(3), 54-74.

Zou, W., Hu, X., Pan, Z., Li, C., Cai, Y., & Liu, M. (2021). Exploring the relationship between social presence and learners’ prestige in MOOC discussion forums using automated content analysis and social network analysis. Computers in Human Behavior, 115, article 106582.






Special Conference Issue: AERA Online Teaching and Learning SIG