Deep Learning Models for Analyzing Social Construction of Knowledge Online

Charlotte N. Gunawardena; Yan Chen; Nick Flor; Damien Sanchez

doi:10.24059/olj.v27i4.4055

Authors

Charlotte N. Gunawardena University of New Mexico https://orcid.org/0000-0002-3952-8908
Yan Chen University of Nevada, Las Vegas https://orcid.org/0000-0002-9479-7377
Nick Flor University of New Mexico https://orcid.org/0000-0002-5162-9277
Damien Sanchez https://orcid.org/0000-0001-5810-4994

DOI:

https://doi.org/10.24059/olj.v27i4.4055

Abstract

Gunawardena et al.’s (1997) Interaction Analysis Model (IAM) is one of the most frequently employed frameworks to guide the qualitative analysis of social construction of knowledge online. However, qualitative analysis is time consuming, and precludes immediate feedback to revise online courses while being delivered. To expedite analysis with a large dataset, this study explores how two neural network architectures—a feed-forward network (Doc2Vec) and a large language model transformer (BERT)—could automatically predict phases of knowledge construction using IAM. The methods interrogated the extent to which the artificial neural networks’ predictions of IAM Phases approximated a human coder’s qualitative analysis. Key results indicate an accuracy of 21.55% for Doc2Vec phases I-V, 43% for fine-tuning a pre-trained large language model (LLM), and 52.79% for prompt-engineering an LLM. Future studies for improving accuracy should consider either training the models with larger datasets or focusing on the design of prompts to improve classification accuracy. Grounded on social constructivism and IAM, this study has implications for designing and supporting online collaborative learning where the goal is social construction of knowledge. Moreover, it has teaching implications for guiding the design of AI tools that provide beneficial feedback for both students and course designers.

Author Biography

Charlotte N. Gunawardena, University of New Mexico

Charlotte N. Gunawardena, Ph.D.

Distinguished Professor

Organization, Information and Learning Sciences

University of New Mexico, Albuquerque NM 87131

Email: lani@unm.edu

References

Ahmad, M., Junus, K., & Santoso, H. B. (2022). Automatic content analysis of asynchronous discussion forum transcripts: A systematic literature review. Education and Information Technologies, 27(8), 11355–11410. https://doi.org/10.1007/s10639-022-11065-w

Ba, S., Hu, X., Stein, D., & Liu, Q. (2023). Assessing cognitive presence in online inquiry‐based discussion through text classification and epistemic network analysis. British Journal of Educational Technology, 54(1), 247–266. https://doi.org/10.1111/bjet.13285

Buckingham Shum, S., & Ferguson, R. (2012). Social learning analytics. Educational Technology & Society, 15(3), 3–26.

Buraphadeja, V., & Dawson, K. (2008). Content analysis in computer-mediated communication: Analyzing models for assessing critical thinking through the lens of social constructivism. American Journal of Distance Education, 22(3), 130–145. https://doi.org/10.1080/08923640802224568

Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Sage.

Commander, N., Zhao, Y., Gallagher, P., & You, Y. (2016). Cross-national online discussions: International learning experiences with American and Chinese students in higher education. Innovations in Education and Teaching International, 53(4), 365–374.

Culture e-Learning 1 (2022a). [Dataset]. https://github.com/professorf/IAM-Data-Code/Culture- eLearning-1.csv

Culture e-Learning 2 (2022b). [Dataset]. https://github.com/professorf/IAM-Data-Code/Culture- eLearning-2.csv

Culture e-Learning 3 (2022c). [Dataset]. https://github.com/professorf/IAM-Data-Code/Culture- eLearning-3.csv

Culture e-Learning B (2022d). [Dataset]. https://github.com/professorf/IAM-Data- Code/Culture-eLearning-B.csv

Dillenbourg, P. (1999). What do you mean by collaborative learning? In Pierre Dillenbourg (Ed.), Collaborative-learning: Cognitive and computational approaches (pp. 1–19). Elsevier.

Duffy, T. M., & Jonassen, D. (1992). Constructivism and the technology of instruction: A conversation. Lawrence Erlbaum Associates.

Duvall, M., Matranga, A., & Silverman, J. (2020). Designing for and facilitating knowledge-building discourse in online courses. Information and Learning Sciences, 121(7/8), 487–501. https://doi.org/10.1108/ILS-04-2020-0081

Floriasti, T. W. (2023). Analysis of online discussion forum in constructing English language skills through collaborative learning of preservice teachers in Indonesia. Retrieved from https://www.mextesol.net/journal/index.php?page=journal&id_article=46400

Garrison, D. R. (2023). Online learning and AI (Editorial). The Community of Inquiry. Retrieved from https://www.thecommunityofinquiry.org/editorial41

Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7–23. https://doi.org/10.1080/08923640109527071

Garrison, D. R., Cleveland-Innes, M., Koole, M., & Kappelman, J. (2006). Revisiting methodological issues in transcript analysis: Negotiated coding and reliability. The Internet and Higher Education, 9(1), 1–8. https://doi.org/10.1016/j.iheduc.2005.11.001

Goodfellow, R., & Hewling, A. (2005). Reconceptualising culture in virtual learning environments: From an ‘essentialist’ to a ‘negotiated’ perspective. E-Learning and Digital Media, 2(4), 355–367. https://doi.org/10.2304/elea.2005.2.4.355

Gunawardena, C. N., Flor, N. V., Gómez, D., & Sánchez, D. (2016). Analyzing social construction of knowledge online by employing interaction analysis, learning analytics, and social network analysis. Quarterly Review of Distance Education, 17(3), 35–60.

Gunawardena, C. N., Lowe, C. A., & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397–431. https://doi.org/10.2190/7MQV-X9UJ-C7Q3-NRAG

Guo, C., Shea, P., & Chen, X. (2022). Investigation on graduate students’ social presence and social knowledge construction in two online discussion settings. Education and Information Technologies, 27(2), 2751–2769. https://doi.org/10.1007/s10639-021-10716-8

Hew, K. F., & Cheung, W. S. (2011). Higher-level knowledge construction in asynchronous online discussions: An analysis of group size, duration of online discussion, and student facilitation techniques. Instructional Science, 39(3), 303–319. https://doi.org/10.1007/s11251-010-9129-2

Hinton, G., Osindero, S., & Teh, Y. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527–1554.

Howell, G. S., Sutherlin, A., Akpanudo, U., James, L., & Chen, M. (2014). The effect of structured divergent prompts on knowledge construction. Online Learning, 18(2). https://doi.org/10.24059/olj.v18i2.410

Hu, Y., Donald, C., Giacaman, N., & Zhu, Z. (2020). Towards automated analysis of cognitive presence in MOOC discussions: A manual classification study. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 135–140. https://doi.org/10.1145/3375462.3375473

John-Steiner, V., & Mahn, H. (1996). Sociocultural approaches to learning and development: A Vygotskian framework. Educational Psychologist, 31(3), 191–206. https://doi.org/10.1207/s15326985ep3103&4_4

Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415

Kanuka, H., & Anderson, T. (1998). Online social interchange, discord, and knowledge construction. Journal of Distance Education, 13, 57–74.

Kavlakoglu, E. (2020). AI vs. Machine Learning vs. Deep Learning vs. Neural Networks: What’s the difference? IBM Blog. https://www.ibm.com/cloud/blog/ai-vs-machine- learning-vs-deep-learning-vs-neural-networks

Kaliisa, R., Rienties, B., Mørch, A. I., & Kluge, A. (2022). Social learning analytics in computer-supported collaborative learning environments: A systematic review of empirical studies. Computers and Education Open, 3, 100073. https://doi.org/10.1016/j.caeo.2022.100073

Koschmann, T. (1996). Paradigm shifts and instructional technology: An introduction. In T. Koschmann (Ed.), CSCL: Theory and Practice of an Emerging Paradigm (pp. 1–24). Lawrence Erlbaum Associates.

Kumpulainen, K., & Wray, D. (2002). Classroom interaction and social learning: From theory to practice. Routledge.

Le, Q., & Mikolov, T. (2014). Distributed representations of sentences and documents. Proceedings of the 31st International Conference on Machine Learning, 32(2), 1188–1196.

https://doi.org/10.48550/ARXIV.1405.4053

Lee, L. H. J., Rahmat, R. B., Lin, L., Lim, P. H., & Tan, T. H. (2023). The development of an implementation framework to support knowledge construction in online networked learning. Professional Development in Education, 49(1), 69–90. https://doi.org/10.1080/19415257.2020.1763430

Lehtinen, A., Kostiainen, E., & Näykki, P. (2023). Co-construction of knowledge and socioemotional interaction in pre-service teachers’ video-based online collaborative learning. Teaching and Teacher Education, 133, 104299. https://doi.org/10.1016/j.tate.2023.104299

Lin, C.-L., Hou, H.-T., & Tsai, C.-C. (2016). Analyzing the social knowledge construction and online searching behavior of high school learners during a collaborative problem solving learning activity: A multi-dimensional behavioral pattern analysis. The Asia-Pacific Education Researcher, 25(5–6), 893–906. https://doi.org/10.1007/s40299-016-0317-y

Lucas, M., Gunawardena, C., & Moreira, A. (2014). Assessing social construction of knowledge online: A critique of the interaction analysis model. Computers in Human Behavior, 30, 574–582. https://doi.org/10.1016/j.chb.2013.07.050

Luebeck, J. L., & Bice, L. R. (2005). Online discussion as a mechanism of conceptual change among mathematics and science teachers. Journal of Distance Education, 20(2), 21–39.

McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5, 115–133.

Megli, A. C., Fallad-Mendoza, D., Etsitty-Dorame, M., Desiderio, J., Chen, Y., Sanchez, D., Flor, N., & Gunawardena, C. (2023a). Interaction Analysis Model Lexicon. University of New Mexico Digital Repository, 1–14. https://digitalrepository.unm.edu/cgi/viewcontent.cgi?article=1001&context=oils_sp

Megli, A. C., Fallad-Mendoza, D., Etsitty-Dorame, M., Desiderio, J., Chen, Y., Sanchez, D., Flor, N., & Gunawardena, C. N. (2023b). Using social learning analytic methods to examine social construction of knowledge in online discussions. American Journal of Distance Education, 1–16. https://doi.org/10.1080/08923647.2023.2192597

Moore, S. L. & Tillberg-Webb, H. K. (2023). Ethics and educational technology: Reflection, interrogation and design as a framework for practice. Routledge.

Nguyen, H., & Diederich, M. (2023). Facilitating knowledge construction in informal learning: A study of TikTok scientific, educational videos. Computers & Education, 205, 104896. https://doi.org/10.1016/j.compedu.2023.104896

O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19, 160940691989922. https://doi.org/10.1177/1609406919899220

Oppermann, A. (2022) What is deep learning and how does it work? Builtin. Retrieved from https://builtin.com/machine-learning/deep-learning

Ouyang, F., Xu, W., & Cukurova, M. (2023). An artificial intelligence-driven learning analytics method to examine the collaborative problem-solving process from the complex adaptive systems perspective. International Journal of Computer-Supported Collaborative Learning. https://doi.org/10.1007/s11412-023-09391-3

Pea, R. D. (1993). Practices of distributed intelligences and design for education. In G. Solomon (Ed.), Distributed cognitions: Psychological and educational considerations (pp. 47–87). Cambridge University Press.

Piaget, J. (1971). The theory of stages in cognitive development. In D. R. Green, M. P. Ford, & G. B. Flamer, Measurement and Piaget (pp. 1–11). McGraw-Hill.

Rannikmäe, M., Holbrook, J., & Soobard, R. (2020). Social Constructivism—Jerome Bruner. In B. Akpan & T. J. Kennedy (Eds.), Science education in theory and practice (pp. 259–275). Springer International Publishing. https://doi.org/10.1007/978-3-030-43620-9_18

Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65, 386–408.

Rourke, L., Anderson, T., Garrison, D. R., & Archer, A. (2002). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12, 8–22. https://telearn.hal.science/hal-00197319/document

Rumelhart, D., Hinton, G., & Williams, R. (1986). Learning internal representations by error propagation. In D. Rumelhart & J. McClelland (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations (pp. 318–362). Bradford Books/MIT Press.

Sammons, M. (2007). Collaborative interaction. In M.G. Moore (Ed.), Handbook of distance education (2nd ed.; pp. 311–321). Lawrence Erlbaum.

Sanchez, D. M. (2019). Building a call to action: Social action in networks of practice [Doctoral dissertation, University of New Mexico]. The University of New Mexico Digital Repository. https://digitalrepository.unm.edu/oils_etds/51/

Sanchez, D., Flor, N., & Gunawardena, C. (2020). Employing social learning analytic methods (SLAMs) to reimagine the social dynamic of online learning collaborations. In M. Brown, M. Nic Giolla Mhichil, E. Beirne, & E. Costello (Eds.), Proceedings of the 2019 ICDE World Conference on Online Learning, Volume 1, (pp. 817–832). Dublin City University, Dublin, Ireland. http://dx.doi.org/10.5281/zenodo.3804014

Schaaf, S. (2020). Social network centrality, social construction of knowledge, and nurse practitioner competency in asynchronous online discussions among adult gerontology acute care nurse practitioner students. [Doctoral Dissertation, University of New Mexico]. UNM Campus Repository. https://digitalrepository.unm.edu/oils_etds/59

Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85–117. https://doi.org/10.1016/j.neunet.2014.09.003

Silverio, S., Hall, J., & Sandall, J. (2020, January). Time and qualitative research: Principles, pitfalls, and perils. [Conference presentation]. Sixth Annual Qualitative Research Symposium—Tick Tock: Unpacking Temporal Aspects of Qualitative Inquiry, University of Bath, Bath, UK.

Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. The MIT Press. https://doi.org/10.7551/mitpress/3372.001.0001

Tao, Y., Zhang, M., Su, Y., & Li, Y. (2022). Exploring college English language learners’ social knowledge construction and socio-emotional interactions during computer-supported collaborative writing activities. The Asia-Pacific Education Researcher, 31(5), 613–622. https://doi.org/10.1007/s40299-021-00612-7

Tirado, R., Hernando, Á., & Aguaded, J. I. (2015). The effect of centralization and cohesion on the social construction of knowledge in discussion forums. Interactive Learning Environments, 23(3), 293–316. https://doi.org/10.1080/10494820.2012.745437

Valtonen, T., López-Pernas, S., Saqr, M., Vartiainen, H., Sointu, E. T., & Tedre, M. (2022). The nature and building blocks of educational technology research. Computers in Human Behavior, 128, 107123. https://doi.org/10.1016/j.chb.2021.107123

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. , ... & Polosukhin, I. (2017). Attention is all you need [Conference presentation]. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, United States.

Vygotsky, L. S. (1978). Mind and society: The development of higher psychological processes. Harvard University Press.