Deep Learning Models for Analyzing Social Construction of Knowledge Online
DOI:
https://doi.org/10.24059/olj.v27i4.4055Abstract
Gunawardena et al.’s (1997) Interaction Analysis Model (IAM) is one of the most frequently employed frameworks to guide the qualitative analysis of social construction of knowledge online. However, qualitative analysis is time consuming, and precludes immediate feedback to revise online courses while being delivered. To expedite analysis with a large dataset, this study explores how two neural network architectures—a feed-forward network (Doc2Vec) and a large language model transformer (BERT)—could automatically predict phases of knowledge construction using IAM. The methods interrogated the extent to which the artificial neural networks’ predictions of IAM Phases approximated a human coder’s qualitative analysis. Key results indicate an accuracy of 21.55% for Doc2Vec phases I-V, 43% for fine-tuning a pre-trained large language model (LLM), and 52.79% for prompt-engineering an LLM. Future studies for improving accuracy should consider either training the models with larger datasets or focusing on the design of prompts to improve classification accuracy. Grounded on social constructivism and IAM, this study has implications for designing and supporting online collaborative learning where the goal is social construction of knowledge. Moreover, it has teaching implications for guiding the design of AI tools that provide beneficial feedback for both students and course designers.
References
Ahmad, M., Junus, K., & Santoso, H. B. (2022). Automatic content analysis of asynchronous discussion forum transcripts: A systematic literature review. Education and Information Technologies, 27(8), 11355–11410. https://doi.org/10.1007/s10639-022-11065-w
Ba, S., Hu, X., Stein, D., & Liu, Q. (2023). Assessing cognitive presence in online inquiry‐based discussion through text classification and epistemic network analysis. British Journal of Educational Technology, 54(1), 247–266. https://doi.org/10.1111/bjet.13285
Buckingham Shum, S., & Ferguson, R. (2012). Social learning analytics. Educational Technology & Society, 15(3), 3–26.
Buraphadeja, V., & Dawson, K. (2008). Content analysis in computer-mediated communication: Analyzing models for assessing critical thinking through the lens of social constructivism. American Journal of Distance Education, 22(3), 130–145. https://doi.org/10.1080/08923640802224568
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research (3rd ed.). Sage.
Commander, N., Zhao, Y., Gallagher, P., & You, Y. (2016). Cross-national online discussions: International learning experiences with American and Chinese students in higher education. Innovations in Education and Teaching International, 53(4), 365–374.
Culture e-Learning 1 (2022a). [Dataset]. https://github.com/professorf/IAM-Data-Code/Culture- eLearning-1.csv
Culture e-Learning 2 (2022b). [Dataset]. https://github.com/professorf/IAM-Data-Code/Culture- eLearning-2.csv
Culture e-Learning 3 (2022c). [Dataset]. https://github.com/professorf/IAM-Data-Code/Culture- eLearning-3.csv
Culture e-Learning B (2022d). [Dataset]. https://github.com/professorf/IAM-Data- Code/Culture-eLearning-B.csv
Dillenbourg, P. (1999). What do you mean by collaborative learning? In Pierre Dillenbourg (Ed.), Collaborative-learning: Cognitive and computational approaches (pp. 1–19). Elsevier.
Duffy, T. M., & Jonassen, D. (1992). Constructivism and the technology of instruction: A conversation. Lawrence Erlbaum Associates.
Duvall, M., Matranga, A., & Silverman, J. (2020). Designing for and facilitating knowledge-building discourse in online courses. Information and Learning Sciences, 121(7/8), 487–501. https://doi.org/10.1108/ILS-04-2020-0081
Floriasti, T. W. (2023). Analysis of online discussion forum in constructing English language skills through collaborative learning of preservice teachers in Indonesia. Retrieved from https://www.mextesol.net/journal/index.php?page=journal&id_article=46400
Garrison, D. R. (2023). Online learning and AI (Editorial). The Community of Inquiry. Retrieved from https://www.thecommunityofinquiry.org/editorial41
Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer conferencing in distance education. American Journal of Distance Education, 15(1), 7–23. https://doi.org/10.1080/08923640109527071
Garrison, D. R., Cleveland-Innes, M., Koole, M., & Kappelman, J. (2006). Revisiting methodological issues in transcript analysis: Negotiated coding and reliability. The Internet and Higher Education, 9(1), 1–8. https://doi.org/10.1016/j.iheduc.2005.11.001
Goodfellow, R., & Hewling, A. (2005). Reconceptualising culture in virtual learning environments: From an ‘essentialist’ to a ‘negotiated’ perspective. E-Learning and Digital Media, 2(4), 355–367. https://doi.org/10.2304/elea.2005.2.4.355
Gunawardena, C. N., Flor, N. V., Gómez, D., & Sánchez, D. (2016). Analyzing social construction of knowledge online by employing interaction analysis, learning analytics, and social network analysis. Quarterly Review of Distance Education, 17(3), 35–60.
Gunawardena, C. N., Lowe, C. A., & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397–431. https://doi.org/10.2190/7MQV-X9UJ-C7Q3-NRAG
Guo, C., Shea, P., & Chen, X. (2022). Investigation on graduate students’ social presence and social knowledge construction in two online discussion settings. Education and Information Technologies, 27(2), 2751–2769. https://doi.org/10.1007/s10639-021-10716-8
Hew, K. F., & Cheung, W. S. (2011). Higher-level knowledge construction in asynchronous online discussions: An analysis of group size, duration of online discussion, and student facilitation techniques. Instructional Science, 39(3), 303–319. https://doi.org/10.1007/s11251-010-9129-2
Hinton, G., Osindero, S., & Teh, Y. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527–1554.
Howell, G. S., Sutherlin, A., Akpanudo, U., James, L., & Chen, M. (2014). The effect of structured divergent prompts on knowledge construction. Online Learning, 18(2). https://doi.org/10.24059/olj.v18i2.410
Hu, Y., Donald, C., Giacaman, N., & Zhu, Z. (2020). Towards automated analysis of cognitive presence in MOOC discussions: A manual classification study. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 135–140. https://doi.org/10.1145/3375462.3375473
John-Steiner, V., & Mahn, H. (1996). Sociocultural approaches to learning and development: A Vygotskian framework. Educational Psychologist, 31(3), 191–206. https://doi.org/10.1207/s15326985ep3103&4_4
Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. https://doi.org/10.1126/science.aaa8415
Kanuka, H., & Anderson, T. (1998). Online social interchange, discord, and knowledge construction. Journal of Distance Education, 13, 57–74.
Kavlakoglu, E. (2020). AI vs. Machine Learning vs. Deep Learning vs. Neural Networks: What’s the difference? IBM Blog. https://www.ibm.com/cloud/blog/ai-vs-machine- learning-vs-deep-learning-vs-neural-networks
Kaliisa, R., Rienties, B., Mørch, A. I., & Kluge, A. (2022). Social learning analytics in computer-supported collaborative learning environments: A systematic review of empirical studies. Computers and Education Open, 3, 100073. https://doi.org/10.1016/j.caeo.2022.100073
Koschmann, T. (1996). Paradigm shifts and instructional technology: An introduction. In T. Koschmann (Ed.), CSCL: Theory and Practice of an Emerging Paradigm (pp. 1–24). Lawrence Erlbaum Associates.
Kumpulainen, K., & Wray, D. (2002). Classroom interaction and social learning: From theory to practice. Routledge.
Le, Q., & Mikolov, T. (2014). Distributed representations of sentences and documents. Proceedings of the 31st International Conference on Machine Learning, 32(2), 1188–1196.
https://doi.org/10.48550/ARXIV.1405.4053
Lee, L. H. J., Rahmat, R. B., Lin, L., Lim, P. H., & Tan, T. H. (2023). The development of an implementation framework to support knowledge construction in online networked learning. Professional Development in Education, 49(1), 69–90. https://doi.org/10.1080/19415257.2020.1763430
Lehtinen, A., Kostiainen, E., & Näykki, P. (2023). Co-construction of knowledge and socioemotional interaction in pre-service teachers’ video-based online collaborative learning. Teaching and Teacher Education, 133, 104299. https://doi.org/10.1016/j.tate.2023.104299
Lin, C.-L., Hou, H.-T., & Tsai, C.-C. (2016). Analyzing the social knowledge construction and online searching behavior of high school learners during a collaborative problem solving learning activity: A multi-dimensional behavioral pattern analysis. The Asia-Pacific Education Researcher, 25(5–6), 893–906. https://doi.org/10.1007/s40299-016-0317-y
Lucas, M., Gunawardena, C., & Moreira, A. (2014). Assessing social construction of knowledge online: A critique of the interaction analysis model. Computers in Human Behavior, 30, 574–582. https://doi.org/10.1016/j.chb.2013.07.050
Luebeck, J. L., & Bice, L. R. (2005). Online discussion as a mechanism of conceptual change among mathematics and science teachers. Journal of Distance Education, 20(2), 21–39.
McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The Bulletin of Mathematical Biophysics, 5, 115–133.
Megli, A. C., Fallad-Mendoza, D., Etsitty-Dorame, M., Desiderio, J., Chen, Y., Sanchez, D., Flor, N., & Gunawardena, C. (2023a). Interaction Analysis Model Lexicon. University of New Mexico Digital Repository, 1–14. https://digitalrepository.unm.edu/cgi/viewcontent.cgi?article=1001&context=oils_sp
Megli, A. C., Fallad-Mendoza, D., Etsitty-Dorame, M., Desiderio, J., Chen, Y., Sanchez, D., Flor, N., & Gunawardena, C. N. (2023b). Using social learning analytic methods to examine social construction of knowledge in online discussions. American Journal of Distance Education, 1–16. https://doi.org/10.1080/08923647.2023.2192597
Moore, S. L. & Tillberg-Webb, H. K. (2023). Ethics and educational technology: Reflection, interrogation and design as a framework for practice. Routledge.
Nguyen, H., & Diederich, M. (2023). Facilitating knowledge construction in informal learning: A study of TikTok scientific, educational videos. Computers & Education, 205, 104896. https://doi.org/10.1016/j.compedu.2023.104896
O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19, 160940691989922. https://doi.org/10.1177/1609406919899220
Oppermann, A. (2022) What is deep learning and how does it work? Builtin. Retrieved from https://builtin.com/machine-learning/deep-learning
Ouyang, F., Xu, W., & Cukurova, M. (2023). An artificial intelligence-driven learning analytics method to examine the collaborative problem-solving process from the complex adaptive systems perspective. International Journal of Computer-Supported Collaborative Learning. https://doi.org/10.1007/s11412-023-09391-3
Pea, R. D. (1993). Practices of distributed intelligences and design for education. In G. Solomon (Ed.), Distributed cognitions: Psychological and educational considerations (pp. 47–87). Cambridge University Press.
Piaget, J. (1971). The theory of stages in cognitive development. In D. R. Green, M. P. Ford, & G. B. Flamer, Measurement and Piaget (pp. 1–11). McGraw-Hill.
Rannikmäe, M., Holbrook, J., & Soobard, R. (2020). Social Constructivism—Jerome Bruner. In B. Akpan & T. J. Kennedy (Eds.), Science education in theory and practice (pp. 259–275). Springer International Publishing. https://doi.org/10.1007/978-3-030-43620-9_18
Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65, 386–408.
Rourke, L., Anderson, T., Garrison, D. R., & Archer, A. (2002). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12, 8–22. https://telearn.hal.science/hal-00197319/document
Rumelhart, D., Hinton, G., & Williams, R. (1986). Learning internal representations by error propagation. In D. Rumelhart & J. McClelland (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations (pp. 318–362). Bradford Books/MIT Press.
Sammons, M. (2007). Collaborative interaction. In M.G. Moore (Ed.), Handbook of distance education (2nd ed.; pp. 311–321). Lawrence Erlbaum.
Sanchez, D. M. (2019). Building a call to action: Social action in networks of practice [Doctoral dissertation, University of New Mexico]. The University of New Mexico Digital Repository. https://digitalrepository.unm.edu/oils_etds/51/
Sanchez, D., Flor, N., & Gunawardena, C. (2020). Employing social learning analytic methods (SLAMs) to reimagine the social dynamic of online learning collaborations. In M. Brown, M. Nic Giolla Mhichil, E. Beirne, & E. Costello (Eds.), Proceedings of the 2019 ICDE World Conference on Online Learning, Volume 1, (pp. 817–832). Dublin City University, Dublin, Ireland. http://dx.doi.org/10.5281/zenodo.3804014
Schaaf, S. (2020). Social network centrality, social construction of knowledge, and nurse practitioner competency in asynchronous online discussions among adult gerontology acute care nurse practitioner students. [Doctoral Dissertation, University of New Mexico]. UNM Campus Repository. https://digitalrepository.unm.edu/oils_etds/59
Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85–117. https://doi.org/10.1016/j.neunet.2014.09.003
Silverio, S., Hall, J., & Sandall, J. (2020, January). Time and qualitative research: Principles, pitfalls, and perils. [Conference presentation]. Sixth Annual Qualitative Research Symposium—Tick Tock: Unpacking Temporal Aspects of Qualitative Inquiry, University of Bath, Bath, UK.
Stahl, G. (2006). Group cognition: Computer support for building collaborative knowledge. The MIT Press. https://doi.org/10.7551/mitpress/3372.001.0001
Tao, Y., Zhang, M., Su, Y., & Li, Y. (2022). Exploring college English language learners’ social knowledge construction and socio-emotional interactions during computer-supported collaborative writing activities. The Asia-Pacific Education Researcher, 31(5), 613–622. https://doi.org/10.1007/s40299-021-00612-7
Tirado, R., Hernando, Á., & Aguaded, J. I. (2015). The effect of centralization and cohesion on the social construction of knowledge in discussion forums. Interactive Learning Environments, 23(3), 293–316. https://doi.org/10.1080/10494820.2012.745437
Valtonen, T., López-Pernas, S., Saqr, M., Vartiainen, H., Sointu, E. T., & Tedre, M. (2022). The nature and building blocks of educational technology research. Computers in Human Behavior, 128, 107123. https://doi.org/10.1016/j.chb.2021.107123
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. , ... & Polosukhin, I. (2017). Attention is all you need [Conference presentation]. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, United States.
Vygotsky, L. S. (1978). Mind and society: The development of higher psychological processes. Harvard University Press.
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Gunawardena Charlotte N. , Yan Chen, Flor Nick, Sanchez Damien
This work is licensed under a Creative Commons Attribution 4.0 International License.
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
