Developing Learning Analytics Design Knowledge in the “Middle Space”: The Student Tuning Model and Align Design Framework for Learning Analytics Use
DOI:
https://doi.org/10.24059/olj.v20i2.783Keywords:
Learning analytics, pedagogical design, analytics use, self-directed learningAbstract
This paper addresses a relatively unexplored area in the field of learning analytics: how analytics are taken up and used as part of teaching and learning processes. Initial steps are taken towards developing design knowledge for this “middle space,” with a focus on students as analytics users. First, a core set of challenges for analytics use identified in the literature are compiled. Then, a process model is presented for conceptualizing students’ learning analytics use as part of a self-regulatory cycle of grounding, goal-setting, action and reflection–the Student Tuning Model. Finally, the Align Design Framework is presented with initial validation as a tool for pedagogical design that addresses the identified challenges and supports students’ use of analytics as part of the tuning process. Together, the framework’s four interconnected principles of Integration, Agency, Reference Frame and Dialogue / Audience provide a useful starting point for further inquiry into well-designed learning analytics implementations.References
Aguilar, S. J. (2014, March). Exploring and measuring students' sense-making practices around
representations of their academic information. Paper presented at Learning analytics and Knowledge Conference 2014, Indianapolis, IN, USA.
Andrusyszyn, M. A., & Davie, L. (1997). Facilitating reflection through interactive
journal writing in an online graduate course: A qualitative study. The Journal of
Distance Education, 12(1/2), 103-126.
Arnold, K. E., & Pistilli, M. D. (2012, April). Course signals at Purdue: using learning analytics
to increase student success. In Proceedings of the 2nd International Conference on Learning
Analytics and Knowledge (pp. 267-270). ACM.
Auerbach, C. F., & Silverstein, L. B. (2003). Qualitative data: An introduction to coding and
analysis. New York: New York University Press.
Avramides, K., Hunter, J., Oliver, M. & Luckin, R. (2014). A method for teacher inquiry in
cross-curricular projects: lessons from a case study. British Journal of Educational Technology, 46, 2, 249–264.
Baker, R. S., & Inventado, P. S. (2014). Educational data mining and learning analytics. In
Learning Analytics (pp. 61-75). Springer New York.
Boekaerts, M., Pintrich, P. & Zeidner, M. (2000). Handbook of self-regulation, 417-450. San
Diego, CA: Academic Press.
Booth, M. (2012, July/August). Learning analytics: The new black. EDUCAUSE Review, 47(4),
-53
boyd, d., & Crawford, K. (2011, September). Six Provocations for Big Data. Paper
presented at A Decade in Internet Time: Symposium on the Dynamics of the Internet and Society, Oxford Internet Institute, UK. Retrieved 12 June, 2012, from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1926431
Brooks, C., Greer, J., & Gutwin, C. (2014). The Data-Assisted Approach to Building
Intelligent Technology-Enhanced Learning Environments. In Learning Analytics (pp. 123-156). Springer New York.
Buckingham Shum, S, & Deakin Crick, R. (2012). Learning dispositions and
transferable competencies: pedagogy, modelling and learning analytics. In
Proceedings of the 2nd International Conference on Learning Analytics and
Knowledge (pp. 92-101). ACM.
Buckingham Shum, S. (2012). Our learning analytics are our pedagogy. Keynote
address given at Expanding Horizons 2012. Macquire University, Australia.
Buckingham Shum, S. & Ferguson, R. (2012). Social Learning Analytics. Educational
Technology & Society, 15(3), 3-26.
Buder, J. (2011). Group awareness tools for learning: Current and future directions. Computers
in Human Behavior, 27(3), 1114-1117.
Butler, D. L., & Winne, P. H. (1995). Feedback and Self-Regulated Learning: A
Theoretical Synthesis. Review of Educational Research, 65(3), 245 -281.
Campbell, J. P., DeBlois, P. B., & Oblinger, D. G. (2007). Academic analytics: A new
tool for a new era. EDUCAUSE review, 42(4), 40.
Campbell, J.P. & Oblinger, D.G. (2007) Academic Analytics, EDUCAUSE Quarterly. October.
Chatti, M. A., Dyckhoff, A. L., Schroeder, U., & Thüs, H. (2012). A reference model for
learning analytics. International Journal of Technology Enhanced Learning, 4(5), 318-331.
Clow, D. (2012). The learning analytics cycle: closing the loop effectively. doi:10.1145/2330601.2330636
Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, 18(6), 683–695. doi:10.1080/13562517.2013.827653Cuban, L. (1986). The classroom use of technology since 1920. New York: Teachers College Press, Columbia University.
Colthrope, K., Zimbardi, K., Ainscough, L., & Anderson, S. (2015). Know Thy Student!
Combining Learning Analytics and Critical Reflections to Increase Understanding of Students’ Selfâ€Regulated Learning in an Authentic Setting. Journal of Learning Analytics, 2(1), 134–155
Corrin, L., & de Barba, P. (2014). Exploring students’ interpretation of feedback delivered
through learning analytics dashboards. In B. Hegarty, J. McDonald, & S.-K. Loke (Eds.), Rhetoric and Reality: Critical perspectives on educational technology. Proceedings ascilite Dunedin 2014 (pp. 629-633)
Cuban, L., (2001). Oversold and underused: Computers in the classroom. Cambridge, MA: Harvard University Press.
Cuban, L. 1986. The classroom use of technology since 1920. Teachers College Press, New York, NY.
Cutumisu, M., Blair, K. P., Chin, D. P., & Schwartz, D. L. (2015). Posterlet: A Gameâ€Based Assessment of Children's Choices to Seek Feedback and to Revise. Journal of Learning Analytics, 2(1), 49–71.
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for
educational inquiry. Educational Researcher, 5-8.
Ferguson, R. (2012). Learning analytics: drivers, developments and challenges.International Journal of Technology Enhanced Learning, 4(5), 304-317.
Ferguson, R., & Buckingham Shum, S. (2011). Learning analytics to identify exploratory dialogue within synchronous text chat. In Proceedings of the 1st International Conference on Learning Analytics & Knowledge, (Banff, Canada)
Ferguson, R., Buckingham Shum, S., & Deakin Crick, R. (2011). EnquiryBlogger: using
widgets to support awareness and reflection in a PLE Setting.
Ghislandi, P. M., & Raffaghelli, J. E. (2015). Forwardâ€oriented designing for learning as a means
to achieve educational quality. British Journal of Educational Technology, 46(2), 280-299.
Gibson, J. J. (1977). The theory of affordances. In R. E. Shaw & J. Bransford (Eds.), Perceiving,
acting and knowing: Towards an ecological psychology. (pp.67-82) Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Govaerts, S., Verbert, K., Duval, E., & Pardo, A. (2012). The student activity meter for
awareness and self-reflection. In CHI'12 Extended Abstracts on Human
Factors in Computing Systems (pp. 869-884). ACM. Hlupic, V., Pouloudi, A., and Rzevski, G. (2002). Towards an integrated approach to knowledge management: ‘Hard’, ‘soft’ and ‘abstract’ issues. Knowledge and Process Management, 9(2), 90-102.
Govaerts, S., Verbert, K., Klerkx, J., & Duval, E. (2010). Visualizing Activities for Self-
reflection and Awareness. In Proceedings of the 9th international conference on Web-based Learning. (pp. 91–100). Springer, 2010.
Greller, W., & Drachsler, H. (2012). Translating learning into numbers: A generic framework for
learning analytics. Journal of Educational Technology & Society, 15(3), 42-57.
Haya, P. A., Daems, O., Malzahn, N., Castellanos, J., & Hoppe, H. U. (2015). Analysing content
and patterns of interaction for improving the learning design of networked learning environments. British Journal of Educational Technology, 46(2), 300-316.
Johnson, L., Becker, S., Estrada, V., & Freeman, A. (2014). Horizon Report: 2014 Higher
Education. Austin, TX: The New Media Consortium.
Janssen, J., & Bodemer, D. (2013). Coordinated computer-supported collaborative learning:
Awareness and awareness tools. Educational Psychologist, 48(1), 40-55.
Kolb, D.A. (1984). Experiential education: Experience as the source of learning and
development. Englewood cliffs, NJ: Prentice Hall.
Kruse, A. N. N. A., & Pongsajapan, R. (2012). Student-centered learning analytics.
CNDLS Thought Papers, 1-9.
Kuhn, T. S. (1962). The Structure of Scientific Revolutions. Chicago : University of Chicago Press.
Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning
Learning Analytics with Learning Design. American Behavioral Scientist, 57, 1439–1459. doi:10.1177/0002764213479367
Locke, E. A., & Latham, G. P. (2006). New directions in goal-setting theory. Current Directions
in Psychological Science, 15(5), 265-268.
M'hammed Abdous, He, W., & Yen, C. J. (2012). Using data mining for predicting relationships between online question theme and final grade. Journal of Educational Technology & Society, 15(3), 77-88.Macfadyen, L. P., & Dawson, S. (2010). Mining LMS data to develop an “early warning system†for educators: A proof of concept. Computers & Education,54(2), 588-599.
Macfadyen, L., & Dawson, S. (2012). Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan. Educational Technology & Society, 15(3), 149-163.
Manouselis, N., Drachsler, H., Verbert, K., & Duval, E. (2012). Recommender systems for learning. Springer Science & Business Media.
Mathan, S., & Koedinger, K. R. (2003). Recasting the feedback debate: Benefits of tutoring error detection and correction skills. In U. Hoppe, F. Verdejo & J. Kay (Eds.), Artificial intelligence in education: Shaping the future of learning through intelligent technologies (pp. 13-20). Amsterdam: IOS Press.
McAlpine, L., & Weston, C. (2000). Reflection: Issues related to improving professors' teaching and students' learning. Instructional Science, 28(5), 363-385.McNely, B. J., Gestwicki, P., Hill, J. H., Parli-Horne, P., & Johnson, E. (2012). Learning analytics for collaborative writing: a prototype and case study. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (pp. 222-225). ACM.
McKenney, S., & Mor, Y. (2015). Supporting teachers in dataâ€informed educational design. British journal of educational technology, 46(2), 265-279.Messick, S. (1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18(2), 5-11.
McNely, B. J., Gestwicki, P., Hill, J. H., Parli-Horne, P., & Johnson, E. 2012. Learning analytics for collaborative writing: a prototype and case study. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (Vancouver, Canada, 2012). ACM, New York, NY, 222-225.
Melero, J., Hernández-Leo, D., Sun, J., Santos, P., & Blat, J. (2015). How was the activity? A visualization support for a case of locationâ€based learning design. British Journal of Educational Technology, 46(2), 317-329.
Miller, M. (2015). Leveraging CSCL technology for supporting and researching shared task perceptions in socially shared regulation of learning. (Unpublished doctoral dissertation) Univesity of Vicotria, Canada.
Norman, D. A. (2013). Design of Everyday Things: Revised and Expanded. New York: Basic Books. London: MIT Press (UK edition)
Nussbaumer, A., Hillemann, E. C., Gütl, C., & Albert, D. (2015). A Competence-based Service for Supporting Self-Regulated Learning in Virtual Environments. Journal of Learning Analytics, 2(1), 101-133.
Oblinger, D.G. (2012). Let’s talk analytics. EDUCAUSE Review, July/August, 10-13
Postman, N. (1985). Amusing ourselves to death. New York: Penguin.
Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of educational Psychology, 95(4), 667.
RodrÃguezâ€Triana, M. J., MartÃnezâ€Monés, A., Asensioâ€Pérez, J. I., & Dimitriadis, Y. (2015). Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations. British Journal of Educational Technology, 46(2), 330-343.
Roll, I., & Winne, P. H. (2015). Understanding, evaluating, and supporting selfâ€regulated learning using learning analytics. Journal of Learning Analytics, 2(1), 7–12.
Romero, C., & Ventura, S. (2013). Data mining in education. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 3(1), 12-27.
Segedy, J. R., Kinnebrew, J. S., Biswas, G. (2015). Using Coherence Analysis to Characterize Selfâ€Regulated Learning Behaviours in Openâ€Ended Learning Environments. Journal of Learning Analytics, 2(1), 13–48
Santos, J. L., Govaerts, S., Verbert, K., & Duval, E. (2012, April). Goal-oriented visualizations of activity tracking: a case study with engineering students. In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 143-152). ACM.
Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books.
Schraw, G. (1998). Promoting general metacognitive awareness. Instructional science, 26(1-2), 113-125.
Siemens, G. (2013). Learning Analytics: The Emergence of a Discipline. American Behavioral Scientist, 57, 1380–1400. doi:10.1177/0002764213498851
Slade, S., & Prinsloo, P. (2013). Learning Analytics: Ethical Issues and Dilemmas. American Behavioral Scientist, 57(March), 1–20. doi:10.1177/0002764213479366
Stolterman, E., & Wiberg, M. (2010). Concept-driven interaction design research. Human–
Computer Interaction, 25(2), 95-118.
Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data.
American Journal of Evaluation, 27(2), 237–246.
van Harmelen M. & Workman, D. (2012). Analytics for learning and teaching. CETIS Analytics
Series 1(3).
van Leeuwen, A. (2015). Learning analytics to support teachers during synchronous CSCL:
balancing between overview and overload. Journal of Learning Analytics, submitted for publication.
Vatrapu, R., Teplovs, C., Fujita, N., & Bull, S. (2011, February). Towards visual analytics for teachers' dynamic diagnostic pedagogical decision-making. In Proceedings of the 1st International Conference on Learning Analytics and Knowledge (pp. 93-98). ACM.
Veenman, M. (2013). Metacognition and learning: conceptual and methodological considerations revisited. What have we learned during the last decade? Keynote given at EARLI 2013.
Verbert, K., Duval, E., Klerkx, J., Govaerts, S. & Santos, J.L. (2013). Learning analytics dashboard applications. American Behavioral Scientist, 57(10), 1500-1509.
Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van Assche, F., Parra, G., & Klerkx, J. (2014). Learning dashboards: An overview and future research opportunities. Personal and Ubiquitous Computing, 18, 1499–1514. doi:10.1007/s00779-013-0751-2
Visser, L., Plomp, T., Amirault, R. J., & Kuiper, W. (2002). Motivating students at a distance: The case of an international audience. Educational Technology Research and Development, 50(2), 94-110.
Winne, P. H. (1995). Self-regulation is ubiquitous but its forms vary with knowledge. Educational Psychologist, 30(4), 223-228.
Winne, P. H. (2011). A cognitive and metacognitive analysis of self-regulated learning. Handbook of self-regulation of learning and performance, 15-32.
Winne,P. H. (2000). Information processing models of self-regulated learning. In B. J. Zimmerman & D. H. Schunk (Eds.), Self-regulated learning and academic achievement: Theory, research, and practice. New York: Longman.
Winne, P. H., & Baker, R. S. J. d. (2013). The Potentials of Educational Data Mining for Researching Metacognition, Motivation and Self-Regulated Learning. JEDM - Journal of Educational Data Mining, 5(1), 1–8. Retrieved from http://www.educationaldatamining.org/JEDM/index.php/JEDM/article/view/28
Winne,P. H.,& Hadwin,A. F. (1998). Studying as self-regulated learning. In D. Hacker,J. Dunlosky,& A. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277– 304). Mahwah,NJ: Erlbaum
Winne, P.H., & Perry, N.E. (2000). Measuring self-regulated learning. In M. Boekaerts, P.R. Pintrich & Zeidner (eds), Handbook of Self-regulation. San Diego, CA: Academic Press.
Winne,P. H.,& Hadwin,A. F. (1998). Studying as self-regulated learning. In D. Hacker,J. Dunlosky,& A. Graesser (Eds.), Metacognition in educational theory and practice (pp. 277– 304). Mahwah,NJ: Erlbaum.
Authors (2013)
Authors (2014)
Authors (2014)
Zeini, S., Göhnert T., Krempel L., & Hoppe H. U. (2012). The impact of measurement
time on subgroup detection in online communities. In Proceedings of the
IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, (pp. 389-394), Instanbul, Turkey: IEEE.
Zimmerman, B. J., & Schunk, D. H. (Eds.) (2001). Self-regulated learning and academic
achievement: Theoretical perspectives (2nd ed.). Mahwah, NJ: Lawrence
Erlbaum.
Zimmerman, B. J., & Schunk, D. H. (Eds.). (2013). Self-regulated learning and
academic achievement: Theoretical perspectives. Routledge.
Zimmerman, B. J., Bandura, A., & Martinez-Pons, M. (1992). Self-motivation for
academic attainment: The role of self-efficacy beliefs and personal goal setting. American Educational Research Journal, 29(3), 663-676.
Downloads
Published
Issue
Section
License
As a condition of publication, the author agrees to apply the Creative Commons – Attribution International 4.0 (CC-BY) License to OLJ articles. See: https://creativecommons.org/licenses/by/4.0/.
This licence allows anyone to reproduce OLJ articles at no cost and without further permission as long as they attribute the author and the journal. This permission includes printing, sharing and other forms of distribution.
Author(s) hold copyright in their work, and retain publishing rights without restrictions
