Analytics Research and Case Studies in Online Learning

We are pleased to present this Online Learning Journal special issue devoted to learning analytics for online teaching and learning. The nine papers contained here provide a range of information including reviewing the literature, examining frameworks in development, presenting a large scale analysis on the effectiveness of learning modalities from the PAR Framework, examining an international study of learning, and providing real-world learning analytics case studies on transfer, facilitation, and medical education. Each of these papers provides new and informative research that we hope can help readers make decisions about applying analytics within the context of their own online teaching and learning environments.


Preview of Articles
In the first article, "Learning Analytics Methods, Benefits, and Challenges in Higher Education: A Systematic Literature Review," Avella, Kebritchi, Nunn and Kanai (2016) provide a comprehensive literature review.As the authors note, there is a need for a review of the recent research within the learning analytics field.Given the importance of this topic, it is important that universities and colleges investigate applying learning analytics, and that practitioners understand methodologies as well as benefits and challenges within learning analytics.The opportunities to leverage analytics are indeed powerful, but it is important to understand the multiple methods in the field such as visualization, network analysis, semantic analysis and data mining techniques like predictive, clustering, and relationship mining.Delineated benefits of utilizing learning analytics include improving course offerings, student outcomes, curriculum development and instructor effectiveness.The authors include multiple challenges to the uptake of learning analytics including data quality, analytics used without a connection to learning sciences, and ethical and privacy issues surrounding potentially sensitive data.The systemic literature review is an excellent jumping off point for the special issue.
The next two papers from Australian authors explore frameworks which support the use of analytics within learning environments.Readers will expand their knowledge and perspectives given the thoughtful approach of these authors.The first framework focuses on applying analytics with a primary focus on student retention.The West, Heath and Huijser (2016) article entitled, "Let's Talk Learning Analytics: A Framework for Implementation in Relation to Student Retention," presents a dialogical tool for the advancement of learning analytics implementation for student retention.The authors were supported by the Australian Government's Office for Learning and Teaching and include perspectives from Australian higher education institutions.The project took a mixed-method approach including a survey with both the institutional level and teaching and other academics staff level with an interest in student retention.The paper includes a series of interviews.The framework was then shared with other colleagues to continue the refinement process.The authors hope their tool will be used to inform discussion about how learning analytics might be implemented when an institution is focused on student retention.
The second framework integrates two widely disseminated best practices approaches for designing learning and aims to create an instrument that would inform practices for online and blended learning courses.Jabar and Albion (2016) present the findings in their paper, "Assessing the Reliability of Merging Chickering & Gamson's Seven Principles for Good Practice with Merrill's Different Levels of Instructional Strategy (DLISt7)."The study was conducted across five faculties at a regional Australian multi-campus university.The authors used an exploratory and confirmatory factor analysis to validate Merrill's Different Levels of Instructional Strategy, leveraging the data that had been collected.Authors James, Swan and Daston (2016) provide a review of a large dataset compiled within the Predictive Analytics Reporting (PAR) Framework by comparing students taking only onground courses, students taking only online courses, and students taking a mixture of both at five primarily onground community colleges, five primarily onground four-year universities, and four primarily online institutions.This exciting work entitled, "Retention, Progression and the Taking of Online Courses," provides further proof that online courses can provide both flexibility and access while improving student completion.
The results suggest that taking online courses is not necessarily harmful to students' chances of being retained.While the PAR Framework dataset represents a microcosm of institutions across American universities, it does include a more representative sample of institutions serving nontraditional students.It is clear from other work including IPEDS recent reports that these students are taking more accessible course modalities like online and blended courses.Their research also reveals that essentially no difference in retention between delivery mode was found for students enrolled in primarily onground four-year universities participating in the PAR Framework.At participating primarily online institutions, students blending their courses had slightly better odds of being retained than students taking exclusively onground or exclusively online courses.This report furthers other seminal research that reviews retention in online and blended learning.
The next paper explores the less structured online learning environment of an interactive social network across multiple countries to examine whether the application of learning analytics could be informative in such a network."The Assessment of Learning in Digital Interactive Social Networks: A Learning Analytics Approach," by Wilson, Scalise and Gochyvev (2016) examined the learning data within the Assessment and Teaching of 21st Century Skills (ATC21S) project on the "ICT Literacy -Learning in digital networks."This project, sponsored by Cisco, Intel and Microsoft, helps educators around the world enable students with the skills necessary for success in their future career and college goals.The instrument developed models of learning progression.Results indicate that the new constructs and the new item forms and analytics that were employed, can be used in a large-scale digital environment.Nadasen and List (2016), in their piece entitled, "Using Community College prior Academic Performance to Predict Re-enrollment at a Four-Year Online University," provide readers with a case study from a large public online university where analytics were used to predict student success of transfer students.The research study examines student learner characteristics, course-taking behaviors from prior community colleges attended, and first-term GPA at a four-year institution to predict the likelihood of re-enrollment for 8,200 students.The logistic regression models showed that gender, age, and first term GPA at the four-year institution were significant predictors of re-enrollment.Researchers and practitioners will have a greater understanding of how community college factors influence the progression and success for transfer students at four-year institutions.
The authors Phirangee, Demmans Epp and Hewitt (2016) compare instructor facilitation to peer facilitation within online courses.The paper titled, "Exploring the Relationships Between Facilitation Methods, Students' Sense of Community, and Their Online Behaviors," compared the two facilitation methods and discovered that students participated more in instructor-facilitated online courses where they wrote, edited and reread posts more, and created more connections to others' writing, than students in peer-facilitated online courses.They identified student activity patterns and described differences in how those patterns manifest themselves.Their findings also show that instructor-facilitated online courses had a stronger sense of community than peer-facilitated online courses.
Finally, the last two articles explore analytics as a lever to improve student learning.The first article by Wise, Vytasek, Hausknecht, and Zhao (2016) focuses on students as users of analytics.The paper, "Developing Learning Analytics Design Knowledge in the "Middle Space" The Student Tuning Model and Align Design Framework for Learning Analytics Use," pulls from the literature challenges to using student-facing learning analytics.It also explores a process model for understanding students' use of learning analytics as part of self-regulation theory.As captured in the title, the authors' exploratory work refers that this approach will need to be regularly tuned for students to leverage analytics.The resulting framework pulls together constructs and informs how pedagogical design can be improved to leverage analytics.The authors note that this initial inquiry will require more research.
The final article in the issue shows how learning analytics can improve student learning in a medical environment.Using large amounts of data from a single virtual patient case, Poitras, Naismith, Doleck and Lajoie (2016) used subgroup discovery methods to identify student learning patterns.The paper, "Using Learning Analytics to Identify Medical Student Misconceptions in an Online Virtual Patient Environment," describes how the researchers examined the data to find patterns of learning.The analysis was then used to create an adaptive algorithm where feedback was received by students leading to an improvement in learning outcomes.Adaptive systems are able to leverage analytics in a way that can be very informative to student learning.
We hope that the collection of articles in this special issue strengthens the research base for leveraging analytics within online learning.Some of the articles inform the use of analytics in a variety of learning modalities.However, online learning modalities generally provide researchers with the ease of collecting much more data to explore, analyze and test hypotheses.The field will need significantly more research as well as researchers exploring both small and large datasets to provide practitioners with informative and effective practices.Authors from these papers help to extend the existing groundwork and indicate the continued need for research within analytics for understanding the impact of online learning by addressing issues such as building frameworks to inform design and improve retention, building a research base for student use of analytics, examining how to build large enough datasets that inform practices within the transfer ecosystems of higher education, and leveraging analytics to design adaptive systems to improve student outcomes.