An Instructor Learning Analytics Implementation Model

With the widespread use of learning analytics (LA) tools, there is a need to explore how these technologies can be used to enhance teaching and learning. Little research has been conducted on what human processes are necessary to facilitate meaningful adoption of LA. The research problem is that there is a lack of evidence-based guidance on how instructors can effectively implement LA to support students. The goal of the study was to develop and validate a model to guide instructors in the implementation of LA tools. Using design and development research methods, an implementation model was constructed and validated internally. Themes emerged falling into the categories of adoption and caution with six themes falling under adoption including:


Abstract:
With the widespread use of learning analytics (LA) tools, there is a need to explore how these technologies can be used to enhance teaching and learning.Little research has been conducted on what human processes are necessary to facilitate meaningful adoption of LA.The research problem is that there is a lack of evidence-based guidance on how instructors can effectively implement LA to support students.The goal of the study was to develop and validate a model to guide instructors in the implementation of LA tools.Using design and development research methods, an implementation model was constructed and validated internally.Themes emerged falling into the categories of adoption and caution with six themes falling under adoption including: • LA as evidence, • reaching out, • frequency, • early identification/intervention, • self-reflection, and • align LA with pedagogical intent.Three themes emerged falling under the category of caution including: • skepticism, • fear of overdependence, and • question of usefulness.The model should enhance instructors' use of LA by enabling them to better take advantage of available technologies to support teaching and learning in online and blended learning environments.Researchers can further validate the model by studying its usability (i.e., usefulness, effectiveness, efficiency, and learnability), as well as, how instructors' use of this model to implement LA in their courses affects retention, persistence, and performance.

Introduction Background
Learning analytics (LA) is the collection, analysis, and reporting of available data to improve the teaching and learning process and environment (Siemens & Long, 2011).There are two main categories of research in the field of LA.The first is on how to capture, process, and present data to educational stakeholders in useful ways.The second, and less common, focus of research is on how to take up and use analytics in practice to inform choices or prompt action (Wise, Vytasek, Hausknecht, & Zhao, 2016).More simply, the majority of research has focused on how to create useful information from large quantities of collected data (Dawson, Gasevic, Siemens, & Joksimovic, 2014).Less research has been conducted on how to actually put this information to use to achieve desired purposes in the educational environment (Ferguson et al., 2014;Lockyer, Heathcote, & Dawson, 2013;West, Heath, & Huijser, 2016;Wise, 2014;Wise et al., 2016).LA holds potential application for a range of stakeholders in higher education including instructors, researchers, curriculum developers, learning environment designers, and university policy makers.LA is utilized at many levels within academic institutions, but a common application is at the course level (Dziuban, Moskal, Cavanagh, & Watts, 2012).Data within the learning management system (LMS) regarding student activity can be tracked and analyzed to monitor student progress, predict student success or failure, or inform instructional design.LA at the course level is an important area of research that promises to improve learning outcomes in online and blended courses by providing rich information regarding participation and performance to instructors and students alike.
Much of the literature in the second category of LA research uses the term "intervention" to describe the act of taking up and using analytics in practice (Lockyer et al., 2013;Wise, 2014;Zacharis, 2015).Wise et al. (2016) pointed out that this term can be useful, but can also include the undesired connotation that LA use is an interruption in the regular teaching and learning process.Instead, they chose to use the term "LA implementation" to describe the use of LA as an ongoing part of the regular monitoring and responsive adjustment to teaching and learning practices.This study also uses the term "LA implementation" to describe the process of taking up and using analytics in practice.

Problem Statement
While LA tools may show that students who regularly log into an LMS perform better than their less active peers, this information alone changes nothing and does not mean the instructor will provide a suitable response (Roll & Winne, 2015).Furthermore, simply telling the student to log into the LMS more often will not be helpful (Dawson et al., 2014).While analytics tools may provide insight, they do not help instructors to provide a systematic and integrated response to such situations that will result in better outcomes for the at-risk student.As Wise (2014) stated, "without a plan for shifting patterns of teaching and learning activity, new technologies often remain ancillary to the teaching and learning process, either used tangentially to marginally enhance existing practices or often simply collecting dust on the virtual shelf" (p.203).Little research has been done on what human processes are necessary to facilitate meaningful implementation of LA.
There is a general lack of research-based guidance on how various stakeholders (i.e., learners, instructors, and administrators) can effectively use LA tools, but researchers have begun to address this in recent years.West et al. (2016) presented a framework for institutional implementation of LA to support student retention efforts.Wise et al. (2016) addressed the problem of how students can take up and use LA in practice, but many LA tools are designed for instructor use and students cannot access the information they generate.Mor, Ferguson, and Wasson (2015) focused on how instructors can use LA to inform their reflective practice and learning design, but very few studies have focused on how instructors can use analytics in practice to support the student learning process.There is a need for a model to support instructor-specific use of LA to encourage its systematic use as an integrated part of the teaching process.The research problem is that there is a lack of evidence-based guidance on how instructors can effectively implement LA in their courses.
This study focused on the use of LA at Southwestern Oklahoma State University (SWOSU).SWOSU is a regional university in western Oklahoma with approximately 5,000 undergraduate, graduate, and professional students enrolled and approximately 225 faculty members employed.SWOSU currently provides faculty with two LA tool options.All faculty have access to Canvas Analytics as part of the Canvas LMS.SWOSU is also piloting AspirEdu's Dropout Detective in two of its fully online programs including RN to BSN and Health Information Management (HIM).Both of these tools are designed for instructor use.This paper will present a brief review of literature concerning LA tools, models, and implementation.The methodology of the study will then be presented.Last, the resulting LA implementation model will be presented along with conclusions and recommendations for research and practice.

Literature Review
A review of the literature guided the identification of what LA tools and models are currently available to instructors, how they are being used, and the benefits and limitations of such tools and models.This review informed the design and development of a preliminary model to guide instructor use of LA.The following review of literature includes a brief overview of the current state of the body of knowledge in the LA field regarding data capture, processing, and display as well as LA implementation.

Learning Analytics Tools
Performance and tracking.The majority of research in the LA field has been on the development and validation of LA tools to support student performance tracking.Spivey and McMillan (2013) as well as Mo and Zhao (2012) presented research studies focused on using Blackboard LMS to track student data.Spivey and McMillan (2013) investigated the relationship between student effort and performance by utilizing data already being tracked in Blackboard.The researchers found that more frequent access and a more evenly spaced study schedule (as opposed to "cramming") had a positive effect on student performance.Mo and Zhao (2012) had very similar findings.Both studies focused on using the tools already built into the LMS to track student data to monitor students and analyze effort and performance.Similarly, You (2015) found a link between academic procrastination and course achievement when examining LMS data.These studies are examples of using the tools at hand to begin implementing the principles of LA in the online classroom.Mazza and Dimitrova (2007) developed and analyzed a student monitoring tool for supporting instructors in online courses.The researchers surveyed users regarding the effectiveness, efficiency, and usefulness of their tool and found that the use of graphical representations of data was important to the user.Similarly, Ruipérez-Valiente, Muñoz-Merino, Leony, and Delgado Kloos (2015) presented a study of another LA tool that visualized data for the user.Ali, Hatala, Gašević, and Jovanović (2012) presented two evaluations of their tool, LOCO-Analyst, which also focuses on visualizing LMS data for instructors, and, last, Macfadyen and Dawson (2010) discussed the development and implementation of another dashboard-like tool that also visualizes LMS data and found that meaningful information can be extracted from LMS data and tools can be developed which visualize student progress and the likelihood of their success.All four of these studies concluded that the visualization aspect is important so instructors are able to readily discern outliers and points of concern and react to such circumstances quickly.

Student retention.
Another common theme found in the literature on LA tools is the development of tools aimed at increasing student retention.Retention efforts begin in the classroom, so this topic has many stakeholders and touches every level of higher education.Agnihotri and Ott (2014) presented the development of an LA tool aimed at student retention.They viewed this issue from an administrative level and sought to provide a tool for retention counselors within the university.The purpose of this tool was to provide retention risk ratings to counseling staff for each new freshman before the start of the fall semester.Agnihotri and Ott (2014) concluded that such tools are capable of increasing student retention, but that the development process must utilize a broad perspective of the entire retention process.
Similarly, Harrison et al. (2015) presented an early alert system designed to identify students at risk of discontinuing enrollment.They included demographic, institution, and learning environment variables in their model resulting in a tool that could accurately predict those at risk of discontinuing.Jayaprakash and Lauría ( 2014) presented yet another early alert system designed to identify students at academic risk for the purpose of increasing student retention rates.Knight and Shum (2014) took the discussion of tool development a step further by introducing the idea that the design LA tools should be informed by epistemology, assessment, and pedagogy.They made the point that it is not the tool itself, but the way in which it is wielded, which determines its value.This idea leads to the discussion of LA models to guide the implementation and use of LA tools.

Learning Analytics Models
More recent research has gone beyond tool development and validation and begun to take a broader view of the issue of LA model development and validation.Martinez-Maldonado et al. (2015) as well as Scheffel, Drachsler, Stoyanov, and Specht (2014) presented frameworks to support the development and evaluation of LA tools, respectively.Ali, Asadi, Gašević, Jovanović, and Hatala (2013) sought to identify what specific factors would lead instructors to use or not use LA tools.Macfadyen and Dawson (2012) pointed out that LA should be consulted and integrated into the institutional strategic planning process.Ferguson et al. (2014) presented a framework to support the implementation of LA at the institutional level.Although no specific framework or model was presented, Dringus (2012) described a number of principles for the adoption of LA tools while expressing an attitude of caution when considering LA as being potentially "harmful."Last, West et al. (2016) presented a framework for LA implementation in relation to student retention.This framework was meant to stimulate a discussion about the institutional implementation of LA.
Perhaps most relevant to this study are the frameworks presented by Wise (2014) and Wise et al. (2016).Wise (2014) presented a discussion of designing interventions based on the output of LA tools pointing out that this part of the process is often ignored and is a relatively unexplored area of research.There are three specific aspects of the application of LA: what traces of learning should be captured, how to present these traces to learners, and how to frame the inclusion of analytics as part of the course activity to guide their use in productive decision-making by learners and teachers (Wise, Zhao, & Hausknecht, 2014).These interventions have to do with the latter two aspects.Wise (2014) pointed out that as LA tools are becoming more prevalent, intervention design becomes critical to their effective implementation and offered the following important research questions: when in the teaching and learning process should analytics be consulted; who should be accessing analytics; why are they being consulted; and most importantly, how the use of the analytics articulates with the rest of the teaching and learning practices taking place.Wise (2014) presented a framework which began to answer some of these questions, but a revised and extended version of this framework was presented by Wise et al. (2016).They presented a model for student use of LA as a part of a self-regulatory cycle of grounding, goalsetting, action, and reflection, the Student Tuning Model.The Student Tuning Model suggests that students engage in a continual cycle of planning, monitoring, and adjusting their learning practices as they are informed by analytics.The element of Grounding has to do with the relationship between the information the analytics provide and the specific educational context in which they are being provided.Students must understand the purpose of the learning activity, what represents meaningful engagement in the activity, and how the LA provided will reflect this to the student.Goal-Setting has to do with the student planning specific objectives and actions for reaching them in relation to the larger context established through Grounding.Action is when students engage in behaviors to realize their goals.Reflection occurs when students use analytics to reflect on the actions they took in comparison to the goals they set.
The Student Tuning Model was meant to outline how students might productively engage with analytics.Wise et al. (2016) also provided a framework for pedagogical design to support student use of analytics.The Align Design Framework includes the four principles of Integration, Agency, Reference Frame, and Dialogue/Audience.The first principle of Integration states that the instructor should position student analytics use as an integral part of the learning process.The second principle of the framework is Agency which has to do with students taking ownership of their learning process.The principle of Reference Frame states that instructors should provide a comparison point to students.The final principle of this framework is Dialogue/Audience.This principle states that the instructor should create an environment where interpretation of analytics is discussed between the instructor and students so that students don't simply feel that they are being watched.
While this framework is a good starting point, the research problem remains that there is a lack of evidence-based guidance on how instructors can effectively implement LA tools which are designed to present information only to the instructor and not the student.Wise's (2016) framework does little to help in this situation.Lockyer et al. (2013) addressed this issue in part by presenting the idea that a conceptual framework should be established for typical LA patterns expected from particular learning designs in order to better help teachers interpret the information that analytics provides.Lockyer's model has a narrow focus on how learning design can inform the use of LA and is difficult to generalize to a variety of learning situations.

Method
A qualitative design and development research approach (Richey & Klein, 2007) was used to address the research problem that there is a lack of evidence-based guidance on how instructors can effectively implement learning analytics (LA).Specifically, model construction and validation methods were used to construct an instructor LA implementation model.The study took place within SWOSU and focused on the use of the LA tools available there.
First, the review of literature served as the basis for answering the first research questions: (a) what LA tools and models are currently available to instructors, (b) how are they using these tools and models to support teaching and learning, and (c) what are the benefits and limitations of such LA tools and models?Next, a needs assessment was conducted to address the second research question: what needs to be considered to design an effective model to guide instructors in using LA tools and implementing interventions?A survey and a follow-up focus group were used to identify needs of stakeholders including instructors, online learning administrators, and online learning committee members.Then, a preliminary model to guide instructors in the use of LA tools was designed based on the review of literature and the needs assessment which addressed the third research question: how can stakeholder needs inform the design of such a model?The next phase included an expert review of the model using Delphi panel technique.This approach addressed the fourth research question: how do instructors perceive the effectiveness and efficiency of the proposed LA model?Last, modifications were made to the model to implement suggestions from the Delphi panel, which addressed the fifth research question: what modifications are needed to improve the proposed LA model?This three-phase process (i.e., needs assessment, model construction, and model validation) resulted in a model, which is useful to instructors wanting to effectively implement LA tools in their courses.

Overview
This study was designed to identify stakeholder needs regarding the implementation of LA at the course level in order to develop and validate a model to support instructor use of LA.The researcher began by conducting a needs assessment including a survey and two focus group sessions.The survey was used to collect quantitative and qualitative data from instructors regarding the use of LA in their courses.The survey included questions in the categories of demographics, prior use and perceptions, efficacy, model construction, and focus group participation.The focus group sessions were meant to elicit more detailed information from participants.The first session had seven participants in attendance, and the second had ten.Next, data from the survey and focus group sessions were analyzed in the context of the research questions and a model was developed based on the review of literature and analysis of the data.Last, the model was reviewed by a Delphi panel until consensus was reached.The model was approved by the panel, which serves as internal validation.

Model Construction and Validation
The survey was sent to approximately 350 full-time and adjunct faculty from both SWOSU campuses.There were 61 (i.e., 17.42%) responses to the survey.The low response rate is explained by the fact that the survey was sent to both SWOSU campuses as well as adjunct instructors.Many of the recipients were unfamiliar with the topic of LA and uninterested in the study.The results of the survey items in the categories of prior use and perceptions and efficacy confirmed the desire to implement LA in the classroom and the need for instruction on how to effectively do so.The next section of the survey asked the open-ended question: "What additional information or training would enable you to use learning analytic tools (e.g., Canvas Analytics or Dropout Detective) more effectively?"Of the 61 respondents, 34 provided a response to this question.These qualitative data was analyzed along with the focus group data.The next section of the survey asked participants if they would be willing to participate in a focus group.Of the 49 participants who answered this question, 31 responded that they would be willing to participate (63.3%).Those who answered yes provided their contact information (name, email address, phone number) in the final section of the survey.
The next phase of the study were the focus groups.The researcher conducted two focus group sessions because of the large number of willing participants.There were initially 31 survey respondents who stated they would be willing to participate.Once scheduled, the focus group sessions were attended by a total of 17 participants.A semi-structured approach was used, with the researcher using the focus group protocol to loosely guide the discussion and asking follow up questions when necessary.The researcher took brief notes and had a teaching assistant take an additional set of notes.Each focus group session was scheduled to run about one hour.Both sessions ran about 15 minutes over the allotted hour due to rich discussion that occurred.The notes from the focus group sessions, as well as the text from the qualitative survey item, were loaded into qualitative research software which was used to analyze, identify themes, and code the data according to the themes.
The resulting model, which is included in the next section, was validated internally using a Delphi panel method.The researcher recruited three participants from the focus group sessions to participate in the Delphi panel.These three participants were considered subject matter experts in LA at SWOSU.Their participation in the focus group sessions also enabled them to assess whether the model addressed the needs and opinions voiced during the focus group session.The model was sent to the panel by email, and they were asked to complete a questionnaire to assess whether the model adhered to what was discussed during the focus group as well as the usability of the model according to the Rubin and Chisnell's (2008) attributes.For the most part, the Delphi panel found the model to be complete, useful, efficient, effective, and learnable.Three suggestions for improvement were made, but after discussion and clarification of the model's purpose, it was determined that the suggestions were beyond the scope of the model's intent and the model was approved.This served as internal validation of the following model.

Resulting Model
Based on a review of the current literature regarding LA and a needs assessment (including a survey and two focus group sessions) regarding LA implementation at SWOSU, the following instructor LA implementation model was developed (Figure 1).The first focus group session was very positive and implementation strategies were discussed and refined.The second group expressed a very cautious attitude toward the implementation of LA.It became clear during the focus group sessions that the themes identified fell into two broad categories: adoption and caution.These contrasting attitudes reflect the various tones of literature concerning LA implementation (Dringus, 2012;Wise, 2014).Although themes fell into these two seemingly conflicting groups, the model is meant to demonstrate that both adoption and caution are part of the overall implementation process.Themes are organized according to these two categories, and practical and conceptual guidelines are presented based on these themes.

Figure 1. Instructor Learning Analytics Implementation Model
Adoption.Many instructors participating in the focus groups already used LA extensively in their courses.Others were eager to learn more and begin the implementation process.Based on the needs assessment and review of literature the following themes emerged: LA as evidence, reaching out, frequency, early identification/intervention, self-reflection, and aligning LA with pedagogical intent.These themes fell under the category of Adoption.
LA as Evidence.It seems that many instructors appreciate that LA provides indisputable facts and information.This type of information can be used in a variety of ways, but it is a common theme that instructors appreciate the ability to look up and report hard data.For example, this information can be used to confirm or dispute a student's story if he claims computer issues prevented him from completing his work.It might also help to support an instructor if a grade is disputed because LA can track student activity as well as student/instructor communication.Instructors might also use charts, graphs, etc. from an LA tool when reaching out to a struggling student.These data can help justify instructor concern and persuade the student that there is a problem that needs to be addressed.Instructors might also benefit from their students knowing that this information is readily available.If a student knows that the instructor can see a high level of detail on student course activity, this increases accountability on the student's part.He will feel that his actions matter and someone is paying attention.
It can be very difficult for instructors to remember details regarding student activity, communication, etc.When implementing LA in a course, instructors should remember that these tools are there to support their teaching practice.When questions arise, instructors should remember to consult these tools because they often reveal more information than instructors can readily recall themselves.In addition, when contacting students regarding participation, activity, or grades, it might be helpful to include data generated by LA in that line of communication.This evidence helps students understand that instructors are not relying solely on instincts or memory, but that specific facts and details are available.
Reaching out.Many instructors expressed that LA helps them to reach out to students who are struggling and can result in a better relationship.Instructors often use the information generated by LA to identify students who are struggling or falling behind, and "reach out" to these students by contacting them personally.This simple act is often enough to help students improve because it lets them know that someone notices and cares.One focus group participant said it "shows the students that you are aware of what they are doing and how they are performing.Giving them a heads up early on makes them feel that they are in charge of their performance.Sometimes they may just admit that they are lazy, but at least it is up to them how they will move forward.It also lets them know that you can identify problems."Another said that acting on non-participation lets students know they are missed.
Instructors can use LA tools to identify students who are struggling and initiate some kind of conversation with them.Sometimes students might just need a little nudge.Often students in large or online courses feel that no one notices whether they succeed or fail, and even a few words can make a big difference.Instructors teaching large or online courses know that it is difficult to monitor the progress of so many students when instructors often do not ever meet these students face-to-face.The job is not easy.LA tools can make that job a bit easier so instructors can be more effective in reaching out.
Frequency.A useful strategy is to consult LA tools consistently as the course progresses.Many instructors make a habit of consulting these tools once or twice per week to see if there is any new information to act upon.This consultation provides instructors with information on student activity in addition to what is observable from the course itself.How often these tools should be consulted depends on the course structure.What is important is to develop a schedule that works for the course and abide by it.Wise et al. (2016) stated "the frequency with which the analytics are provided or accessed as well as the schedule for reflective activity will vary depending on the context.The goal is to create a specific timing for cyclical review" (p.12).
Early identification/intervention.Many instructors feel that LA tools are most beneficial early in the course because it is important to identify struggling students early when there is still time to get them back on track.One focus group participant suggested always having an assignment due during the first week of the course and using these tools to see which students are not putting that effort in right off the bat.Identifying and intervening early with these students is key.Another participant recommended identifying where the "point of no return" is in each course and being mindful as it approaches.Helping students get on track with the course before this point can increase the probability of success.
Additionally, it is important for instructors to develop consistent intervention strategies to use when acting upon the information provided by LA tools.Many instructors benefit from the use of preformatted messages.These messages can be used to reach out to struggling students, advise them on where to find help, and direct them to campus resources such as retention, tutoring, writing center, etc.These messages should by no means be restricting and should be edited and customized to whatever degree the instructor prefers, but having preformatted messages makes this kind of communication more consistent and can save instructors' valuable time.It is also beneficial to decide beforehand what constitutes a need for intervention and what kind of intervention is appropriate.Many instructors develop a flowchart or similar visual depiction of their policies, which helps them to decide when and how to intervene.A flowchart like this also provides consistency and saves time.

Self-reflection.
A common theme in the literature as well as in the needs assessment is the use of LA for the purpose of self-reflection.LA can provide a wealth of information to instructors wanting to assess their course and teaching practices.Focus group participants discussed how LA can be used to analyze teaching and adjust courses based on findings (e.g., when students are actively involved, when they lack interest, and where there are areas of improvement).Using LA for test item analysis is useful to this end.Analyzing which exam questions are most frequently missed can reveal what teaching areas need more focus or perhaps might reveal some "bad questions."One focus group participant mentioned that she uses LA to see what level of instructor discussion participation results in higher student evaluations.This helps her to identify how much participation is appropriate so as not to monopolize the conversation or have too small a presence.
Instructors wanting to implement LA in their course structure can greatly benefit from using LA as a tool of self-reflection.The information can supplement the traditional course and instructor evaluation and perhaps reveal more detailed information.This type of self-reflective activity can take place throughout the teaching and learning process, but also at the end of each semester before beginning another.Instructors can use what they learned from LA in one semester as they design and make changes to the course for the next semester.
Align LA with pedagogical intent.The last theme of LA adoption identified in the literature and needs assessment is that the use of LA tools must align with the instructor's pedagogical intent.These tools are not one-size-fits-all.There are some circumstances in which certain features are not useful in a course.There are even situations where LA is not useful at all in a course.Instructors must always be mindful of what is being measured and reported and whether this information is an accurate reflection of learning based on their course design.Pedagogy must drive the use of LA.Wise et al. (2016) presented the Align Design Framework which offered principles for pedagogical practice to support the use of LA.The idea is that instructors can adjust their pedagogy to support LA implementation.While there is an important relationship between pedagogy, course design, and the use of LA, focus group participants felt that pedagogy is of greatest importance and the use of LA must be aligned and adjusted to fit the developed pedagogy.One focus group participant stated, "You must analyze what elements of LA tools will add value to your course and know this before the course begins so you have a plan for how to use LA in your course."It is important for instructors to understand what is being measured by these tools and how, consider how these measures align with the course structure and pedagogy, and remember this when consulting these tools and acting on the information they provide.
Another participant noted that the use of LA also depends on the interest of the faculty member.This model is useful for faculty who desire to utilize LA, but the use of these tools should not be forced.Some instructors are not interested in these tools and feel that they can serve their students and develop relationships without the use of this type of technology.LA should only be used to supplement and assist instructors but will never be able to replace the personal connection between instructors and students.
The idea of aligning the use of LA with the instructor's pedagogical intent was discussed from a number of perspectives relating to the implementation and adoption of LA, but it was also discussed from a cautionary perspective.Many participants felt that instructors implementing LA in their courses must be wary of these tools and consider how much weight should be placed on the information they reveal.These concerns relate to the second category of themes, which reflect an attitude of caution.
Caution.Much of the literature, as well as the qualitative data collected in the needs assessment, revealed a very cautious attitude towards the implementation of LA in the classroom.Many felt that these tools can be inaccurate, impersonal, or intrusive.It is common for users to be wary of new technologies, and LA is no exception.A number of themes emerged within this category such as: skepticism, fear of overdependence, and the questioning of the overall usefulness of LA.
Skepticism.If LA is going to be useful in a course, it is essential that the use of LA aligns with the instructor's pedagogical intent; however, many instructors question whether this can be the case.When these tools are not transparent about how they collect, analyze, and report data, instructors become skeptical as to whether the data can be trusted.The way these tools measure student success is not always representative of the students' effort and performance.In addition, different tools use different metrics so it is difficult to compare them.Transparency is essential if instructors are going to trust that the information provided by LA tools is accurate and can be acted upon.One participant noted that she wants to see exactly what measures are going into the algorithms that detect and label "at-risk" students.
Similarly, there is concern that LA is too often about the bottom line and does not take the cultural context of the students and campus into account.An example of this is that many students at SWOSU work full time, often on a family farm.These students might begin to struggle to keep up, and LA does not reflect these types of situations.LA is unable to identify students who are personally at-risk in some way rather than academically at-risk.While LA cannot detect this level of detail regarding students' personal circumstances, they can accurately reflect symptoms of a deeper problem.One participant noted that these tools must be used critically to help instructors understand these underlying causes.
Another concern is that students may begin to understand what activity these tools measure and how they measure it, and these students may begin to "work the system."For example, if an LA tool measures how long students are logged into the LMS, they may log in and stay logged in while working on other things and not actively engaged in the course.Another example is if an LA tool measures the number of clicks (e.g., click tracking software) students may use this to their advantage by clicking their mouse randomly to increase their participation level.One participant said that students might think "you want more clicks, I'll give you more clicks!"Some LA tools measure student performance in relation to the performance of the class as a whole.Some participants expressed concern that students may attempt to take advantage similarly to when a class is graded on a curve.Everyone underperforms because they know their performance is measured as it relates to the class as a whole.While it is uncommon for instructors to actually assign grades based on LA data, the concern about this misuse of LA is real.
Participants expressed the fear that LA tools may encroach on privacy in some way.This concern has already been expressed in the literature on LA.Picciano (2014) pointed out that "as well-intentioned as LA might be in terms of helping students succeed, this 'big data' approach may also be seen as 'big brother is watching' and, as such, an invasion of privacy that some students would find objectionable" (p.41).Many fear that it might make students uncomfortable for instructors to have this level of detailed information, but they also fear that administration will use this information to monitor instructor performance.This fear of surveillance is closely related to the fear that these tools do not always measure performance accurately because there is no "onesize-fits-all."The concern is that administration will use LA destructively to monitor employees, which might create a privacy issue.
One participant mentioned that faculty and administration alike must come to a level of "rhetorical literacy" in order to make proper use of LA.Selber (2004) introduced the idea that there are different levels of literacy, which can be developed regarding the use of technology: functional literacy (computers as tools), critical literacy (computers as cultural artifacts), and rhetorical literacy (computers as hypertextual media).The participant noted, "The basic idea is functional literacy is the most basic kind of usage of technology, while rhetorical literacy requires a much more sophisticated self-awareness of the technology user.Selber (2004) argues that most users get stuck in the critical literacy stage and think that there is no other place to go, especially when it comes to using technologies responsibly and ethically."This participant felt that users of LA tools should reach a level of rhetorical literacy in order to use LA properly, but also felt that is unlikely to happen.The main concern was that administration could inappropriately use this technology to monitor instructors without having a true understanding of the technology, the course, the instructor, or the pedagogy.Rhetorical literacy would mean that these things are critically understood which would enable users to make effective use of LA.Many participants felt that LA should be used as a tool, not a weapon.

Fear of overdependence.
A similar theme found in the needs assessment is the fear that users will become overly dependent on these tools.The concern is that faculty and administration might put too much stock into these tools and treat them as the "end-all-be-all" solution to the problem of helping at-risk students and increasing retention.One participant noted that it is a problem in our society in general for people to want a quick-fix answer or something that will make everything better, but that is not how it works.There is a time and a place for LA.He advised to not be too critical or too enthusiastic about the use of LA.Just as LA should be used as a tool, not a weapon, users need to remember that it is only one tool in the toolbox.

Question of usefulness.
Finally, some instructors question the overall usefulness of LA tools.Many mentioned that some students are just not prepared for a course and there are no interventions that would enable the student to succeed.One participant also questioned to what degree instructors should commit to helping the students succeed, and what should simply be left to the student.While it is ultimately up to the student to succeed in a course, instructors should also be available and willing to use whatever resources and time they have available to support students.LA tools ultimately save instructors time and act as an assistant for instructors wanting to look deeper into the level of student participation.

Model Conclusions.
It is important to be mindful of these themes and cautious about the implementation of LA, but these concerns do not mean that LA cannot be implemented successfully when approached cautiously.Instructors should remember that LA is a powerful tool, but should not be used as a weapon, and this tool is only one in the toolbox.LA is not a quick fix answer that will ease all of the retention problems faced by instructors, but it can serve to assist them in their efforts to support students, which is the ultimate goal.These tools must be used critically while seeking to reach a level of rhetorical literacy concerning this new technology, which can greatly benefit students and instructor practice if implemented appropriately and effectively.

Discussion
The purpose was to develop and validate an instructor learning analytics (LA) implementation model.This model was developed to enable instructors to effectively implement whatever LA tools they have available in their courses.Although based on the research conducted at a single institution using only two available LA tools, the model is intended to be generalizable to a number of environments and LA tools.A thorough review of the existing literature on LA as well as a needs assessment guided the development of the model.

Implications
This study helped to identify the needs of instructors wanting to implement LA in their courses.The results informed the design of an instructor LA implementation model.The model was validated internally by a panel of experts.The final model includes practical and conceptual guidelines regarding the use of LA and is meant to be generalizable to a number of environments and LA tools.

Recommendations
This section includes two categories of recommendations.First, recommendations for future research are presented.Second, recommendations for professional practice in relation to the implementation model are presented.
Future research.This study could be expanded to include external validation of the model presented here.Using the instructor LA implementation model from this study, researchers can work with an institution of higher education to study the impact of the model's use.This type of study would also measure the model's usability (usefulness, effectiveness, efficiency, and learnability) in a setting external to the one where the model was developed.
The model could also be studied in relation to student retention.Researchers could seek to study how instructors' use of this model to implement LA in their courses might affect course grades and student persistence.Researchers could also study the effect of the model's use on the overall teaching and learning process.
Recommendations for practice.The first recommendation is that instructors at SWOSU wanting to implement available LA tools and technologies (i.e.Dropout Detective and/or Canvas Analytics) use the model presented here to support their efforts.Review of this model will enable instructors to better understand how to effectively implement LA in their courses.The model demonstrates the benefits of LA and practical and conceptual guidelines to guide LA implementation.It also includes some areas of caution that instructors should be aware of so as not to fall into common pitfalls in the implementation of LA.The model should be made available to SWOSU instructors through the Center for Excellence in Teaching and Learning, and a workshop should be offered by the researcher for interested faculty.
Second, since the model was designed to be generalizable to a number environments, instructors at other universities can use the model to implement LA in their course.This model is meant to be something that can be adopted and used by individual instructors in individual courses.The institution as a whole does not have to implement this model as a standard of practice.Instructors can use this model at will, and it should be used only by those who have an interest and desire to do so.The researcher will make this model available to any interested parties who might put it to use in order to improve their teaching practices.