Troublesome Knowledge: Identifying Barriers to Innovate for Breakthroughs in Learning to Teach Online

Embedded within advice for starting simple with online, blended, or technology-enhanced teaching are practices that can be troublesome for some faculty who are learning to teach this way. For example, embedded within the principle of a clear, organized, navigable course can be the concept of chunking content into modules, the skills associated with screen casting and posting a course tour, and the practice of socializing students to the course organization through demonstration, explanation, and reinforcement. This empirical-qualitative study collected 123 cases of troublesome knowledge from 41 participants and analyzed them through Perkins’ troublesome knowledge framework. Results include subcategories and common themes across cases of inert, ritual, conceptually difficult, tacit, and foreign/alien knowledge. From these results, we recommend that faculty development approaches should take specific aspects and cases of troublesome knowledge into consideration for online teaching preparation.

In higher education, new professional roles are evolving with the aim of preparing and supporting faculty to teach in online, blended, and technologically-enhanced modalities. These roles include instructional designers, learning designers, academic technologists, and other educational developers. For clarity in this paper, we use the term "instructional design professional" categorically to include those roles. This introduction begins with an example that illustrates a common dilemma for instructional design professionals in higher education that, when unpacked, reveals larger questions addressed in this research study about how faculty learn how to teach in online, blended, and technologically-enhanced modalities.

Example
The instructional designer concluded his second consult of the day and knew there would be more to come. An update to the learning management system (LMS) had gone live overnight, and now several of the most commonly used tools-the quiz function, assignment drop box and others-were grouped together in a new submenu. The overall update made the LMS look cleaner and more in-step with modern websites, but it was causing trouble for many instructors. A certain group of faculty members had clearly been using a rigid step-by-step process to access these tools, and now that things had moved around, they were unable to find them again. It seemed clear to the instructional designer: These tools were grouped together under a new heading of "Assess Learning" in the minimized side menu. But something about the way this group of instructors used the LMS made them unable to find these tools they had relied on for years. One thought the apparent disappearance of menu items was his fault: He must have done something to "break" that feature. One became frustrated: "No one told me this was coming and now I have the time-consuming hassle of trying to get this resolved." Another feared that students would have trouble locating class activities and assessments, and that this would reflect poorly on her teaching, or set back the instructional timeline to accommodate the confusion.
This story illustrates the wide range of reactions and experiences that instructional design professionals see during their interactions with faculty, and how challenging it can be to learn to teach online. Even with extensive programming, available starter templates, regular webinars and other resources, there are still challenges that function as barriers to learning to teach online. Our study aims to better understand what these barriers are and how they are unique to online teaching, an especially relevant problem to higher education institutions as they continue to increase their online offerings, and as the number of available tools and features only grows each year. In this paper, we will first outline the current understanding of these barriers and introduce the concept of troublesome knowledge (Perkins, 1999). Then, we will describe our study design (a survey instrument given to instructional designers and those with similar roles), our results, and the implications that our findings have on the field of instructional design.

Research Questions
In this empirical-qualitative study, we sought to investigate sources and stories of Troublesome Knowledge (Perkins, 1999) associated with learning how to teach online. Our study examined two, complementary research questions: 1. What aspects of learning to teach online are troublesome for some faculty learners?
2. What experiences of tacit, foreign/alien, conceptually difficult, inert, or ritual knowledge do educational developers encounter in their work with faculty as they learn to teach online, blended, or technology-enhanced modalities? We predicted that participants would have anecdotes to share from their experiences preparing faculty for multi-modal instruction and that, despite the wide call for participation and anonymous submissions, participant stories would have common themes around the challenges they shared.

Literature Review
Instructional design professionals have long been studying and seeking the most effective ways to teach university faculty how to teach in online, blended, and technology-enhanced modalities. In many, well-documented cases (e.g., Nilson & Goodson, 2018;Wiggins & McTighe, 2012;O'Keefe, Rafferty, Gunder, & Vignare, 2020), one common practice involves working from a set of research-based evaluation criteria, such as those found on the Online Learning Consortium's (OLC) OSCQR Course Design Review Scorecard, the Quality Matters Rubric, and several others. Faculty don the role of a learner in these preparation courses and learn through both the program content and facilitator modeling how online instruction can be achieved. For many faculty-learners, programming designed in this way-coupled with individual consultations and support, as well as ongoing professional development-is enough to help them navigate, and sometimes thrive with, online teaching and learning.
However, we also know that many faculty members continue to struggle with learning how to teach online, even though they may have decades of teaching experience, expertise within their field, or relative comfort with learning new technologies. Depending on several factors, including size and capacity of institutional Centers for Teaching and Learning or Academic Technology, the much-needed support for these faculty learners can be time-consuming and complicated (see Intentional Futures, 2016, p.15). These struggles occur even though instructional design teams are often well-versed in online teaching. For learning/instructional designers, technologists, and faculty developers, this work can be frustrating because it feels like a lack of buy-in, trust, or understanding on the part of the faculty learner about how online learning is different from face-to-face-and recently, how remote learning is even different from online learning (see Hodges, Moore, Lockee, Trust, and Bond, 2020).
Moreover, the COVID-19 pandemic propelled more faculty than ever to learn how to teach in some form of "virtual" modality, whether or not they wanted to, with rosters of students that were also propelled into the online learning, whether or not they wanted to. The online teaching and learning community of practice was tasked with preparing and supporting all faculty with this effort (see, for example, Koenig, 2020;Decherney & Levander, 2020;O'Keefe, et al., 2020). While much of the advice for those new to online teaching and learning involves 'keeping it simple' (Cavanagh & Thompson, 2019), members of the online teaching and learning community of practice recognize that, embedded in those simple principles, are practices that can still be difficult. For example, embedded within the principle of a clear, organized, navigable course can be the concept of chunking content into modules, the skills associated with screen casting and posting a course tour, and the practice of socializing students to the course organization through demonstration, explanation, and reinforcement. While these attributes can become fluid with iterative practice, they may not necessarily be intuitive to new learners. Not only are these more than "simple" tasks that can be checked off of a course design list, but they are also potential technological and pedagogical barriers (Mishra & Koehler, 2006) for some faculty new to teaching this way. Either way, the impacts of inadequate teaching preparation are multifold: The institution's students receive a disservice when their courses are not utilizing well-designed and implemented instruction (Xu & Xu, 2019); equity gaps compound for minoritized students (Plotts, 2020;Wood & Harris III, 2020); and faculty sense of competence and satisfaction in their work can be compromised (Callo & Yazon, 2020).
In this study, we explored the idea that difficulties in learning to teach online do not lie with any one party, either faculty learners or instructional designers, but rather that there are specific aspects of learning to teach online that are difficult all on their own. Similar to the shift in understanding that is common in K-12 education, from students having deficits that education is meant to fix, to using students' strengths to help them grapple with inherently difficult concepts, we want to explore what happens when we shift the discussion from being what the faculty learners can or cannot do, to what is it about online learning that might be difficult for anyone who is not an expert.

Theoretical Framework
We explore this question through Perkins' (1999) framework of troublesome knowledge. Perkins proposes the idea that certain types of knowledge may be troublesome in the context of a constructivist classroom. In this setting, an instructor would want to help learners reach a deeper level of understanding through their own sense-making (to construct the knowledge themselves with the teacher's guidance). However, a skilled instructor recognizes that when a learner is struggling with a concept or skill, there may be different types of struggles, and therefore different ways a teacher would respond. Perkins expands on this and posits that there are four types of troublesome knowledge: inert, ritual, conceptually difficult, and foreign. Later, the category of tacit knowledge is also incorporated into this framework (see Meyer & Land, 2003, who named a knowledge type that Perkins had described but not labeled). Table 1 summarizes each of the five types: Table 1 Types of Troublesome Knowledge

Knowledge Type Definition
Conceptually Difficult Generally, a mixture of new ideas mixed with previously held misunderstandings that conflict.

Foreign
Occurs when there is a perspective that conflicts with the learner's own, often unknowingly.

Inert
Knowledge used in very specific circumstances, that otherwise is not actively used by the learner in other situations.

Ritual
Routinized knowledge that may be accessed often but becomes divorced from its original meaning or intent.

Tacit
Knowledge that becomes implicit or second-nature to someone experienced in this area.
This idea, that there are different types of knowledge that can pose problems for learners marks an important shift in approaching the instructor-student relationship, or the instructional designer-faculty learner in this case. Describing the knowledge itself as troublesome shifts the problem onto what it is being learned and away from the learner. While establishing that learning to teach online contains unique manifestations of troublesome knowledge, it is important to also establish that online teaching itself is a standalone discipline and skillset, and not necessarily one that instructors at higher education institutions will be fluent in, or even recognize as a practice to be developed. Faculty development is a broad area, responsible for many different goals. Providing faculty development specifically in the area of online teaching competes with many other demands on faculty time, many other priorities for faculty development, both pedagogical and non-pedagogical and even with other areas of technology not specific to online teaching (Belt & Lowenthal, 2020).
Applying a constructivist approach to learning how to teach online creates a set of paradigms for instructional design professionals to consider. In scholarship that applies constructivist theory to instructional design contexts, Karagiorgi and Symeou (2005) outline these implications for the three main phases of instructional design: analysis, development, and evaluation. Each of these phases are opportunities to consider how the faculty learner may be grappling with different types of troublesome knowledge. For example, when an instructional designer is analyzing what type of tasks might be most appropriate for their faculty learner's situation, considering how a specific type of troublesome knowledge may be interfering with what the learner knows so far may be most relevant.
Troublesome knowledge is also important as learning designers consider how to design opportunities for authentic learning. In the example story in the introduction, we mention an instructional designer who has seen multiple faculty members struggle with an LMS menu redesign. If we apply a troublesome knowledge perspective to that scenario, the instructional designer could attempt to identify which troublesome knowledge type is at play for that learner in that scenario, and then facilitate scenarios or tasks that attempt to help the learner address the root of the troublesome knowledge, instead of developing a new heuristic to avoid it for the time being. However, before instructional design professionals can develop strategies for addressing individual troublesome knowledge types, it is important to better understand how they are perceiving troublesome knowledge in their existing interactions with faculty learners.

Participants
Participants were recruited through a combination of methods, including purposeful and "snowball" sampling where participants are encouraged to suggest additional subjects for participation (NSF, n.d.). A survey was distributed through a variety of higher education, instructional design, and educational development listservs. Criteria for participation in this study included the following: 1) minimum of one year employment as an instructional designer, technologist, or educational developer at an institution of higher education; 2) as part of instructional design or education development work, have responsibilities for teaching/training faculty how to teach online. Conditional formatting was incorporated into the research instrument to send people to an end page if they did not meet criteria for participation. The Recruitment Message, Consent Form, and Survey Overview identified the purpose of the research in the following way: "[This is] a study to learn more about the difficulties that some people encounter when they are learning to teach online." The recruitment message also included the goals for participation: "[W]e hope to draw from your experiences with faculty development around online, blended, and/or digitally-enhanced instruction for this research." Therefore, we were confident that responses collected through the instrument were coming from a place of relative experience and expertise with the learning experiences of faculty who were new to teaching online.
It is important to note that we did not seek participation from faculty learners to selfreport about their experiences learning how to teach online. While the first-hand perspectives of learners are relevant, our study design was informed, in part, by the Technological, Pedagogical, Content Knowledge (TPACK) framework (Mishra & Koehler, 2006), which suggests that teachers (in our case, instructional designers and educational developers) have enough content and pedagogical knowledge to plan and address situations where learners (in our case, facultylearners) get stuck. To offer a more familiar example, let's consider the difficulty that a student might encounter when learning math. He might, for example, be able to calculate the area or perimeter of a rectangle or recite the equation for area as base times height, but struggle with explaining what area or perimeter is, or get stuck with how to proceed on a word problem that requires the learner to recognize which calculation to perform in order to solve it. In this example, there is certainly much to unpack from observing the learner as he works to understand the concepts, but his teacher will be able to speak with some expertise about what is conceptually difficult about this content (area and perimeter) as well as where is the source of the barrier and what pedagogical approaches may help him overcome the barrier. From our perspective as learning innovation researchers, our ideal participants were the instructional design professionals whose professional work includes teaching people how to teach online, blended, and technologyenhanced classes because they possess relative online teaching and learning-related TPACK that would allow them to articulate the areas of difficulty we hoped to uncover through our research instrument.
The study design and protocols were approved by the University Institutional Review Board. Participants were informed that participation was voluntary and that they had the right to withdraw from the study at any time. Familiarity with Perkins' (1999) troublesome knowledge framework was not required or even preferred for participation in this study, though we anticipated that participants would submit stories that could be classified according to that analytical framework.

Instrument
The survey instrument for this study was a qualitative questionnaire, containing three blocks of questions: Demographics (8 questions), open-ended questions related to troublesome knowledge types (5 questions), and one optional, open-ended question about their professional experience considering the COVID-19 pandemic. To control for survey fatigue, where participants might tire of long-form answering, the five open-ended questions in the second block were randomized to ensure that, even if participants exited the survey prior to completion, enough responses would be collected across the five questions to conduct analysis. This mechanism was implemented after a pilot phase revealed that the short survey took up to 30 minutes to complete.

Analysis
The five, open-ended questions on the survey instrument were already drafted using the major categories of troublesome knowledge from Perkins' (1999) framework: inert, tacit, conceptually difficult, ritual, and foreign/alien. These five knowledge types served as initial categories for a first round of analysis. In this first round, entries that contained multiple examples within a single text field were separated into individual units of meaning. For example, if a participant responded with two examples to a question about rituals that function as barriers to online teaching development, that single response was separated into two, individual entries from the same participant ID. A second round of axial coding (Marshall & Rossman, 2016) was applied to individual entries to identify patterns or subcategories within the troublesome knowledge category.
Following the coding process, an inter-rater reliability analysis (Armstrong, Gosling, Weinman, & Marteau, 1997) was conducted across all entries. Typically applied to quantitative studies, inter-rater reliability testing involves a process whereby multiple researchers (often two or more) independently code a common dataset and then compare the codes for agreement. In empirical-qualitative studies, this process has been used to establish accuracy in data representation and credibility when other qualitative methods, like triangulation, are not applied. To conduct the inter-rater reliability analysis in this study, all individual entries were stripped of both the categories for which they were submitted by participants and secondary codes applied by the researchers during axial coding. The stripped entries were then randomized and re-ordered for the inter-rater reliability analysis, wherein two independent raters with expertise in troublesome knowledge (Perkins, 1999) coded each entry for tacit, ritual, foreign/alien, conceptually difficult, and/or inert knowledge. This round of coding was then compared for agreement and is reported in the Results section. The Results section also includes a metric for agreement between a rater and the participants-in other words, the raters' labels were compared for agreement with the category for which participants submitted their case examples. While agreement with the participant is not a necessary step in inter-rater analysis, this process offers another point of data representation and reliability.

Findings Demographics
The survey instrument collected information from 48 unique participants, but 7 submissions were omitted from analysis for failing to meet the minimum eligibility for participation. The study's pilot phase included participants who met the criteria, as well, so their entries are counted among the data. Criteria for participation included instructional/learning design, academic technologist, or educational development roles in higher education. Therefore, results are reported for 41 participants (n=41).
In the Demographics block, participants were asked to indicate their higher education role(s): 15% (n=6) of participants indicated their role as instructional/academic technologists, 54% (n=22) were instructional/learning designers, and 31% (n=13) selected "Other," with educational developer/administrative responses, like educational developer, faculty, or center for teaching and learning director entered the text field. Because this study positioned the intended participants as professionals with relevant experience and expertise in online, blended, and technology-enhanced educational development, participants were also asked to report their years of educational/instructional design experience. The majority of participants (73%; n=30) reported four or more years' experience, while 17% (n=7) indicated 2-3 years and 10% (n=4) indicated up to one year of experience. Gender participation included 63% female (n=26), 32% male (n=13), one non-binary participant, and one declined to state.

Troublesome Knowledge
The survey collected 123 unique scenarios from 41 participants. Five, randomly-ordered, open-ended questions prompted participants to self-report stories from their experiences teaching and supporting faculty with online, blended, and technology-enhanced course design and pedagogical development. Perkins' (1999) troublesome knowledge framework was used to collect and categorize participant submissions as well as to independently code unique responses during a round of inter-rater reliability analysis. Results for this section report only those cases for which raters reached 100% agreement. Those cases for which agreement was not reached continue to be analyzed for patterns and outliers.

Inert Knowledge
Inert knowledge involves concepts and processes that are learned but are seldom accessed. Perkins (1999), citing cognitive psychologists Bransford, Franks, Vye, and Sherwood (1989), and Bereiter and Scardamalia (1985), analogizes inert knowledge as " [sitting] in the mind's attic, unpacked only when specifically called for by a quiz or a direct prompt but otherwise gathering dust" (Perkins, 1999, p.8). In the present practice of online course design and teaching, such knowledge includes technical processes, such as the multiple steps and mouse-clicks involved in uploading a file or embedding a video on a course site. More than just the steps, this knowledge can also include remembering to perform certain tasks that are necessary only once per term, such as making a course visible for students to access or posting grades to the Registrar's database even though the learning management system may house an online gradebook.
In this study, participants submitted 32 cases to the question for inert knowledge. During inter-rater reliability, raters reached 81% agreement with the participant (26/32 cases) and 69% agreement with each other (22/32 cases). Across only those cases for which raters achieved 100% agreement (n=22), patterns observed included new academic terms or courses, checking for or managing updates, roles and processes, and multi-step processes. The brief table, below, shows example participant responses that demonstrate inert knowledge.

Inert Knowledge Category Example Participant Response
Technology Updates "Lots of things related to the many technology programs we use -our student advising software is clunky and not well used, so every time there's a new push we all have to go back in and relearn how to do everything, which is a huge barrier." New Academic Term/Course "Remembering to re-synch various apps such as Flipgrid or Screencast-o-matic" Roles & Processes "Modifying dashboard -people will hide their classes and then forget how to locate them. Or, they don't know how to modify."

Multi-
Step Processes "Our LMS organizes quizzes separately from the individual quiz questions. When importing between quizzes, you have to remember to bring the 'question bank.' This is not a frequent occurrence and the analogy between a physical quiz and the digital quiz do not align (if you were to "copy" a physical quiz, you never have to worry about the questions being separated). This is a difficult concept to explain and often needs to be repeated even after an instructor has successfully completed it before." Perkins (1999) describes ritual knowledge as routine or habitual in nature. He explains that ritual knowledge is "how we answer when asked such-and-such, the routine that we execute to get to a particular result" (pp. 8-9). An innocuous example of ritual knowledge is the series of mouse clicks or keyboard strokes that a person executes to take a screenshot. Depending on the software used for this purpose, the act of taking a precise screenshot can take few or several steps. What makes ritual knowledge troublesome is that it can be difficult to break habits or consciously establish new, more effective routines. In online teaching and learning, there is often more than one way to perform a single function, and troublesome rituals contribute to the frustration of processes taking too long, being overly complicated, or being so inflexible that changes (e.g., technology updates) cause the ritual to break down all together.

Ritual Knowledge
Perkins acknowledges that inert knowledge and ritual knowledge can manifest in common ways, clarifying that "Whereas inert knowledge needs more active use, ritual knowledge needs more meaningfulness" (p. 9). Participants submitted 21 scenarios for the question about ritual knowledge; but during inter-rater reliability testing, raters tended to apply other possible labels, such as inert or tacit, depending on the way the case was presented in writing. Raters did achieve 81% agreement (17/21 cases) with participants, and representative cases for which 100% agreement was reached between raters are shown in Table 3, below. Of those cases for which agreement was reached, common themes included ritualized pedagogies (i.e., a ritualized way of thinking about or approaching one's teaching), task-based rituals, and the problem of trying to perform the same physical ritual to the digital environment.

Ritual Knowledge Category Example Participant Response
Ritualized Pedagogies "The biggest one is the lecture. Faculty immediately think that they will just do what they do in class but over Zoom. The toughest are the seminar instructors. They swear they don't lecture at all during their 3hr seminar session, and just automatically assume they can do three hours over Zoom, but also that without a physical presence, learning will be impossible." Task-based Rituals "In our campus' LMS, I have seen this knowledge prevent new learning whenever there is an update to the LMS features or interface. For example, managing users in a course has a new, mainstreamed interface, but I have seen instructors continue to use the routine they are familiar with even though it will take longer." Physical Ritual in Mediated Space "Teachers of younger children wanting to use highly coloured and varied font styles to make it attractive, as they do for offline materials and then not understanding why the text editor does not allow them to do this online or their instructor does not advise them to do it."

Conceptually Difficult Knowledge
Conceptually difficult knowledge exists in many facets of human experience, and Perkins (1999) describes this knowledge as troublesome as it mixes with misimpressions from learner experience and ritualized responses to new problems (p. 9). Because teaching online is paradigmatically different from teaching face-to-face, there are some aspects of online teaching and learning that can be considered conceptually difficult, especially for some faculty-learners new or skeptical to the paradigm. The term "module," for example, is not new-even in teaching and learning-but the concept of an online module is one with a distinct start and end point, with learning objectives and a series of steps for student learners to take through a learning pathway designed to help them practice and ultimately achieve those outcomes. For faculty who are not thinking about their teaching or organizing their content in this way, the idea and execution of modular course design and instruction can be conceptually difficult.
Twenty-four scenarios were submitted to the question in this study for conceptually difficult knowledge. Raters reached 100% agreement with participants (where at least one rater agreed with the participant) and 54% with each other (13/24) during inter-rater reliability testing. Across those cases where 100% inter-rater agreement was achieved (n=13), patterns included the physical metaphor for the digital application, pedagogical concepts, organization of the digital content, and the idea that not all digital tools are the same. Table 4 includes a brief illustration of participant responses for conceptually difficult knowledge.

Physical Metaphor
"Groups is a difficult knowledge piece because while some of the basic features are comparable to how groups would be used in a face-to-face setting, there are additional features that would often help instructors streamline their course, but for many this is hard to understand without prior experience." Pedagogy "The role of assessment can be difficult because we often use it as an evaluative tool rather than a learning experience." Organization of Content "[...] Last week I worked with a faculty member who wanted modules in their course because they were told they needed them. In a screen share of their course it became apparent that they had learned how to create module headers but didn't know what to do with them, or how the content got attached to a specific module [...]" Digital Tools (Differences) "Moving to a new LMS is exactly what I am dealing with. Things are similar but different and instructors are often tempted to 'figure it out on their own' instead of asking for help and this results with confusion for the instructor and more importantly the students."

Tacit Knowledge
Expanding on Perkins' (1999) troublesome knowledge framework and noting that sometimes troublesome are the subtle, often invisible ways of thinking and doing that practitioners with relative expertise employ, threshold concepts theorists Meyer and Land (2003) add tacit knowledge as "understandings [that] are often shared within a specific community of practice" (Meyer & Land, 2003, p. 7, citing Wenger, 2000. Although learning innovation is an emerging field (Kim & Maloney, 2020), and the nature of technology-enhanced teaching and learning continues to change, instructional design professionals and experienced online educators possess shared ways of thinking and doing that function as tacit knowledge. Examples include internal processes for deciding which academic technology to use for particular pedagogical purposes, or other heuristics for designing learner pathways through a course of study, thinking of the learning management system as a vehicle for experience, rather than a repository for course materials. This knowledge can be troublesome to those new to teaching online because it includes practices and processes that become so internalized that they are invisible to practitioners.
Study participants submitted 22 cases to the question for tacit knowledge. Raters reached 73% agreement with the participant (16/22 cases) during inter-rater reliability testing. Across those cases for which agreement was achieved, patterns included ways of thinking about the learning management system, notions of alignment between intended learning outcomes and course elements (e.g., assignments, resources, etc.), and approaches to the integration of academic technologies with the learning design. Table 5 includes a brief representation of cases upon which agreement was reached for tacit knowledge.

Tacit Knowledge Category Example Participant Response
Academic Technology (tools) "Gradebook. Online gradebooks have great potential but without a clear, intentional strategy, they can cause more confusion than necessary. In my experience, many instructors do not think about how an online gradebook functions (especially for students) until towards the end of the academic term or when a student brings it to their attention." Alignment "I see this when faculty begin creating a course. The conversation often begins with a 'beginning to end' of course perspective. The focus is immediately on the 'order of topics' without first considering alignment between outcomes -objectives-activities -assessment. Tacit knowledge comes out in the traditional plan to 'teach' through the order of the textbook and give a midterm and final exam. The idea of learner centered and outcome driven design is not often an immediate 'starting' place. Because of this, I begin most design conversations with 'What do you want students to be able to 'do' by the end of this course?' How will you know they've achieved this?" Learning Management System (Purpose/Possibilities) "Certain actions are default. When creating a new course, it should be default to hide certain things from the course menu-e.g., pages, assignments, etc., that you don't want students to have access to, or that you might want to release later. There is a time and place when a course should make everything available for users to see; a time where you want to restrict what people have access to. The workflow that you use to release information to users: e.g., using pre-reqs, locking until a certain date."

Foreign/Alien Knowledge
Foreign knowledge, according to Perkins (1999), "[...] comes from a perspective that conflicts with our own" (p. 10). Sometimes, foreign knowledge is simply a perspective that one hadn't thought to consider before (e.g., why to include alt text with images or use proper text formatting on web pages), and sometimes it can stem from an individual's experience, values system, or cultural context. In online teaching and learning, foreign knowledge can be troublesome because the faculty-learner may not recognize this knowledge at play, or they may initially or fundamentally disagree with some of the instructional design advice offered by instructional design professionals.
Across the 24 scenarios submitted by participants for this question, the most common form of foreign knowledge reported was the faculty-learner's failure to consider the studentlearner's perspective in the course design or instruction. Other themes included the idea that one's discipline was not conducive to online teaching and learning, and the idea that online teaching and learning should be a replication of face-to-face instruction. During inter-rater reliability testing, raters achieved 100% agreement with participants (24/24 cases) and 83% agreement with each other (n=20). Representative cases for which 100% agreement was reached are shown in Table 6. Inflexible Discipline "I consistently come across this notion when I work with faculty who believe that their subject area 'cannot be taught online'.They usually phrase this belief in a way that demonstrates an unfamiliarity with the techniques and tools that would allow you to convert or modify a learning experience so that it would work in a virtual format. Nonetheless, their belief immediately contradicts both my philosophy of teaching and learning and established learning frameworks like TPACK, SAMR, or RAT." Re-Create Face-to-Face "The idea that face to face teaching is the gold standard so that when teaching online they should just try and replicate that face-to-face style. For example, record 60-minute-long lectures or have 60 minute long, 3 times a week synchronous sessions that replicate the face to face lecture."

Discussion
Although this study was widely distributed and participation came from varied higher education roles and contexts, we saw common cases and categorical patterns during analysis of the 123 unique cases of troublesome knowledge submitted by participants for this study, suggesting that there are common aspects of learning to teach online that are difficult for some faculty learners. Traces of a deficit perspective toward faculty learners surfaced in cases categorized as ritual and foreign knowledge. For example, in a case of ritual knowledge, where the faculty needs to change text formatting habits, particularly for emphasizing important information to students, the participant summarized the problem: "Much of my work is about explaining why those habits no longer serve students." Likewise, foreign/alien knowledge proved to be problematic because of the sheer time it can take to have a breakthrough in perspective or to be able to speak with fluency about online teaching and learning. One participant shared: "When faculty make requests that are unspecific, it is mentally taxing to figure out what exactly their issue is. They don't have robust enough vocabulary to describe their own problems in a way that will help me help them." These descriptions are phrased in a way that places the problem on the learner, whereas viewing these scenarios as examples of troublesome knowledge would reframe the struggle to learning to teach online as a natural thing.
We did see a more constructivist viewpoint with some types of troublesome knowledge. We received the most unique cases for inert knowledge and participants described these as understandable challenges inherent in the technology: "If there are changes made to the LMS, this can be perplexing and lead to anxiety and stress." One implication of this analysis is that development for instructional design professionals to view their work with faculty through the lens of these knowledge types can help dismantle the deficit view of faculty learners. It seems instructional design professionals may be apt to do this for some troublesome knowledge types already, so this could be an entry point for expanding this viewpoint across other types.
Our results add to the growing body of literature on online, blended, and technologyenhanced education, specifically as it relates to the work of instructional/learning design and educational development. With better understanding of the challenges some faculty encounter when they are learning to teach online, instructional design professionals can more effectively support faculty in these areas. Currently, much of the professional development for these roles includes webinars and programs on course design, program development, quality assurance, and a la carte programming on specialized topics, such as assessment models, equity-minded course design, models for teaching in various modalities, high-impact practices, and many others. These opportunities are essential elements in a train-the-trainer model of educational development. What is missing in the professional development of educational development staff is attention to andragogy and ways to help all faculty make breakthroughs for learning innovation. For example, these results could be useful in informing how learning design teams plan their work and programming throughout the academic year. If there is a consistent set of inert knowledge problems that come up at specific times each year (such as setting up new courses at the start of the term), the faculty development calendar could anticipate this with targeted programming or other communication with faculty. Likewise, a learning design team could decide to adopt the troublesome knowledge lens when performing consults with faculty members and use common language to help faculty learners identify for themselves when they are experiencing challenges (e.g., "I notice that the option you are looking for has moved from its familiar place. I can show you how to take a minute to familiarize yourself with a new layout for when this happens").
This study was designed to be replicable and aggregable. Given the consistency across cases for each troublesome knowledge type, we suppose that a re-launch of this study in the short-term would yield similar cases to those already collected. Given rapid changes in academic technology innovations and the teaching modality possibilities explored in a post-COVID-19 era, we would expect that long-term replications might collect a similar distribution of cases across knowledge types, but that the stories themselves will reflect different acute problems. Additionally, some of the cases collected for this study did not organize neatly into one of the five troublesome knowledge categories. Further analysis may reveal additional categories that may be specific to online teaching and learning. Further research with experts in cognitive science, online teaching and learning, educational development, and andragogy may bring evidence-based recommendations for how to address troublesome aspects of learning to teach online. This way, educational developers can approach faculty members with stronger strategies. By more effectively supporting faculty in their teaching, we hope to see improved outcomes for students who take online, blended, and technologyenhanced classes.