Exploring the Factors Associated with Undergraduate Students’ Online Learning Anxiety: Development of the Online Learner Anxiety Scale

The purpose of this research was to explore the factors associated with online learning anxiety by carefully designing, developing, and providing preliminary validity and reliability evidence of a scale to measure undergraduate students’ online learning anxiety. We created a conceptual framework to organize the literature surrounding online learning anxiety and used this framework to develop an initial item pool of 30 items. The researchers recruited N = 297 undergraduate student participants from four public universities in the southeastern United States from whom we collected and analyzed data for descriptive statistics, internal consistency reliability, exploratory factor analysis, and correlational analysis. Following systematic analytic procedures, we arrived at a three-factor model explaining approximately 65% of the variability in these data and retained 24 items in the final model with minimal cross-loadings in the pattern matrix. We labeled the identified factors as (1) online learner feelings of negativity and inadequacy, (2) online learner apprehension towards personal communication, and (3) online learner discomfort with instructor capacity and communication. The final instrument was named the Online Learner Anxiety Scale (OLAS). Scores on the OLAS were correlated with five other measures hypothesized to relate to online learning anxiety thereby providing stronger construct validity evidence. The OLAS was found to produce reliable scores that can be validly inferred as measures of online learning anxiety among undergraduate students in institutions of higher education. These findings are discussed and framed in light of current literature on online learning and possible future research directions.

How do we measure online learning anxiety and why is there a need to focus on measuring this construct in educational research and practice? Prior research studies have endeavored to address various aspects of online learner anxiety. Moreover, the existing body of literature contains various tools for the measurement of aspects of online learning anxiety. However, the majority of these studies lacked a theoretical foundation or a direct approach to exclusively measure this construct (Abdous, 2019;Alibak et al., 2019;Bolliger & Halupa, 2012;Conrad, 2002;Hauser et al., 2012;Heckel & Ringeisen, 2019). With the changing dynamics of the learning ecosystem and the increasing acceptance of online learning, we, as researchers and educational practitioners, must focus on addressing the factors contributing to online learning anxiety among students.
The American Psychological Association (APA) dictionary of Psychology defines anxiety as "[A]n emotion characterized by apprehension and somatic symptoms of tension in which an individual anticipates impending danger, catastrophe, or misfortune…" (APA, 2020a). Anxiety is a common reaction to life events such as taking a final exam or speaking in front of a crowd. It is a prevalent mental health condition, that can also signify a more serious mental health disorders such as generalized anxiety disorder or social anxiety disorder. It is important to differentiate what is "normal" anxiety from what is an anxiety disorder. Severe anxiety and anxiety disorders can lead to other serious mental health issues such as depression, posttraumatic stress disorder, or obsessive-compulsive disorder that can harm people and create adverse effects such as the risk for suicide, school dropouts, and abuse of drugs and alcohol (Nepon et al., 2010). Anxiety is more common issue than an average person assumes. One in every fourteen people around the world has an anxiety disorder, and each year anxiety sufferers spend over $42 billion to address this mental health problem (Devane et al., 2005).
Online learning is an applied and evolving paradigm in contemporary educational research and practice (Benson, 2002;Carliner, 2004;Conrad, 2002;Martin et al., 2020). As an evolving concept, there are multiple features of online learning that can potentially contribute to online learning anxiety among students, such as the newness of technology, feelings of apprehension in an online environment, communication barriers, preparation of the online instructor, etc. These factors of online learning anxiety can impact various aspects of a student's learning process, learning outcomes, and performance. In this fifth generation of distance education (i.e., online learning), the advancement of technology has enabled learning to occur through a wide range of information and communication technologies, such as audio/videoconferencing, Learning Management Systems (LMSs), asynchronous learning platforms, etc. (Moore & Kearsley, 2011). The use of these new online learning systems and technologies affects the level of anxiety experienced by online students (Saadé & Kira, 2009). Students who are new to technology may feel uncomfortable working with these tools, and they can experience higher anxiety as a result (Helms, 2014).
Learners may experience feelings of isolation in their online learning environment and often feel apprehensive about communicating their ideas (Autman & Kelly, 2017). These feelings can upsurge during global emergencies, like the COVID-19 pandemic, when students are forced to switch to the modality of online learning. Lack of familiarity and comfort with the style of instruction and learning is another factor that can create feelings of anxiety within students (Hammond, 2006). All these factors contributing to a learner's anxiety can potentially impact academic performance in an online learning environment. As a general construct, learning anxiety can often negatively impact a student's educational experience and academic success by blocking cognitive processes connected to learning (Slovák & Fitzpatrick, 2015).
Research shows the general trait of anxiety and academic performance have a negative correlation (Hauser et al., 2012). For example, anxiety can reduce a learner's cognitive efficiency, and they often experience thoughts of self-evaluation, self-doubt, and general worry related to their academic performance (Saade et al., 2017). It is critical for the design of online learning experiences to identify and address factors that can intensify learner anxiety so that no student is put at a disadvantage academically due to design choices (Cinquin et al., 2019). Monitoring these factors influencing online learner anxiety can help us generate practical guidelines to reduce the feelings of anxiety in online learning environments. Still, first one must be able to measure online learner anxiety.

Prior Research on Online Learning Anxiety
The existing literature on online learning and learning anxiety in online environments, such as self-paced online training, online exams, synchronous and asynchronous online courses, etc., suggests that online learning anxiety is not a novel construct. Although researchers have attempted to study various aspects of anxiety that occur during online learning experiences, the literature on the assessment of factors influencing online learners' anxiety based on a theoretically grounded framework is fairly limited. Abdous (2019) conducted a study to assess the influence of factors such as demographics, prior experience, satisfaction, and preparedness on the feeling of online learning anxiety among students. The study relies exclusively on a single item measurement for the dependent variable and concludes that factors such as demographics, prior online learning experience, and sense of preparedness impact online students' feelings of anxiety (Abdous, 2019). The reliance in this study on a single item may have resulted in underrated feelings of anxiety reported by students. Building upon this study, we have included items in our scale that represent the feelings of online learning anxiety caused by learner satisfaction and prior experience with online learning. Another study conducted by Bolliger and Halupa (2012) examined the relationship between online course anxiety and learner satisfaction. The study used an 18-item anxiety tool with domains including computer, internet, and online course anxiety. The results of the study show a negative correlation between learner anxiety and satisfaction (Bolliger & Halupa, 2012). However, the anxiety tool design did not take into consideration other domains that can induce anxiety, such as feelings of isolation in an online learning environment or instructor-to-student or student-to-student communication barriers. We have attempted to include these additional domains into our scale to provide an accurate assessment of factors inducing anxiety among online learners. Alibak (2019) presents an online test anxiety inventory (OTAI) scale to monitor test anxiety among online students. The OTAI is an 18-item multidimensional scale comprising three factors: online, psychological, and physiological. The findings of the study indicate that student anxiety during online exams can be related to factors such as working with technology, lack of communication, and the quality of teaching. Although this study provides preliminary validity evidence for the OTAI scale, it does not take into account other known measures related to online learning anxiety and focuses intentionally on online test anxiety (Alibak et al., 2019). Similarly, Conrad (2002) assesses learners' perceptions about online courses, and in this study, the learners responded to the survey with a description of fear and anxiety in starting an online course coupled with statements of apprehension. The factors contributing to the feelings of anxiety discussed in the study include comfort and familiarity, comprehensiveness of the website, and preparation time in advance of the course (Conrad, 2002). We have built upon this study to include these factors in our scale.

Conceptual Framework
We have designed the Online Learner Anxiety Scale (OLAS) guided by these existing research studies and theories. The first step in the process of creating the OLAS was to operationally define online learning anxiety, which includes perceptible expressions of the feelings of anxiety, the situations it is experienced in, and the factors that cause it. We operationally define online learning anxiety as "The feelings of fearfulness, apprehension, and uneasiness that learner experiences in an online learning environment, while interacting with content, instructor, and/or fellow students." There are a couple of important aspects of this definition that we need to consider. First, the definition is connected back to the American Psychological Association's definition of anxiety to include various forms of feelings of anxiety such as fearfulness, apprehension, and uneasiness experienced by the learner. Second, we have carefully defined the factors that cause this feeling of anxiety, such as interacting with content, instructor, and other students in an online learning environment. These dimensions are grounded in the interactions described by Moore (1989).
A learner can experience a range of feelings while interacting with the online learning environment and during the learning process. These feelings, when experienced during the learning process, can impact their cognition and learning outcomes. The Control Value Theory (CVT) provides a conceptual framework for facilitating this work (Pekrun et al., 2007;Pekrun & Stephens, 2012). CVT argues that emotions, such as anxiety, are elicited by human cognition when they are involved in the learning process. CVT assumes that when a learner anticipates failure or low control in their learning environment, they may experience negative emotions, like anxiety. These perceptible expressions of the feelings of anxiety and the factors causing it are derived from the existing literature that is discussed above and form the basis of our item pool for the OLAS.

Feelings of Negativity and Inadequacy
A learner's feelings of negativity and inadequacy in an online learning environment are of particular importance to explore the multidimensional construct of online learning anxiety. A learner's perception of an online learning environment is based on their past experiences, their personality, and on the information that they receive through their interactions within their community. Unfortunately, the perceptions and experiences surrounding online learning are not always favorable among students (Lowenthal et al., 2015). These perceptions and experiences can often create feelings of negativity and inadequacy toward online learning (Abdous, 2019). A study investigating the relationship between learning outcomes and achievement emotions such as pride and anxiety concluded that anxiety was negatively associated with learner self-efficacy and satisfaction in an online learning environment which leads to feelings of negativity and inadequacy (Heckel & Ringeisen, 2019). Similar findings are reported in a Bolliger and Halupa (2012) study that showed a negative correlation between learner anxiety and satisfaction. Other research, including a study by Abdous (2019), shows that demographics, prior online learning experiences, and sense of preparedness impact online students' feelings of anxiety. Building upon this literature and consistent with CVT, we have included items in our scale to monitor experiences of negativity and inadequacy in online learning environments caused by perceptions and past experiences about online learning, feelings of lack of control, feelings of performance failure, and reductions in self-efficacy and satisfaction.

Discomfort with Instructor Capacity and Communication
To understand online learner discomfort with instructor capacity and communication, we build upon the theory of transactional distances by Moore (1993). The theory defines "transactional distance" as "a psychological and communication space to be crossed, a space of potential misunderstanding between the inputs of instructor and those of the learner" (Moore, 1993, p. 23). This transactional distance can increase in an online learning environment based on various factors, resulting in a learner's discomfort with instructor capacity and communication. The theory argues that pedagogy has a greater impact on learning outcomes than the physical or temporal distance between the teacher and student. Many instructors who are asked to teach in an online learning environment may not be prepared for it in terms of the skills and strategy to design and deliver effective online learning experiences (Cutri & Mena, 2020;. These experiences can frame how students perceive their capacity and connection with their online instructors. Hauser (2012) studies this construct by examining the relationships between computer self-efficacy and computer anxiety and their impact on performance in online learning. The study concludes that lower transactional distance (Moore, 1993) is an anxietyreducing mechanism (Hauser et al., 2012). Related to the theory of transactional distance, instructor presence has long been recognized as a critical aspect of online learning and connected to student learning outcomes (Baker, 2010). Building upon the tenets of this theory, we have included items in our scale to identify the impact of the effectiveness of pedagogy and instructorstudent and student-student communication on learner anxiety.

Apprehension towards Personal Communication
Online learners may experience apprehension towards personal communication due to various factors. Where an online environment can provide the comfort of learning in your own space and time, there are various elements that may make a learner feel apprehensive. These elements potentially include fear of communicating with a new acquaintance in an online course. Students who are camera conscious worry about other people seeing them in videoconferencing. Hence, they may feel anxious to express themselves in an online course. A learner's feelings of worry about communicating effectively with other learners in an online course may result in anxiousness. Prior research studies have suggested that the feeling of apprehension in personal communication can lead online learners to develop feelings of isolation or hopelessness from online courses that are not properly designed and facilitated to build presence and community among the instructor and students (McInnerney & Roberts, 2004;Zembylas, 2008). As discussed in the theory of transactional distance, the application of effective pedagogy can reduce the "transactional distance," but if the pedagogy in the online course does not provide opportunities for learners to express and communicate and does not create a sense of community among the learners it can adversely impact their experiences causing feelings of negativity and apprehension and hence increasing the "transactional distance." Thus, feelings of apprehension towards personal communication can potentially induce anxiety among online learners, and consequently, we have included various items in our pool as an expression of such feelings.
The OLAS presented in this study is built upon our operational definition and factors possibly leading to online learning anxiety, as identified in previous studies. We built upon existing theory, including the Control Value Theory (CVT) and Theory of Transactional Distances, as well as relevant studies on online learning and anxiety to develop an initial item pool of 30 items for the OLAS. Our item pool includes statements representing factors such as feelings of isolation, feelings of apprehension, self-doubt in interaction with technology, and difficulty in student-student and student-instructor communication, as shown in Appendix A. The design of OLAS is aimed at bridging the research gap by systematically operationalizing and measuring this emerging and possibly evolving construct using a scale that can be used by researchers for low-stakes research and evaluation purposes to assess this potentially multidimensional construct. A conceptual map of the online learner anxiety scale is shown in Figure 1.

Figure 1
Conceptual map of the Online Learner Anxiety Scale (OLAS)

Convergent and Discriminant Validity Evidence
Dating back to the seminal work of Campbell and Fiske (1959), our development and validation procedures for the online learning anxiety scale included the correlation of the multidimensional construct with five factors assessed using the same method (e.g., survey response scales) that we theorized would relate to online learning anxiety among undergraduate students in a specific way. As illustrated in Figure 1, our research plan included five other measures: (1) frequency of technology use, (2) technology self-efficacy, (3) attitudes towards technology, (4) general anxiety, and (5) technology anxiety. Specifically, we anticipated significant inverse relationships with frequency of technology use, technology self-efficacy, and attitudes towards technology, and significant positive relationships with general anxiety and technology anxiety. Our intentions were to provide stronger evidence of construct validity as operationalized by both convergent and discriminant validity evidence of the OLAS.

Purpose and Research Questions
The purpose of this research was to design and develop a scale to measure undergraduate students' online learning anxiety supported by initial validity and reliability evidence. The guiding research questions were: 1.
What are the factors and relationships among those factors associated with online learning anxiety among undergraduate students in higher education? 2.
What evidence of construct validity exists for scores from the OLAS in relation to hypothesized measures expected to relate to online learning anxiety?

Method Participants
Two-hundred ninety-seven participants (N = 297) were recruited from four different public universities in the southeastern United States. Sixty-four percent of the sample were female participants, 33% were male, seven indicated non-binary/third-gender/other, and two preferred not to answer the gender item on the survey. More than 80% of the undergraduate student participants reported their age within the range of 18 to 22, indicating a sample of mostly traditional undergraduate students. The participants represented a range of racial diversity with 63% classified as White, 15% classified as Hispanic, 6% classified as Black, 11% classified as Asian, and the remaining indicating Other or a preferred non-response. The participants were from a variety of academic majors, including education, computer science, business administration, mathematics, and more.

Online Learner Anxiety Scale
The OLAS was developed from our conceptual framework, which structures the research from the online learning community and prior attempts to measure similar constructs in the context of online learning. The team consisted of three university professors within the realm of educational technology, and one educational technology doctoral student. Following a systematic procedure, the research team developed an initial item pool of 30 items relating to the anxious feelings an undergraduate student might have towards different aspects of online learning. Our conceptual framework served as a blueprint for the item writing, which is a necessary step to establish content validity. Each item intentionally included words to express anxious feelings. We attempted to incorporate items employed in similar research endeavors, such as a study that provided a scale to measure online learning text anxiety (Alibak et al., 2019) or other attempts to measure anxiety in the context of online learning research (Bolliger & Halupa, 2012;Hauser et al., 2012). The full list of items from the initial item pool can be gleaned in the Appendix A. The items made statements about a student's anxious feelings towards aspects of online learning, such as communication with the instructor (e.g., "Lack of student-instructor communication in an online course is stressful for me") or peers (e.g., "I worry if I can communicate effectively with other learners in an online course") in the online course, and general negative personal feelings about online learning (e.g., "Online courses scare me."). The initial item pool of 30 items was intended to capture the anxieties associated with online learning experiences that have been documented in the online learning research literature. As the purpose of this study is to examine the structure of internal and external validity evidence of the OLAS, the results provide the measurement details associated with the final items retained and factors identified from the analyses.

Technology Self-Efficacy
The technology self-efficacy measure contains 19 items related to common tasks an individual would perform using technology (e.g., Delete a computer document or file). The scale uses a four-point response set that relates to self-efficacy: 1. I don't know what this means, 2. I know what this means but I cannot do it, 3. I can do this with help from someone, and 4. I can do this very well by myself. The respondents are asked to rate how well they can do each of these tasks using this response scale. The technology self-efficacy has been used in prior works that reported various pieces of validity and reliability evidence (Hohlfeld et al., 2010;Hohlfeld et al., 2013;Huggins et al., 2014;Ritzhaupt et al., 2013) and demonstrated acceptable internal consistency for these data with a Cronbach's α = .78. We hypothesized an inverse relationship between technology self-efficacy and the factors on the OLAS.

Frequency of Technology Use
The frequency of technology use measure has ten items stated as questions (e.g., "The Internet to look up information about people, things, or ideas?") in which the respondents are asked how frequently they use technology to address the question. The frequency of technology use measure uses a standard five-point scale with the following response set: 5. Almost every day, 4. A few times each week, 3. Between once a week and once a month, 2. Less than once a month, and 1. Never. The frequency of technology use measure has been used in prior research that has reported various pieces of validity and reliability evidence (Hohlfeld et al., 2010;Hohlfeld et al., 2013;Huggins et al., 2014;Ritzhaupt et al., 2013) and demonstrated data with a moderate degree of internal consistency at α = .63. We expected the frequency of technology use measure to negatively relate to the factors on the OLAS.

Attitudes Towards Technology
The attitudes towards technology measure contains five positively stated items about an individual's attitudes (e.g., Using a technology helps me with my work) and asks the respondents to indicate the extent to which they agree with each of the five statements using a standard fivepoint Likert scale: 5. Strongly agree, 4. Agree, 3. Neither Agree nor Disagree, 2. Disagree, and 1. Strongly disagree. The attitudes towards technology scale has been used in prior research that reported various pieces of validity and reliability evidence (Hohlfeld et al., 2010;Hohlfeld et al., 2013;Huggins et al., 2014;Ritzhaupt et al., 2013), and demonstrated data with an acceptable level of internal consistency at α = .74. We expected the attitudes towards technology measure to inversely correlate with the factors associated with an undergraduate students' online learning anxiety.

Abbreviated Technology Anxiety Scale
The technology anxiety measure was operationalized using the Abbreviated Technology Anxiety Scale (ATAS), an 11-item scale designed to assess an individual's level of technology anxiety. The ATAS asks respondents to indicate their level of agreement with negatively stated sentences about technology (e.g., "I feel technology complicates simple tasks") using a standard five-point Likert scale of agreement: 5. Strongly agree, 4. Agree, 3. Neither Agree nor Disagree, 2. Disagree, and 1. Strongly disagree. The ATAS has been previously evaluated in prior research and was deemed appropriate for research and low-stakes evaluation purposes based on multiple pieces of validity evidence (Madley et al., 2015). The ATAS demonstrated a high level of internal consistency for these data with a Cronbach's α = .91. We projected that online learning anxiety would positively relate to technology anxiety as measured by the ATAS.

Generalized Anxiety Disorder scale
The general anxiety measure employed in the present study includes seven unique items designed to gauge an individual's general anxiety at a given point in time. The scale is referred to as the Generalized Anxiety Disorder scale (GAD-7) and asks participants to indicate how frequently they have experienced statements related to anxious feelings (e.g., "Feeling nervous, anxious, or on edge") in the past two weeks. The GAD-7 uses a four-point scale of frequency: 4. Nearly every day, 3. Over half the days, 2. Several days, and 1. Not at all sure. The GAD-7 has been evaluated in prior research as a measure of general anxiety (Spitzer et al., 2006). The scale resulted in a high degree of internal consistency for these data at α = .92. We hypothesized a positive relationship between general anxiety and online learning anxiety.

Data Collection
We recruited participants by reaching out to our professional network of colleagues in institutions of higher education in the southeastern United States. After making prior arrangements with course instructors and securing IRB approval, the OLAS was released to an audience of undergraduate students enrolled in four public institutions of higher education in the southeastern United States. All of the students were currently enrolled in online courses as these data were collected during the COVID-19 pandemic. Since the survey was anonymous, course instructors were encouraged to share the survey with other instructors at their institutions of higher education presently teaching online courses-a snowball sampling approach. The online battery of measures was accessible for a 3-week period, and during this time two reminder emails or notifications were sent out to all course instructors who agreed to invite their students to complete the study. Since the battery of measures administration was anonymous, exact response rates cannot be determined for these data. The battery of measures was administered using the Qualtrics online survey platform with automated reminders for missed items before advancing in the survey and took participants an average of approximately 10 to 15 minutes to complete. Only participants that completed all of the items were retained because of our data analysis plan. A total of N = 369 participants had at least opened the informed consent page. After removing non-responses, our total dataset had N = 297 participants retained.

Data Analysis
These data were subjected to a variety of analyses, including descriptive statistics analysis, internal consistency analysis, exploratory factor analysis (EFA), and correlation analysis (i.e., Pearson r correlations among factors and hypothesized variables). Since this was the first draft of the OLAS and there is uncertainty about the factor structure, EFA was an appropriate choice to examine the structure of the items in relation to our conceptual framework. Specifically, EFA was conducted to explore the underlying structure of the data collected using the OLAS and to assist the researcher team in providing meaningful labels to the factors on the OLAS, thus providing internal validity evidence. Factors were ultimately labeled by carefully studying the pattern matrix, the contents of each item, and the review of theory and literature that inspired our conceptual framework. Factors were formed into composite variables using the arithmetic average of the responses to the items forming the construct. Descriptive statistics analysis was conducted to examine the patterns in this cross-sectional dataset, and to characterize the various factors underlying the OLAS data. Internal consistency for the data was examined with Cronbach's alpha. Correlation analyses were employed to examine the internal structure of the measures and to relate the resulting factors to other hypothesized variables expected to relate to online learning anxiety for both convergent and discriminant validity evidence. The underlying assumptions of the various statistical methods were evaluated. All quantitative analyses were conducted using SPSS version 26. An alpha level of .05 was used for all statistical tests.

Sampling Adequacy
The initial item pool for the OLAS included 30 unique items generated by the research team using the conceptual framework. The descriptive statistics for these 30 items can be found in Appendix A for review. Bartlett's test of sphericity for these data had a chi-square of 5,330.29 (p < .001), which suggested the intercorrelation matrix contained adequate common variance. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.961, which is above the 0.50 recommended limit (Kaiser 1974). The participant-to-item ratio for the data was approximate ~10:1. This participant-to-item ratio is consistent with the 10:1 ratio suggested by Kerlinger (1974), which is more than adequate based on prior research about maintaining factor stability (Arrindell & Van der Ende 1985;de Winter, Dodou, & Wieringa, 2009;Guadagnoli & Velicer, 1988). The skewness and kurtosis for each of the individual items were within acceptable ranges at -1 to 1, and -2 to 2, respectively, suggesting no severe departures from univariate normality. Thus, these data appeared to be well suited for EFA.

Exploratory Factor Analysis
The EFA models were executed using principal axis factoring and an oblique (promax) rotation, as the factors were anticipated to be represent the larger construct of online learning anxiety. The number of factors retained was based on the Kaiser criterion (Eigenvalue > 1) and inspection of the Scree plots generated for the EFA models. Items were assigned to factors based on the greatest coefficients in the pattern matrix. The EFA data from the initial unconstrained model showed three factors, and data were extracted in five iterations. The data from the initial model did not exhibit a truly simple structure in the pattern matrix with cross-loadings on items 5, 13, 14, 26, 29, and 30. After careful review of these items, we decided to remove them from the EFA model and run a new EFA model in an attempt to secure a simple structure in the pattern matrix. The resulting model showed a far simpler pattern matrix structure with only one mild cross-loading from these data. After the EFA model and removal of items, 24 items were retained in the final OLAS. Appendix B shows the pattern matrix, which was the final model adopted for this research. Table 1 provides the factors identified from the EFA model, the associated Eigenvalues, the cumulative percent of variance explained, Cronbach's alphas, and the number of items. As can be gleaned, the three-factor EFA model explains approximately 65% of the variability in these data. Also notable is that the Cronbach's alpha for each factor is quite high, suggesting internal consistency of the data comprising each factor. The three factors identified were carefully labeled based on the essence of the items within each factor, the pattern matrix, and the literature and theory that inspired our conceptual framework, which resulted in the following factor names: (1) Online learner feelings of negativity and inadequacy, (2) Online learner apprehension towards personal communication, and (3) Online learner discomfort with instructor capacity and communication.  Table 2 provides the descriptive statistics associated with each of the factors. As can be gleaned, when we computed the average item response across the items within each factor, we found the largest average to be (3) Online learner discomfort with instructor capacity and communication, followed by (1) Online learner feelings of negativity and inadequacy, and (2) Online learner apprehension towards personal communication.
In a quick review of the skewness and kurtosis coefficients for each factor, there does not appear to be any severe departures from univariate normality for the factors, meeting the assumption of Pearson's correlation. Table 3 provides the Pearson correlation coefficients among the three factors, which show strong, positive, and statistically significant correlations among the three factors on the OLAS.

Correlational Analysis with Other Measures
While the findings reviewed show sufficient evidence of internal validity for the OLAS, our second research question posed the issue of external validity evidence by correlating the scores of the OLAS with other known measures we hypothesized would relate to the identified constructs, or convergent validity evidence. Specifically, we provide the correlation coefficients between the OLAS factors and the following technology and anxiety constructs: (1) frequency of technology use, (2) technology self-efficacy, (3) attitudes towards technology, (4) general anxiety, and (5) technology anxiety. We provide these relationships in Table 4 to provide the external validity evidence of scores on the OLAS for the target population of undergraduate students. As we hypothesized, both technology self-efficacy and attitudes towards technology are inversely and significantly related to the three factors from the OLAS. However, the construct of frequency of technology use has little to no relationship with the OLAS. Additional evidence is shown by the positive and significant relationships among the three factors of the OLAS and both general anxiety and technology anxiety.

Limitations and Delimitations
Interpretation of these results should be viewed within the limitations and delimitations of the current study. As our purpose was to explore online learning anxiety among undergraduate students by developing a scale and to provide initial validity and reliability evidence of this scale, our approach was to first read and review all prior works that had incorporated some aspect of online learning anxiety to inform our approach. Unfortunately, only a handful of prior works have addressed this construct intentionally, so we turned to the general online learning literature to form our conceptual framework to guide the process. It is possible that we missed an important dimension of online learning anxiety in the generation of the initial item pool and our guiding conceptual framework. Our sample of respondents is representative of a range of typical undergraduates within the four universities in which we recruited participants, but before generalizing our findings to a larger audience of undergraduate students, additional data should be collected with the OLAS from other regions of the United States and beyond. Finally, the temporal aspect of this work happening during the COVID-19 pandemic may also have influenced the results since many institutions of higher education had to swiftly pivot to online learning, which we know was not a smooth process for faculty and students alike. Regardless of these considerations, we do believe that we have identified some important findings.

OLAS and Online Learning Considerations
We believe that student impressions of online learning are formed from their personal experiences from prior online learning encounters (e.g., enrolling in a high school course online). Additionally, what students read on the internet about online learning, which in many cases can be limited and conflated with bias and misinformation, may play a role in their impressions. While online learning has seen tremendous growth in the past two decades, the growth has been uneven between different classifications of institutions of higher education, and only 14% of higher education students have enrolled in an online course for their program of study (Allen & Seaman, 2016). While the realities of the COVID-19 pandemic certainly changed these statistics for undergraduate students, we do not believe the experiences during the 2020 calendar year truly exhibit the qualities of effective online learning. Rather the term emergency remote teaching (Hodges et al., 2020) is a better description of what actually happened this past year. However, the typical undergraduate student on the receiving end of this situation likely does not see this distinction, which may have exacerbated their feelings of inadequacy or negativity towards online learning. Years of online learning research have suggested that online learners can develop feelings of isolation or hopelessness from online courses that are not properly designed and facilitated to build presence and community among the instructor and students (McInnerney & Roberts, 2004;Zembylas, 2008). That is, instructors must consider both the instructor-to-student and student-to-student interactions (Moore, 1989) beyond sheer student-tocontent interactions in course design, as students often feel apprehension towards communicating in both synchronous and asynchronous online learning settings. These experiences can also frame how students perceive their capacity and connection with their online instructors. The harsh reality during the COVID-19 pandemic is that many instructors who suddenly had to pivot to an online course format. As such, many were not prepared for the realities of creating and facilitating effective online learning experiences (Cutri & Mena, 2020). Effective online learning requires a breadth of knowledge and skills that is not common knowledge among many instructors in higher education .
The final set of 24 items retained in the OLAS and the three factors identified in the EFA model were: (1) online learner feelings of negativity and inadequacy, (2) online learner apprehension towards personal communication, and (3) online learner discomfort with instructor capacity and communication. We remind our reader that we conducted an exploratory analysis within this study, and thus additional work (e.g., confirmatory factor analysis model on new data) is necessary to provide more concrete validity and reliability evidence of the OLAS. The three-factor model explains approximately 65% of the variance in these data and appears to have univariate normal distributions. As noted, we derived the items from an extant review of literature on online learning which was organized into our conceptual framework to define the contours of online learning anxiety. This conceptual framework ultimately serves as the evidence of content validity of the OLAS and is necessary to document the features used to write the initial items to operationalize online learning anxiety. The results also provide the evidence associated with the internal structure validity evidence of the OLAS data, which appears to be measuring three factors with a degree of internal consistency and strong, positive correlations among the factors identified as demonstrated by the correlation matrix. The strong and positive relationships among the factors of the OLAS suggest a cohesive yet distinct set of constructs manifest undergraduate students' online learning anxiety. Finally, we have provided strong evidence of convergent and divergent validity among the scores from each OLAS factor and other measures hypothesized to relate to online learning anxiety factors. The OLAS appears to reflect the themes that emerged from our extant review of literature organized into our conceptual framework. This alignment suggests a potentially stable theoretical grounding for online learning anxiety among undergraduate students in higher education and suggests the OLAS should next be subjected to a stronger theoretical structure using confirmatory methods.

Relationships Between OLAS and Theorized Related Measures
We anticipated substantive and meaningful relationships among the factors from the OLAS and other known measures related to online learning experiences, or what is sometimes labeled as convergent validity evidence. In the fifth generation of distance education (Moore & Kearsley, 2011), which we aptly call online learning, students use a wide range of information and communication technologies to learn, such as videoconferencing, Learning Management Systems (LMS), asynchronous discussion forums, video, and more. Thus, we anticipated that known measures of technology would meaningfully relate to the construct of online learning anxiety. In the present work, we provided correlational evidence among the OLAS factors and the measures of frequency of technology use, technology self-efficacy, attitudes towards technology, and technology anxiety. Our predictions were that the factors of the OLAS would have inverse relationships with frequency of technology use, technology self-efficacy, attitudes towards technology, and a positive relationship with technology anxiety. These hypotheses were all confirmed with statistically significant relationships except with the frequency of technology use construct, which appeared to have no relationship with online learning anxiety-a form of discriminant validity. We also anticipated that a student's general anxiety would positively relate to their notion of online learning anxiety, which was also confirmed by the significant correlations among the variables. Given that these data were collected during the COVID-19 pandemic, we felt that assessing a student's general anxiety was critical to providing the external validity evidence of the OLAS.
While we do not claim to be the first to identify the factors associated with online learning anxiety, we do believe the OLAS is the first solid and theoretically grounded attempt to operationally define and measure the construct as the primary focus of the research. Other scholars have attempted to describe and measure different aspects of anxiety among online learners (Abdous, 2019;Alibak et al., 2019;Bolliger & Halupa, 2012;Conrad, 2002;Hauser et al., 2012;Heckel & Ringeisen, 2019); however, to the best of our knowledge, none focused on developing a theoretically grounded scale to measure this elusive and increasingly important concept. For instance, Abdous (2019) focused on the notion of online learning anxiety as the primary dependent measure in his study, but only used a single item to measure the construct. Alibak, Talebi, and Neshat-Doost (2019) provided the design and preliminary validity evidence of a scale to measure online learning test anxiety, which did not address many other facets of online learning experiences among undergraduate students. Bolliger and Halupa (2012) related three forms of anxiety to learner satisfaction in online courses: (1) computer anxiety, (2) internet anxiety, and (3) online course anxiety. All of these prior works inspired the present study by incorporating the findings into our conceptual framework and preliminary item pool, but again, none of these studies address the operational definition and measurement of this potential multidimensional construct.

Recommendations for Future Research and Practice
There are myriad opportunities to employ the OLAS in future research involving online learning anxiety, and broadly for online learning experiences among undergraduate students in institutions of higher education. However, first, we believe a natural next step in scale development is to conduct rigorous confirmatory factor analyses on new data from undergraduates from different institutions of higher education. Additionally, as previously mentioned, testing the OLAS further outside a period of a global pandemic will better define how scores may or may not generalize to various contexts in which online learning is occurring. Any future validation studies would provide additional and generalizable evidence of validity and strengthen the conceptual framework and theoretical grounding upon which we based our exploration of the concept of online learning anxiety and initial tool development. Additionally, using the factors in statistical models to detect potential differences among sample demographics and other relevant characteristics (e.g., major) would be a valuable contribution to online learning scholarship. We recommend scholars of online learning administer the OLAS to different populations first within the United States with the intent of gathering additional validity and reliability evidence. Eventually, we would hope the OLAS would be translated into different languages and disseminated to other online learning students across the globe. The OLAS was intended for undergraduate student populations in institutions of higher education. However, the scale may prove promising with slight alterations and adaptations to other student populations, such as K-12 students or students enrolled in graduate programs. Given the highly cohesive nature of the correlations among the three factors of the OLAS, it might be evidence that a second higher-order measurement model would be appropriate, but as noted, we need to first collect additional data and use confirmatory factor analysis methods. In terms of educational practice in online learning, the OLAS may prove to be an important measurement system with the deployment of new online learning programs among undergraduate students. As an early measure in deployments of online learning, the data may assist educators and administrators in planning necessary interventions and supports for their students. An important question we cannot answer in this research is the extent to which online learning anxiety is a malleable construct subject to change with appropriate interventions with students. Again, only future research will help address some of these larger questions.

Closing Remarks
While the online learning research literature has provided some preliminary evidence of the existence of online learning anxiety and some disjointed attempts to assess the construct have been made, we believe this research provides a first and solid attempt at systematically operationalizing and exploring the measurement nature of this emerging and possibly evolving construct by providing a coherent conceptual framework for online learning anxiety based on relevant research in online learning, and an initial 24-item scale for further validation studies. Certainly, the results from the present study suggest the OLAS is measuring a highly internally consistent set of factors that have meaningful relationships with other known indices of online learning among undergraduate students in higher education. We do not claim the OLAS as the only, or even best, option for future research and practice, but it might eventually prove to be a helpful tool and starting point among the online learning community to better understand our students and their anxieties about online learning experiences, which have been exacerbated by the recent COIVID-19 pandemic. We hope this article serves as an impetus for the online learning community to better understand this issue.