Examining the Reliability and Validity of a Turkish Version of the Community of Inquiry Survey

The aim of this study was to describe the validity and reliability of a Turkish language version of the CoI survey developed by Arbaugh et al. (2008). Data were obtained from 1150 students enrolled in online courses in various departments in three Turkish state universities. The data were randomly divided into two parts: the first part was subjected to exploratory factor analysis; the second part underwent confirmatory factor analysis. A three-factor structure of the CoI framework explained 75.28% of the variance in the pattern of relationships among the items using the first split-half sample. All three presences had high reliabilities (teaching presence=.965, social presence=.953, and cognitive presence=.972). The three-factor structure of the CoI framework with teaching, social, and cognitive presences confirmed the validity of the Turkish version of the CoI survey.

Cognitive presence-the most challenging type of presence in the CoI framework to study and develop in online courses (Garrison & Arbaugh, 2007)-was defined as "the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse in a critical community of inquiry" (Garrison, Anderson, & Archer, 2001, p. 11).Cognitive presence is expressed as critical thinking and is operationalized via the practical inquiry model (Garrison et al., 2001).The practical inquiry model, shown in Figure 1, consists of four phases: triggering event, exploration, integration, and resolution.These elements reflect the critical thinking process thought to contribute to cognitive presence (Garrison et al., 2001).In this context, cognitive presence is defined as the inquiry process with problem definition, exploration of relevant content and ideas, integration of those ideas in a meaningful structure, and testing the usefulness of outcomes directly or indirectly (Garrison, 2006).

Figure 1. Practical inquiry model
Social presence-the most extensively studied theme among core elements in the CoI framework (Arbaugh, 2007)-is defined as "the ability of learners to project themselves socially and emotionally in a community of inquiry" (Rourke, Anderson, Garrison, & Archer, 2001, p. 51).According to Garrison (2009), however, the concept of social presence has, over time, changed from its original conceptualization.Therefore, Garrison (2009) has updated social presence to mean "the ability of participants to identify with the community (e.g., course of study), communicate purposefully in a trusting environment, and develop interpersonal relationships by way of projecting their individual personalities" (Garrison, 2009, p. 352).
Teaching presence occurs both before and during the course.It is defined as "the design, facilitation, and direction of cognitive and social processes for the purpose of realizing personally meaningful and educationally worthwhile learning outcomes" (Anderson, et al., 2001, p. 5).Anderson et al. (2001) stated that all the participants can contribute to teaching presence in online courses and, therefore, they preferred to use "teaching presence" rather than "teacher presence." According to Garrison et al. (2000), the elements of CoI can enhance or inhibit the quality of educational experience and learning outcomes.Hence, one of the challenges educators face is to implement CoI in the online learning environment.Determining students' CoI perceptions is important for evaluating educational activities presented by some stakeholders such as course designers, program administrators, and instructors.
When studies related to the CoI framework were analyzed, it was initially understood that while CoI was examined in qualitative studies (e.g., Anagnostopoulos, Basmadjian, & McCrory, 2005;Garrison & Cleveland-Innes, 2005;Oriogun, Ravenscroft, & Cook, 2005;Schrire, 2004), the number of studies in which the three main elements (cognitive, social, and teaching presence) in the model were analyzed together increased with the development of surveys which provided opportunity for determining perceptions of CoI.Thus, it has become possible to work relatively efficiently in larger and wider samples and to increase the generalizability of the findings.In this respect, a variety of data collection tools were developed by researchers (Arbaugh, 2007;Arbaugh et al., 2008;Garrison et al., 2004).However, a review of the literature suggests that the CoI survey developed by Arbaugh et al. (2008) is widely accepted.The study conducted by Arbaugh et al. (2008) aimed to develop a valid and reliable CoI survey.The generalizability of the studies carried out through a single institution was limited.In this context in the summer of 2007, 287 graduate students in education and business were reached in four different institutions in the USA and Canada.In scoring the 34-item survey, a rating between 0 (Strongly Disagree) and 4 (Strongly Agree) was used.Analyses confirmed the reliability and validity of CoI conceptual framework consisting of cognitive, social, and teaching presences.
While the Arbaugh et al. (2008) study was conducted solely with online students, the reliability and the validity of this research were also tested with students in blended or online learning environments (e.g., Arbaugh, Bangert, & Cleveland-Innes, 2010;Bangert, 2009).In the evaluation of the survey developed in the original study, a rating between 0 (Strongly Disagree) and 4 (Strongly Agree) was utilized.In other studies, different rating options such as 1-4, 1-5, and 1-6 were used (e.g., Arbaugh et al., 2010;Bangert, 2009;Díaz, Swan, Ice, & Kupczynski, 2010;Shea & Bidjerano, 2009).In addition, some changes were made to the original survey items by different researchers (e.g., Díaz et al., 2010;Shea & Bidjerano, 2009).In one of these CoI survey studies conducted by Shea & Bidjerano (2009), Arbaugh et al.'s (2008) CoI survey was carried out.However, the 12 th item ("The instructor provided feedback that helped me understand my strengths and weaknesses") was a modified restatement of the same item in the original survey ("The instructor provided feedback that helped me understand my strengths and weaknesses regarding the course goals and objectives").When the items in the CoI survey were examined and shared at https://coi.athabascau.ca/,the interactive web site designed for sharing and discussing the research related to the CoI framework ("CoI Survey," 2015), the items reflected those used in the Shea and Bidjerano (2009) research.
In another study conducted by Díaz et al. (2010), a CoI survey developed by Arbaugh et al. (2008) was used; however, the 12 th item ("The instructor provided feedback that helped me understand my strengths and weaknesses") and the 28 th item ("Discussing course content with my classmates was valuable in helping me appreciate different perspectives") were changed from the original survey.The results of the study also confirmed the factor structure of the CoI framework.Kozan and Richardson (2014) examined the factor structure of the adapted form of the survey developed by Arbaugh et al. (2008).In this context, in a study conducted by Díaz et al. (2010), the CoI survey was used to determine students' perceptions of cognitive, social, and teaching presences.With a high degree of reliability, the result confirmed a three-factor structure of the CoI framework.
In a study on the determination of student perceptions of CoI by Olpak, Yagci, & Basarmak (2016), it was mentioned that the survey developed by Arbaugh et al. (2008) had been adapted into different languages such as Korean and Arabic (Alaulamie, 2014;Yu & Richardson, 2015) and was used in different disciplines such as education, business, and health care.In addition, Olpak et al. (2016) stated that the final draft used in the study conducted by Díaz et al. (2010) could be adapted into different languages.Therefore, in the scope of the present research, we aimed to adapt the final draft of the CoI survey used in the Diaz et al. study into Turkish.

Methods Sample
The study was carried out with 1150 students enrolled in online courses in different departments and disciplines (education, nursing, business, engineering, science and math, tourism, etc.) at three state universities in Turkey during the 2015-2016 spring term.Females comprised 60.78% of the participant sample and 39.22% of the sample was male.The average age of a survey respondent was 19.71; ages of participants across the sample ranged from 17 to 41.

Data Collection Tools
Data was collected using the CoI survey which was developed by Arbaugh et al. (2008) and used in the study conducted by Díaz et al. (2010).An individual information form (gender, age, and department etc.) was also used.First, the authors translated the survey into Turkish.Then, two experts in Turkish provided feedback about the accuracy of the Turkish translation.In response to the feedback, edits were made, and an expert evaluation form was prepared.This evaluation form was used to gather expert opinion related to the appropriateness of the items in the Turkish form of the CoI survey.The experts' evaluations dealt with the appropriateness of the items in terms of scope and Turkish culture; it utilized a 3-point Likert scale (1=Not appropriate, 2=Moderate, 3=Completely appropriate).An explanation column was added to each item to encourage the experts to provide additional comments if necessary.When the data obtained from the seven expert evaluation forms were considered, it was decided that the items with an item average score of 2.50 or higher would be appropriate to be included under the relevant factor; the items with an item average score lower than 1.50 would be excluded from the related factor, and the other items would be corrected according to the recommendations of the experts.Per the feedback of the experts, the Turkish survey consisting of 34 items was finalized.The items in the finalized Turkish survey were given to the three English language experts who "back translated" (a procedure according to which a translator or team of professional translators interpret a document previously translated into another language back to the original language) the survey items into English again.After the consistency between Turkish and English forms was confirmed, the study was conducted.

Data Analysis
The total sample (n=1150) was randomly divided into two equal halves by using the Statistical Package for the Social Sciences (SPSS, version 20).Exploratory factor analysis (EFA) was performed on the first sample (n=575) and confirmatory factor analysis (CFA) was performed on the second sample (n=575).Also, the internal consistency reliability was tested by using Cronbach's α for each competency.In such studies, the sample size significantly influences the number of factors that will arise from the analysis.Various opinions exist about sample size.Kass and Tinsley (1979) recommended 5 to 10 participants per item; Nunnally (1978) recommended at least 10 or more respondents per item, and Comrey and Lee (1992) claimed that a sample size of 200 was fair and a sample size of 300 was good.Based on these figures, it was concluded that the number of participants in each sample in this study was sufficient.

Exploratory Factor Analysis (EFA)
EFA is a statistical method which is used to increase the reliability of a survey by removing inappropriate items.The method also identifies the dimensionality of constructs by examining relations between items and factors when the information of the dimensionality is limited (Netemeyer, Bearden, & Sharma, 2003).To describe the factor pattern of the CoI survey, an EFA was conducted on the 34 items with varimax rotation using SPSS 20.The Kaiser-Meyer-Olkin measure confirmed the sampling adequacy for the analysis, KMO = .983. Bartlett's Test of Sphericity, χ2 (561) = 22928.265,p < .000indicated that correlations between items were sufficiently large for the EFA.Two factors emerged with an eigenvalue of 1 over 34 items.The contribution of these factors to the total variance is 72.441%.However, it was decided that the analysis should be repeated with three factors to be consistent with the number of factors expected in the theoretical structure determined during the development of the survey.
In the repeated analysis for the three factors, the total contribution of the factors to the total variance was 27.187% for the first factor, 25.424% for the second factor, and 22.665% for the third factor.The total contribution of these factors to the variance is 75.277%.The three factors above the 1 eigenvalue are shown in the Scree Plot in Figure 2. The factor pattern and the factor loadings of the items obtained as the result of the repeated analysis as three factors are given in Table 2.The bold numbers shown in Table 2 represent the specific CoI survey items that comprise each of the three factors emerging from the exploratory analysis.

Reliability Analysis
An item analysis was conducted to test the reliability of each presence as well as the overall CoI survey.Cronbach's α yielded internal consistencies equal to .965 for teaching presence, .953for social presence, and .972for cognitive presence.In addition, Cronbach's α for the instrument overall was .984.According to Blunch (2008), if the α value is over .70, it shows the internal consistency is acceptable and if the α value is over .90, the internal consistency is excellent.In this context, the α values of the Turkish CoI survey are excellent.

Confirmatory Factor Analysis (CFA)
Following the EFA, a CFA was conducted.The CFA tests whether a previously defined and constrained structure is confirmed as a model.The CFA is also used to confirm a conceptual structure or model (Maruyama, 1998).In this context, the CFA is used for predictive validity (Floyd & Widaman, 1995;Kline, 2005) and is useful for the development, regulation, and re-examination of measurement tools (Floyd & Widaman, 1995).
The CFA was conducted on the second subsample (n=575) using Lisrel 8.80 to test the stability and reproducibility of the three-factor CoI model produced by the EFA.Fit indices were calculated for the three-factor model.However, since the appropriateness and effectiveness of indices depend on sample size, estimation procedure, model complexity, and/or violation of the underlying assumptions of multivariate normality and variable independence, no clear consensus emerged regarding model fit (Byrne, 2010).However, Root Mean Square Error of Approximation (RMSEA), the Comparative Fit Index (CFI) and the Non-Normed Fit Index (NNFI) values provide optimal information by benefitting from the studies made by the different researchers in the study conducted by Bangert (2009).When fit indices are calculated as the result of CFA with the limit values, CFI and NNFI values have excellent conformity, RMSEA values have weak conformity, and χ2/df values do not have good conformity (χ2 = 2968.10,df = 524, p = .00,RMSEA = .090,CFI = .98,NNFI = .98).
When we consider the modification indices related to the analysis results, a considerable relationship can be determined between the error covariance of item 17 and item 16 in the social presence factor and between item 1 and item 2 in the teaching presence factor.Considering roughly the same measurement of properties, one might consider subtracting an item from the item pairs.However, in accordance with the experts' opinions, it was decided that it would be more appropriate to examine the structure examined with EFA for validity by adding the high error correlations observed between the items into the model.
When the fit indices calculated in the second CFA result are compared with the limit values, CFI and NNFI values demonstrate excellent conformity; RMSEA values have weak conformity, and χ2/df values have conformity at an intermediate level (χ2 = 2539.14,df = 522, p = .00,RMSEA = .082,CFI = .99,NNFI = .99).When the other fit indices are compared with the limit values, Root Mean Square Residual (RMR) values indicate a good fit while Standardized RMR and Incremental Fit Index (IFI) values indicate a perfect fit (RMR = .063,SRMR = .039,IFI = .99).The obtained t values for the factor loadings ranged from 19.74 to 26.86, which indicate that all items were significant at p < .001.When the results related to the CFA given in Figure 3 were examined, the factor loadings of the items ranged from 0.72 to 0.88.Finally, the results of the CFA confirmed that the model fit between the proposed model and the observed data is excellent.

Discussion
The purpose of this research was to test the validity and reliability of a Turkish version of the CoI survey.Participants' responses were analyzed using EFA to identify the underlying dimensions assessed by the CoI survey; CFA was used to test the fit of the hypothesized model against the model predicted to exist in the population.Results from the EFA identified two-rather than three-factors for the CoI survey.When EFA is repeated with three factors to be convenient with the CoI model, all items were loaded significantly.The three factor model identified by the EFA was then subjected to a CFA with a second subsample of participants.When the modification indices related to the analysis results are examined, there is a strong relationship between the error covariance of two each items involved in teaching presence and social presence factors.It was decided that it would be appropriate to test the high error correlations observed between the pair of items by adding them to the model; the results of the second CFA confirm that the model is a good fit.Cronbach's α values calculated for the Turkish form change between .95 and .97.Cronbach's α value calculated for the whole survey is .98.As in the original and Turkish versions of the CoI survey, there are 34 items in total: 13 of them related to teaching presence, 9 items related to social presence and 12 items related to cognitive presence.Regarding many analyses of the CoI survey, the Cronbach's α values calculated for internal consistency in Arbaugh et al. (2008), Díaz et al. (2010), Kozan & Richardson (2014) and the Turkish version of the CoI survey are given in Table 3.When the results of the current study are compared to other studies in which the CoI survey was translated into other languages, the CoI survey developed by Arbaugh et al. (2008) emerges as the most suitable for a Turkish language translation.The results of factor analysis in an Arabic adaptation study conducted by Alaulamie (2014) showed that items were loaded appropriately in the expected factors.Only the 24 th item in the instrument had a cross-loading issue.Therefore, it was suggested that this item could be dropped in future studies or new validity and reliabilty studies could be conducted.In another study conducted by Yu & Richardson (2015), the adaptation of a Korean language survey showed that two items were omitted from the survey: the 4 th item in the teaching presence category ("The instructor clearly communicated important due dates/time frames for learning activities") and the 14 th item in the social presence category ("Getting to know other course participants gave me a sense of belonging in the course").Additionally, the internal consistency coefficients (teaching presence=.95,social presence=.91,cognitive presence=.96and total=.97) in 32 items of the Korean version of the CoI survey confirmed the structure of the CoI framework with three factors.

Conclusions
EFA and CFA results conducted during the adaptation of the CoI survey consisting of three factors showed that the conformity of the model with the actual data is at an acceptable level, and the analyses performed related to reliability show that the survey is reliable.A review of the literature revealed that the CoI survey, developed as a result of a pioneering work by Arbaugh et al. (2008), was translated/adapted into different languages such as Korean and Arabic (e.g., Alaulamie, 2014;Yu & Richardson, 2015) and it was used in other disciplines, such as education, management and health care (e.g., Arbaugh, 2013;Arbaugh et al., 2010;Bangert, 2009;Carlon et al., 2012).When studies of the use of the CoI survey are examined in detail, various variables (e.g., student educational level, number of online courses taken by students, characterization of the course as fully online or blended, discipline, gender, and age) are considered.Therefore, new studies can be conducted by considering these variables.In addition, new studies related to the adaptation of the CoI survey into different languages can be planned by considering the changes in survey items and the ratings of the items.Utilization of the Turkish adaptation of the CoI survey by various stakeholders (such as course designers, program administors, and instructors) may provide insights regarding the evaluation of applications in distance education in Turkey.Since the survey is internationally accepted, it can be used as a bridge to compare international studies with national studies and suggestions can be developed for future improvement.Finally, this study only concerned itself with students in online courses.In future studies, administering the CoI survey in blended learning environments and engaging students at different educational levels and in different institutions may contribute to a better understanding of the structure related to the CoI survey.

Figure 2 .
Figure 2. Scree plot for the Turkish version of the CoI survey

Table 2 .
CoI Survey Items and Factor Loadings Table 2 (cont).CoI Survey Items and Factor Loadings

Table 3 .
Cronbach's α Values of the CoI Survey Factors