Development and Validation of the German Version of the Community of Inquiry Survey

The Community of Inquiry (CoI) Framework describes success factors for collaborative online-based learning. The CoI Survey is a validated instrument to measure these factors from the perspective of course participants. Until now, no validated translation of this Survey to German was available. The aim of this work was to translate the original English Survey to German and to validate the translated Survey instrument. After a systematic translation process, we validated the German translation in two higher education settings in two countries (entire data set of n=433 Surveys). By conducting item analysis, reliability analysis, exploratory factor analysis, and confirmatory factor analysis, we were able to confirm the reliability and validity of the German CoI Survey. Only one item (CP6) shows cross-loadings on two factors, a finding that was already discussed for the original CoI Survey. To conclude, the validated German version of the CoI Survey is now available.

Online-based learning environments in higher education offer great flexibility to students but are challenging in fostering cooperative learning (Ferguson, 2012).The Community of Inquiry (CoI) (Garrison et al., 1999) is a conceptual, collaborative-constructivist framework to foster collaborative learning in online learning environments.It was initially developed in the context of computer-mediated asynchronous communication in higher education.
The CoI framework describes three overlapping elements that are seen as crucial success factors for a deep and meaningful educational experience (Figure 1): Cognitive presence is "the extent to which the participants … are able to construct meaning through sustained communication" (Rourke et al., 1999, p. 51/52) Social presence is the "ability of participants … to project their personal characteristics into the community, thereby presenting themselves to the other participants as 'real people'" (Rourke et al., 1999, p.52).Teaching presence includes the "selection, organization, and primary presentation of course content, as well as the design and development of learning activities, assessment, and the facilitation of learning processes" (Rourke et al., 1999, p. 52) The CoI framework has become a "robust guideline" to analyze and improve online-based courses in higher education (Castellanos-Reyes, 2020).

Figure 1
The Community of Inquiry Framework Source: http://thecommunityofinquiry.org/coi;CC-BY-SA) Since the development of the Community of Inquiry framework, two major approaches to measuring these presences have been widely used: manual coding of online discussions and surveys (Stenbom, 2018).Rourke et al. (1999) provided the manual coding schema with different indicators based on the three overall categories to measure and describe social presence, cognitive presence and teaching presence.Since then this procedure has been intensively used to manually code students' postings in various online learning environments (e.g.Kovanovic et al., 2018;Richardson et al., 2017;Richardson & Swan, 2003).Nevertheless, this form of measuring the three presences is time-consuming and it has been shown that inter-rater reliability is partly relatively low, as different coders may assign different indicators (Hughes et al., 2007;Swan & Shih, 2005;Whiteside, 2015).
In 2008, Arbaugh (2008) in conjunction with some of the original CoI authors developed a 34-item instrument, the CoI Survey that allows measuring the three CoI presences in larger online communities across institutions (Arbaugh et al., 2008).The CoI Survey contains 13 items for teaching presence, 12 for cognitive presence, and 9 for social presence.The reliability and validity of this English CoI Survey were demonstrated in various settings and countries, and the CoI Survey was also translated to other languages (Stenbom, 2018).The translated versions of the survey showed good results in terms of reliability and validity, for example in Turkish (Olpak & Kiliç Çakmak, 2018), Korean (Yu & Richardson, 2015) or Portuguese (Moreira et al., 2013).However, a German translation is still missing.In 2017, we, therefore, started to develop and validate a German translation.This paper aims to present the development and validation of this German translation of the CoI Survey instrument.

Development of the German Translation
We developed the German translation of the original CoI Survey in a systematic forward and backward process.First, two academic translators independently translated all items into German.Differences in translations were solved by discussion between the academic translators.The resulting German translation was then back-translated into English by a third bi-lingual and experienced academic translator.Differences between the original CoI items and the backtranslated CoI items were then discussed by a fourth bi-lingual and experienced academic translator and a team member with expertise in educational research and CoI.Differences were resolved by carefully assessing whether the translations matched the intention of the CoI as a collaborative-constructivist framework.In three cases (items TP5, TP6, and SP9, see Table 1), the CoI team at Athabasca University was contacted by e-mail to clarify the specific meaning of the original items, and feedback was considered in the translation.The resulting translation of the CoI instrument was then used in a pilot survey with 16 German-speaking students in an onlinebased course to verify the understandability of the wording of all items.The data collected was not analysed, but the understanding of the questionnaire items by the students was verified and confirmed.
The translation was then used in two settings: at a university in Austria (since 2017) and a university consortium in Germany (since 2019).In both settings, slightly different variants of six items were used to accommodate different organizational and educational settings.In January 2020, the data with the German CoI Survey were analysed and discussed by both partners, carefully considering the original intention of the CoI, and the final consensus translation was agreed on (Table 1).After this date, this consensus CoI Survey was used at all sites.
IRB approval was received by the Research Committee for Scientific and Ethical Questions, 2309/17.

Research Context
Two partners from two German-speaking countries participated in this validation study of the German CoI Survey.The first partner is the Austrian University UMITprivate University for Health Sciences and Health Technology with its fully online master's program in Health Information Management.This master's program's instructional design is firmly based on the Community of Inquiry framework.This post-graduate master's program has a duration of five semesters.The master's program starts annually.Previous student numbers ranged from seven to 20 per cohort.The program consists of 13 online courses, where each course has a typical duration of six weeks.All courses comprise asynchronous e-tivities and written discussions and follow the same instructional guideline.The student groups in the courses usually remain the same, instructors (typically one instructor per course) vary throughout the courses.Moodle is used as a learning management system.The format of e-tivities is used throughout all courses (Salmon, 2013) to provide common structures for all activities and support meaningful discussions.All students are invited to three networking days at the university once a year to promote socialization and team building.
The second partner is the HiGHmeducation Consortium consortium, comprising 12 different universities in Germany that offer study programs in Medical Informatics.This consortium aims to boost Medical Informatics by jointly offering online courses.Students in bachelor's and master's programs in the field of Medical Informatics of the participating universities can voluntarily complete various online courses from different partner universities to further their education and obtain an additional certificate.The cohort size in the courses ranges from six to 41 participants, with an average of 16 students participating in each course.The periods in which the courses take place are aligned with the semester periods of the offering universities so that a course usually takes place over a period of 16 weeks.The courses are conducted according to the instructional design of the HiGHmeducation Consortium which can be characterized by the Community of Inquiry framework, the use of asynchronous e-tivities (Salmon, 2013), and by course phases that carefully introduce participants to the online setting.Within the HiGHmeducation Consortium different learning management systems are used, depending on the university: Moodle, Ilias and Stud.IP.

Participating Students
Overall, 242 students participated in this validation study (Germany: n=171, Austria: n=71).The 71 students from Austria were all participants of the online master's program, although from three different cohorts.The 171 students participating from Germany were all participants attending courses offered by different consortium partner universities.
Demographic data collected were gender and language skills in German.123 (51 %) of the students were female, 105 (43 %) were male, 14 students (6 %) didn't specify.The language skills were relevant because participants with insufficient German language skills would have been excluded from the validation study.A total of 199 (82 %) of the students had German as their native language, but all students were sufficiently fluent in German to follow Germanspeaking courses.
In Austria, students were enrolled in an ongoing master's program that included multiple courses and thus typically completed several CoI surveys, one for each course.In Germany, students mostly attended only one online course and thus completed mostly only one survey.
Overall, the 242 participating students completed 433 CoI surveys (Germany: n=171, Austria n=262).All surveys used were the same German translations.Only in a few items, the translation differed (see Appendix A for details).

Data Collection
All students participating in an online course were invited to fill in the German version of the CoI survey at the end of each course.An online questionnaire was used here, and the access link was sent to the students by e-mail or by personal message within learning management systems.The survey contained the 34 items of the German CoI Survey and used a 5-point Likert scale (Strongly Agree = 5 to Strongly Disagree = 1).Participation was voluntary and anonymous, and it was also possible for students to skip items.Consent forms were obtained from all students at the beginning of their study.

Data Analysis
Overall, 433 complete datasets from 242 students were available for our data analysis.To assess whether the slightly different wording of the survey variants at both partner institutions may influence validation results, we first divided the data set based on the three questionnaire versions: the Austrian version (n=186), the German version (n=86), and-after the final consensus of the translation of all items-the final version (n=161).
An in-depth item analysis as well as an exploratory factor analysis were performed using SPSS 27 (IBM Corp., 2020).
As item analysis and exploratory factor analysis confirmed no differences in the Survey variants, confirmatory factor analysis was subsequently calculated over the entire data set of 433 surveys.
We calculated this sample size as follows: According to Kass & Tinsley (1979), five to ten participants are required per item, which would sum up to a needed sample size of 340 students given the 34 CoI items.Comrey & Lee (1992) suggest that a sample size of 200 is fair and 300 is good.Similarly, other authors also suggest that total sample sizes of N=300 are sufficient (Tabachnick & Fidell, 2007).

Item Analysis
A descriptive item analysis was conducted.Item difficulty, means, and standard deviations, kurtosis of items, discriminatory power, and mean inter-item correlation for the three different survey versions were analyzed.

Item Analysis for Reliability
As a prerequisite for the exploratory factor analysis and to check the internal consistency of the German translation, a reliability analysis of the items was conducted.In addition, we checked whether the items were sufficiently highly correlated (Kaiser-Meyer-Olkin criteria) and whether the missing values in the data sets arose by chance (Little's MCAR test).

Exploratory Factor Analysis (EFA) for Validity
After the in-depth item analysis, exploratory factor analysis was conducted using SPSS 27 (IBM Corp., 2020).Due to the positive results of the initial item analysis of each variant, the whole data set (N=433) was analyzed.EFA is a multivariate method often used in test and questionnaire construction to "identify the common factors that explain the order and structure among measured variables" (Watkins, 2018, p. 220).By EFA and scree plot, MAP test, and parallel analysis, we attempted to identify the German translation's three-factor structure (i.e., teaching, social, and cognitive presence).

Confirmatory Factor Analysis (CFA) for Predictive Validity
Based on the item analysis results and the exploratory factor analysis, a confirmatory factor analysis was conducted using R (R Core Team, 2014) and AMOS (Arbuckle, 2014).In contrast to EFA, CFA is "the foundation of structural equation modeling" (Moore & Brown, 2012) and compares models for their empirical fit to the data (Bühner, 2011).
Different fit indices are available to assess model fit and predictive validity of the item structure.For example, Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI) closer to 1 indicate higher fitting between variance/covariance of the tested model with more restrictive independence model (Schreiber et al., 2006).Standardized root mean squared residual (SRMR) looks at correlation matrices and unlike root mean square error of approximation (RMSEA) does not consider model complexity, so these two should be considered in combination.Cut-off for the SRMR is < .11and RMSEA sample-dependent, for n=>250 sample size in our case an RMSEA cut-off of <.06 (Bühner, 2011).

Descriptive Statistics and Item Analysis Over the Different German Translations
Table 1 presents the results of the in-depth item analysis of the three German CoI variants.Results show no difference in the descriptive analysis for the items independent of the wording used, which could be expected due to minor translation changes.Both the mean inter-item correlation and the reliability analysis support the final German CoI version.
All Likert scale response options were used for all items (min = 1, "strongly disagree," max = 5, "strongly agree"), but the distribution of the items is right-skewed.All students reported high levels of perceived teaching presence, social presence, and cognitive presence over all survey variants (see Table 1).

Exploratory Factor Analysis (EFA) for Validity
EFA was performed on the whole data set (n=433), as the item analysis indicated no differences in survey variants.Keyser-Meyer-Olkin (KMO) yielded .955 of sampling adequacy, implying that EFA should explore distinct and reliable factors with sample data.Barlett's test of sphericity (χ2 (561)=9,805.38, p <.000) indicated that correlations were sufficiently high for the EFA.All MAS (measure of sampling adequacy) coefficients had values higher than 0.80, indicating the suitability of the test characteristic values for factor analysis.
According to Stenbom (2018) most previous authors used principal component analysis (PCA) using oblimin rotation, followed by varimax rotation when validating the Community of Inquiry Survey.As our data was not normally distributed (teaching presence, social presence, and cognitive presence scales were not normally distributed, as assessed by the Shapiro-Wilk test and Kolmogorov-Smirnov test, p < .05)and based on recommendations for factor analysis (Costello & Osborne, 2005;Watkins, 2018), we choose maximum likelihood (ML) extraction and varimax rotation with Kaiser normalization.Here we follow other authors who validated other translations of the CoI Survey (Olpak & Kiliç Çakmak, 2018).Table 2 shows the results.
The scree plot shows the three factors with eigenvalues greater than 1 (Figure 2).Parallel analysis conducted in R suggested three factors for the underlying data.A minimum average partial test Test) was conducted to prove the three-factor structure, confirming three factors.

Figure 2 Scree Plot for the German Version of the Community of Inquiry (CoI) Survey
With the three-factor structure of the German CoI Survey, EFA shows that 60% of the variance in the patterns of the relationship among the items could be explained.The first factor (teaching presence) explains 24%, the second factor (social presence) 18%, and the third factor (cognitive presence) 18% of the variance.
In a sub-analysis, we conducted EFA on the final German CoI (n=161) only.KMO and Bartlett's test of sphericity again proved that the data fit the analysis, and the three-factor structure was confirmed as well.In total, findings were slightly better for this final German CoI.The three factors explained 61% of the variance: The first factor explains 23% (teaching presence), the second 20% (social presence), and the third 18% (cognitive presence).Note.Extraction method: maximum likelihood with varimax rotation (N=433).

Item Analysis for Reliability
All 34 items were analyzed for reliability, first for the three variants and then for the overall data set (Table 3).All items showed very high internal consistencies and reliability in all variants and the final German CoI Survey.Likewise, in comparison with the reliability analyses of the other translations, our results show themselves to be reliable and comparable (Table 4).

Confirmatory Factor Analysis (CFA) for Predictive Validity
Following EFA, we conducted CFA for the whole sample to analyze factor structure using R (R Core Team, 2014) and AMOS (J.Arbuckle, 2014) to visualize the factor structure.Due to the findings of EFA, the item Cognitive Presence 6 was excluded in CFA and the structure of 33 items was analyzed.
CFA yielded a good fit of the model to the sample data (χ2 (492, N=432)=1,505.93,p<.001, CFI=.87,SRMR=.06,RMSEA=.06).Table 3 presents the variance/covariance matrix for the 33 items.Figure 3   We carefully and step-wise translated the items to capture the meaning of the original CoI items in the German translation.For some items, two possible translations were discussed and then tested.The item analysis and reliability analysis showed comparable good results for all variants.Thus, we are now able to present the final, validated German CoI Survey (Table 1).
The German CoI Survey was applied in different university contexts in Austria and Germany, thus reflecting a specific diversity of organizational and educational approaches and confirming its generalizability to different settings.
Nevertheless, certain limitations must be taken into account.In the Austrian sample, students participated in several online courses and thus submitted several CoI Surveys.The data thus may be felt to contain some connected samples.In an analysis of these samples, however, we could see that students did not use typical response patterns when answering the CoI questionnaire for different courses in which they participated, but rather evaluated each course differently.Likewise, there was typically a time delay of several weeks between various courses and the related CoI surveys.We thus considered the data as unconnected, independent samples.We applied three slightly different variants of the survey, which reduced the overall sample size in each group.Our statistical analysis did not show any differences between the groups.Thus, we conducted the exploratory and the confirmatory factor analysis on the whole data set.Here, the sample size (N=433) is sufficiently high.However, we will continue to collect and analyze data from future courses to confirm our findings.For the final German CoI Survey (N=161), CFA was conducted and predicted a perfect model fit.Due to sample size issues, these findings are not reliable and not ready for publication at this time but will be reported and analyzed in further studies.
The analysis of the difficulty index of all 34 items reveals that most students perceived the CoI level as quite good.The distribution of the items used is right-skewed and most students reported high levels of perceived teaching, social, and cognitive presence.While not all previous validations of the CoI Survey presented means and skewness of items analyzed, some authors reported the same findings as we did (e.g.Moreira et al., 2013).Further studies would be needed to investigate whether this result reflects a good CoI in the analyzed online courses or whether aspects of social desirability play a role.
When analyzing the factor loadings of the 34-item structure of the German translation, we found that Item CP6 ("Online discussions were valuable in helping me appreciate different perspectives") showed cross-loadings with the social presence factor.First, we took a closer look at the wording in German, as well as in the original version, but we could not find any conspicuous features.When we looked at the previous validations in different languages, we noticed that this item in particular shows difficulties in some translations (e.g.Velázquez et al., 2019).Likewise, it has been shown that there seem to be cross-loadings for non-native speakers of English in the original version (Kovanović et al., 2018).It should be checked here whether the wording regarding the adoption of different perspectives shows differences in different linguistic customs.The results indicate different interpretations in non-native English speakers, as well as in German and Spanish.

Conclusion
We systematically translated, piloted, and formally validated a German version of the CoI Survey in two countries.We expect that the availability of the CoI Survey in German as well as in other languages will allow the CoI to be further validated and developed from a stronger international point of view.Future research and the application of the German CoI survey should improve the measurement and understanding of the Community of Inquiry framework in German-speaking online learning environments and thus support universities to improve online teaching.Also, in our German translation, we confirm the cross-loading of one item that needs to be investigated in more detail.We plan to continue the confirmatory factor analysis as soon as a larger sample is available, but given the previous results of the validation, the German version of the CoI Survey seems promising and suitable.
presents the standardized loadings, which are all above .60.Highest loadings were found in the items Teaching Presence 5 and 6, Cognitive Presence 2, and Social Presence 6, whereas the lowest loadings were found in the items Teaching Presence 2, Cognitive Presence 4 and Social Presence 2.

Figure 3
Figure 3 Factor Solution for the 33-Item Structure with Standardized Factor Loadings

Table 1
Original CoI items in English Taken from Arbaugh et al. (2021) and the Final German CoI Survey Ich kann das im Kurs entwickelte Wissen imRahmen meiner Arbeit oder bei anderen Aktivitäten außerhalb des Kurses anwenden.

Table 1
Item Analysis of the German Translation of the CoI Survey

Table 2
Factor Loadings of the Community of Inquiry (CoI) Items After Factor Reduction Procedures

Table 3
Cronbach's Alpha for All Variants and the German CoI Survey

Table 4
Reliability of Different Translations of the CoI survey DeclarationThe author(s) declare no conflicts of interest associated with the research in this article.