Using Word Clouds in Online Discussions to Support Critical Thinking and Engagement

Being actively engaged in a task is often associated with critical thinking. Cultivating critical thinking skills, such as purposefully reflecting and analyzing one’s own thinking, is a major goal of higher education. However, there is a challenge in providing college students opportunities to clearly demonstrate these skills in online courses. This research explores the effectiveness of incorporating word clouds—visual representations of word frequency in a given passage of text—into online discussions. We sought to establish whether implementing word clouds in online discussions would result in a higher incidence of critical thinking and engagement. Survey results from undergraduate participants (n=132) revealed that students analyzing text in word clouds reported moderately higher scores on critical thinking and engagement than students analyzing the text in a linear fashion. A positive relationship was found between critical thinking and engagement, as well as peer interaction. This strategy can be applied to a wide range of educational environments to stimulate critical thinking and engagement.


Introduction
Supporting students to develop critical thinking skills is an overarching goal in higher education (Behar-Horenstein & Niu, 2011), yet it remains among the most nebulous of teaching goals in academia today.It has been variously defined as "the correct assessment of statement" (Ennis, 1962, p. 81), "reasonably reflective thinking that is focused on deciding what to believe or do" (Ennis, 1987, p. 10), "calling into question the assumptions underlying our customary, habitual ways of thinking and acting and then being ready to think and act differently on the basis of this critical questioning" (Brookfield, 1987, p. 1), the ability to identify, assess, and make arguments (Tsui, 2002), and finally as "not only an act of following the rules of logic; it is also both an act and attitude of rebellion" (Waggoner, 2014, p. 40).McLean (2005) notes that some definitions of critical thinking focus on both process and product, while others focus exclusively on one or the other.For purposes of this article, critical thinking is defined as the ability to purposefully reflect and challenge one's own thinking while engaging in tasks that require the evaluation, analysis, and application of previous knowledge.College students with developed critical thinking skills have the ability to evaluate their own arguments as well as the arguments of others, resolve conflicts, and generate well-reasoned resolutions to complex problems (Behar-Horenstein & Niu, 2011).Given that there is an exponential increase in the information and knowledge being generated, it is extremely important that college graduates have the ability to make swift, yet well-informed decisions.Possessing critical thinking skills fulfills the higher education goal of nurturing students to become responsible citizens in a complex society.
Engagement, also a nebulous term, is often described as another desired goal in higher education.In a meta-analysis of educational studies about engagement, Fredricks, Blumenfeld, and Paris (2004) described three types of engagement: behavioral (participating), emotional (valuing), and cognitive (making a concerted effort).Engaging activities tend to activate prior knowledge, foster active investigation, promote interaction, allow for choice, and nurture independent thinking (Jablon & Wilkinson, 2006).For purposes of this study, engagement is defined as the perception of enjoyment, interest, and motivation to complete a given academic task.Previous research has found that engagement is positively related to gains in critical thinking (see Tsui, 2002).In other words, as critical thinking increases, engagement is likely to increase as well.
The rise of online university programs has prompted a serious conversation among educators about the efficacy of online teaching and learning, as there is a challenge in providing students the opportunities to clearly demonstrate critical thinking skills and meaningfully engage in an often asynchronous environment (Joyner, 2012).Asynchronous online discussions are often used to encourage students to think and engage with one other (Comer & Lenaghan 2012).However, mixed findings have been found about the effectiveness of online discussions in stimulating critical thinking.Online discussions rarely exhibit the highest levels of cognitive presence such as connecting ideas and viewpoints, and applying ideas to other contexts (Cleveland-Innes & Campbell 2012;Darabi, Arrastia, Nelson, Cornille, & Liang, 2011;Lee, 2014).In addition, the "limited responsiveness and interactivity found in many online discussions suggests a lack of attention to the ideas of others…many students interpret discussion participation as being more about 'making posts' than engaging in dialog" (Wise, Speer, Marbouti, & Hsiao, 2013, p. 324).It is necessary to conduct classroom research in order to identify specific online discussion strategies that have the ability to support not only a high level of critical thinking, but also more engaging peer interaction.
Words clouds, which are visual representations of a document's text (Kaptein, Hiemstra, & Kamps, 2010), emerge as a potential strategy to support both critical thinking and engagement within the context of online discussions.A word cloud takes the most frequently used words in a particular text and randomly displays them by size, based on their frequencies (DaPaolo & Wilkinson, 2014).Word clouds also display variation in color, typography, and composition, offering an aesthetically pleasing look.Because of this, word clouds are used in a myriad of creative and playful ways by a broad array of users (Viégas, Wattenberg, & Feinberg, 2009).Wordle, a website containing a tool for making word clouds, was launched in June 2008 and over 600,000 word clouds were created within its first nine months.Viegas et al. (2009)  (2014) offer several ways they could be used to assess learning, such as providing a graphical representation of student learning, analyzing papers or writing, and comparing responses.
With word clouds quickly coming into prominence within the academic environment in general, it is important to explore their potential for use in the college classroom.For educators striving to promote critical thinking and engagement in the online classroom, word clouds consequently possess potential as pedagogical tools within asynchronous discussions.Therefore, the focus of this study is to gauge the effectiveness of incorporating word clouds in online discussions in promoting critical thinking and engagement within the context of college classrooms.

Literature Review Online Discussions
Certain strategies within online discussions have been linked to fostering and promoting critical thinking.Szabo and Schwartz (2011) found that the use of online discussion forums as an instructional tool in face-to-face courses improved students' critical thinking skills.Strategies matter, however.For instance, Richardson and Ice (2010) found that students generally scored lower in critical thinking achievement levels in open-ended discussions, even though students reported preferring this format to others.While these results are promising, examining the effects of specific strategies in online discussions remains generally understudied in research.
In an article about future directions for online discussions, Gao, Zhang, and Franklin (2013) recommended that future work should explore the integration of emerging technologies to address the constraints of current environments, the design of multifunctional environments, and the development of appropriate activities for these environments.Studying the impact of word clouds within online discussions serves to fulfill this call.

Word Clouds in the Classroom
Although some teachers who use word clouds in the classroom have published anecdotal evidence about its efficacy in promoting engagement or improving teaching strategies (see Huisman, Miller, & Trinoskey, 2011;Nickel, 2012;Perry, 2012), there is growing empirical evidence regarding the effectiveness of using word clouds in online class settings (Bandeen & Sawin, 2012;Bromley, 2013;Edyburn, 2010) and for qualitative assessment of student performance (DePaolo & Wilkinson, 2014).For example, Baralt, Pennestri, and Selvandin (2011) asked students to create word clouds from their own writing and found that students generated more vocabulary and new grammatical tenses, and student comments exhibited engagement and peer interaction.They recommended word clouds be used to promote reflection and brainstorming, as well as defining main ideas.Word clouds have also been used effectively in other areas of primary research (Ahearn, 2013;McNaught & Lam, 2010).
There is less known concerning the incorporation of word clouds into online discussions.A notable exception is Joyner's (2012) study, which evaluated the use of word clouds in a discussion assignment.Joyner found putting students' posts into word clouds elicited modestly greater evidence of critical thinking, which she identified as sharing underlying thinking, using illustrative examples, and connecting to a framework.In a related study, Hamm (2011) examined students as they created world clouds that displayed their perception of course content and posted them in an online discussion.It was discovered that students engaged in critical reflection of their own work and the work of classmates found the activity fun.Both studies had students analyze their own writing in word cloud form.Little research has been uncovered that concerns word clouds containing other forms of text.
Given that previous preliminary research suggests the association of word clouds with engagement and critical thinking, the purpose of the present study is to analyze the effectiveness of incorporating word clouds in online discussions regarding engagement as well as critical thinking.

Methodology
A mixed-methods approach was used for this research.A quasi-experimental design included the use of a classroom intervention using word clouds as part of a discussion prompt, followed by a survey where students assessed their own critical thinking and engagement experiences.Research was conducted in two sections of an undergraduate anthropology course, Language and Culture, in a large public university in the United States.One section of the course was taught in a blended format during the Fall 2013 semester, and the other was fully online in Spring 2014.Both sections were taught by the same instructor and were identical in terms of course content and materials.The word cloud strategy was piloted by the instructor in the Spring and Summer 2013 semesters, with noticeable success.Once IRB approval was secured to carry out this research study, the assignment prompt was designed and implemented for the Fall 2013 and Spring 2014 semesters.Since this is an initial research investigation of this strategy, the study approach described in this article is exploratory.While the results cannot generate strong generalizable results, they may lay the groundwork for future research.
A sizeable portion of the course relies on graded discussions that take place in small groups of 8-10 students.Students are randomly assigned to these small groups at the beginning of the semester, and complete several graded discussions throughout the term.The word clouds studied here are implemented in Discussion 5 (see Appendix A), which takes place close to the end of the term, after students have had the opportunity to get to know one another and interact in their small groups.Fifty percent of the groups were randomly presented with two famous speeches in the form of word clouds (see Figure 1 and Figure 2) and given a discussion prompt.Students in the word cloud condition were also provided a list of the words present in the speech, and how often they were used.The other 50% were given the same speeches in linear form and presented with an identical prompt (Appendix A).All students were asked to analyze the speeches and speculate about who might have written them, at what point in history, and with what purpose.They were also expected to review particular course concepts and relate them to their analyses of the speeches.Finally, they were asked to reply to someone in their group who had a different interpretation.
All students in the course sections participated in this discussion, but in order to be included in this study, students had to consent to have their data included in the sample.deNoyelles, the instructional designer for this course, physically visited the blended class to recruit students to participate.In the fully online section, deNoyelles posted the study information sheet as an announcement within the class website and emailed the class.The Fall 2013 class had 87 students registered, of which 62 (71% of students enrolled) opted to participate in the study.The Spring 2014 class had 127 students enrolled, of which 70 participants (55% of students enrolled) participated.All participants received a 3% extra credit incentive for participating in the study but were also presented with an alternative extra credit activity if they chose not to participate.
This study took a mixed-methods approach, in that both quantitative and qualitative data was collected for analysis.After completing the discussion activity, all research participants completed a survey (Appendix B) containing 18 closed-ended items and one open-ended item that aimed to measure their perceptions of critical thinking and engagement, as well as peer interaction.Survey items were generated from the various definitions and coding schemes employed in previous articles (see Behar-

Online Learning -Volume 19 Issue 4 -September 2015
Horenstein & Niu, 2011; Yang, Newby, & Bill, 2005).Cronbach's alpha based on standardized items for the critical thinking scale was .70 and engagement was .84,which are acceptable levels for reliability.The quantitative data was analyzed using a t-test to determine if the means of the two groups (word cloud and linear) were significantly different concerning critical thinking.The open-ended item, which simply asked students to provide feedback about the discussion, was analyzed for emerging themes related to critical thinking and engagement and used to complement and expand understanding of the quantitative results.

Findings
A t-test revealed that students analyzing the text in word clouds reported higher scores on critical thinking than students analyzing the same text in a linear form, F (1, 114) = 1.86, p = .176.They also had higher scores on engagement, F(1, 49) = 1.60, p = .211.A positive correlation was found between engagement and critical thinking (r = .63,p < .05).Therefore, if a student found the discussion engaging, it is likely he/she also reported a higher score of critical thinking.A positive correlation was also found between critical thinking and peer interaction (r = .30,p < .05).Seventy-one percent of students in the word cloud condition agreed or somewhat agreed that classmate interaction was promoted through the activity, while only 52% of linear condition students said the same.
There are a couple of explanations for these moderate, though not statistically significant, results.First, the sample was relatively small in size.Second, participants in both groups (word cloud and linear) indicated in the survey that they felt engaged in the discussion activity, in part because it was a change from the traditional question-and-answer discussion activities that had come before in the course.Students in both conditions noted the challenge of guessing the speech authors' identities as engaging.
While statistical inference is a valuable tool in research, it is not the sole determinant of important findings (Hays, 1973).The open-ended comments from the survey serve to illuminate the underlying effect of the word cloud on students' critical thinking and engagement.Themes from the survey comments include the change in thinking, quality of learning, and peer interaction.

Change in Thinking
Students in the word cloud condition identified a change in thinking, which was often associated with a sense of engagement.For instance, one student wrote, "I thought the assignment was definitely more interesting than past discussions.It made me think and analyze material from class in a new, interesting way."Another explained, "I think this discussion format was very helpful for me.It kept me interested and forced me to think differently than I normally would."Analyzing the text in word clouds prompted the act of problem solving.Two students noted that it encouraged them to "think outside of the box," which was associated with overall enjoyment of the task.This change in thinking was also associated with a sense of creativity.One student shared, "I was able to use more of my creativity and look at things more in depth than I would for a regular assignment."This stands in contrast to a student from the linear condition who wrote, "I was simply analyzing a text without a name, just as I would analyze information from a textbook…I don't think it made me think differently than I normally do." While encouraging students to think more freely, the word clouds subtly fostered objective thinking.One student shared, "The one thing I really liked about this assignment was how it forced me to think more objectively."This was noted in another comment, "I think that this is a valuable tool to use as a method of analyzing patterns in speeches from a fresh and objective perspective…."Analyzing the words separate from the full context of the speeches resulted in a more constructive approach.While the reading process was quite different in the word cloud condition compared to the linear condition, one student reported reading more closely, which made it a unique and enjoyable experience.A student reflected, "It definitely forced me to think about the speeches a little bit more.[Word clouds] force you to look at specific words and derive meaning from those individual terms instead of just reading an entire sentence and taking it for what it is."Another student explained further, "Considering the speeches outside of their sentence syntax and concrete cultural context provided for an unusual framework from which to assess the intention and power behind the words themselves.That perspective shift made the assignment more enjoyable and thought-provoking, I think."This association of engagement with critical thinking was often noted in the survey comments.While some in the linear condition also acknowledged this ("I really enjoyed this assignment since we had to deconstruct the speeches and utilize these pieces in order to provide evidence for our claims"), it seems presenting the text only in word clouds may facilitate this process.

Quality of Learning
Students in the word cloud condition often reported an enhancement in the quality of learning, and again related to perceptions of engagement.One student wrote, "This was the most engaged I have felt all semester.It was a more fun way to approach daunting concepts and helped to work through them to a more solid understanding."Bringing in the real-world examples of the speeches helped make the course concepts come alive.A student explained, "I thought this assignment was the most fun from the entire course so far, which is really rewarding because it is also the assignment where I learned the most.It really helped me put into practice the concepts that we are learning, which helps me understand them better."By comparison, some students in the linear condition acknowledged something similar: "I found the assignment more challenging when we were asked to use the textbook to back up our opinions on the speeches.It wasn't impossible, but took more thought processing."However, an element of the learning quality only noted in the word cloud condition was the act of purposeful reflection; "Not only did it require us to apply concepts learned within the class, but it allowed for us to reflect on our own thought processes as we attempted to decode the speeches."

Peer Interaction
A noteworthy difference between the word cloud and linear groups was the frequency in which peer interaction was mentioned in the survey.Five students in the word cloud condition specifically lauded the peer interaction that transpired, as opposed to one in the linear condition.They felt this activity "definitely ignited conversation among my peers and helped us all think more critically."One student found it interesting to "see what cue words were most important for peers, while another reported, "I found this assignment very interesting and enlightening as to how I think compared to my peers."This interaction prompted further ways of thinking about the text.A student reflected, "After I read a different response, it made me ponder this assignment in a completely different way…for me, it added an additional dimension to analyzing other's words."It is not surprising that positive correlation was found between critical thinking and peer interaction (r = .30,p < .05).Comments from four of the five students in the linear condition concerning peer interaction were not positive.One explained, "While some people will actually read through and respond thoughtfully, I feel as though more students will just type anything to get credit."This echoes the sentiment of Wise et al. (2013), who reported that student participation is sometimes perceived to be more about fulfilling the posting requirements rather than engaging in dialogue.It seems the challenge of "decoding" the word cloud provided an effective talking point for students to thoughtfully engage in dialogue.forums, depending on the overall objectives.
The findings implicitly suggest the conditions in which word clouds may be an especially effective strategy in real world use.First, it is an ideal way to encourage open exploration of a concept or topic without being hindered by a large amount of text.Second, it allows the instructor to gauge students' prior and current knowledge, as well as overall approach to the concepts.Most importantly, analyzing text in word cloud forms is especially effective when the process is more important than right-or-wrong answers.In this study, most students guessed the authors of the speeches incorrectly, but did exhibit the critical thinking skills that can be generalized to other tasks.
Based on the survey comments from students and our reflections as practitioners and researchers, there are some recommendations that could improve this particular assignment, as well as other assignments involving word clouds.For example, one student explained, "I don't really understand word clouds, so I didn't really understand the assignment."It is important to clearly articulate the nature of word clouds in general, and the origins of the particular word cloud, before proceeding with an assignment.It is also recommended educators engage students in word cloud analyses as a whole group prior to incorporating it as an individual assignment, as a few students reported difficulty.As one student shares, "It was difficult to analyze just the words without a context of sentences."However, it should be noted that the analysis of word clouds is, and should be, challenging.One student explained, "The word cloud doesn't really give a good context for an analysis of the speeches and makes it quite a bit harder to figure out what is being talked about.I think that the only reason I was able to 'see' what was being talked about was that something just clicked in my brain."Finally, attention should be focused on the word cloud with regards to the location and placement of the words.If the software allows, certain words should be kept together in order to preserve certain phrases crucial for understanding.
Although the present study bears promising results, it is important to note the research is limited in scope and should be considered exploratory.First, the sample size was small in nature.Therefore, it is recommended future studies incorporate a larger sample size of participants.This study included one course, one teacher, one university, and one online discussion.There are nearly countless ways to incorporate word clouds in online discussions, and this study features a single approach.Future research should include involving varied types of classes and college settings, as well as trying out different techniques with the word clouds.For instance, students have the ability to create their own word clouds based on their own writings.The critical thinking that results could be very different from reading other people's words.Also, it is possible that students found the discussion engaging simply because it was a deviation from the norm.It is recommended in future studies to employ this strategy several times to gauge variation in engagement and critical thinking.
For these researchers, one thing is clear: despite the challenges of online teaching and learning, online discussions contain great potential for ways to creatively promote critical thinking and engagement in both online and blended settings, when careful consideration is given to the strategy employed.Purposefully selecting strategies that engage students and enhance critical thinking, such as the incorporation of word clouds, helps educators to cultivate these skills in college students, nurturing these citizens to analyze evidence and make informed decisions in a world of complex problems.

Figure 1 .
Figure 1.First speech in word cloud form.

Figure 2 .
Figure 2. Second speech in word cloud form.

DaPaolo and Wilkinson Online Learning -Volume 19 Issue 4 -September 2015
shares, "Users seem to revel in the possible applications: Scientists wordle genetic functions, fans wordle music videos, teacher wordle literary texts, spouses wordle love letters….Wordles have graced YouTube videos, homemade gifts, posters, scrapbooks, birthday cards and valentines" (1137).Word clouds are also being implemented more in academic spaces.