Effective Tagging Practices for Online Learning Environments : An Exploratory Study of Approach and Accuracy

This exploratory study examines student tagging activity within a five-week social bookmarking unit. Students in six sections of a course were tasked with locating, tagging, and then highlighting and discussing course-related materials using Diigo, a social bookmarking tool. Three different tagging approaches were tested: dictionary only, freestyle only, and dictionary + freestyle. Analysis focused on accuracy, rates of student tagging, and popularity of different tag types. Findings show that most students were able to tag with high rates of accuracy after a single brief lesson. The dictionary-only approach led to fewer tags overall as well as fewer single-use tags than freestyle tagging. It also resulted in students applying useful classes of tags, such as type of content, that did not emerge within the freestyle tag groups’ folksonomies. However, freestyle tagging was not without its merits, and it provided opportunities for students to include tags that reflect relevant interests and more specific topics that were not addressed in the tag dictionary. The combined approach, if carefully taught and applied, appears to have the greatest potential for supporting student information literacy skills.


Effective Tagging Practices for Online Learning Environments: An Exploratory Study of Approach and Accuracy
Online learners generate a tremendous amount of content, whether in the form of personally written messages or shared online resources (Dennen, 2016).Sifting through this information to find topically relevant contributions can be a daunting task.Students suffering from time constraints may struggle to identify the most pertinent readings and discussion questions in a class (Chen, Pedersen, & Murphy, 2011), and learners benefit from some form of guidance in this process (Buder, Schwind, Rudat, & Bodemer, 2015).In order to help learners effectively and efficiently sift through the content generated by an online class, a means of organizing and identifying relevant items is necessary.
Tagging is only effective when the end users are able to efficiently find the information that they seek.Although students may be familiar with the concept of tagging from using hashtags on social networking sites, they may not independently be able to tag effectively in an online learning context.In this study, students were taught how to tag during a five-week resource-sharing and discussion activity hosted in a social bookmarking tool.The purpose of this exploratory study was to examine how students tagged learning resources, focusing on accuracy and rates of tag application, and whether three different tagging approaches (dictionary, freestyle, and dictionary + freestyle) yielded different tagging results.A secondary focus was to examine student perceptions of the utility of tagging while collaboratively building a knowledge base, searching the knowledge base, and discussing relevant resources within a social bookmarking tool.

Background
Social bookmarking applications, such as Diigo, the now-defunct but previously popular del.icio.us,and Pinterest, provide support for sharing links to online resources with others.Beyond link sharing, these tools facilitate the addition of notes and allow users to classify resources.From a user perspective, social bookmarking is an activity that involves locating Internet-based materials, classifying and describing them, and sharing the annotated URL for those materials within a web-based tool for later retrieval by other Internet users.
The classification part of the social bookmarking process uses tags.Tags are a form of descriptive metadata, or data that provide information about an object so it can be effectively stored with similar items and retrieved as needed (Pomerantz, 2015).Tagging is a standard part of the sharing process enabled by social bookmarking tools, although many people are familiar with the concept from popular social media platforms, such as Twitter.Twitter in particular popularized a variant of the practice, called hashtagging.Twitter tags use the hash or pound sign (#) immediately preceding the tag to denote a keyword for searching and aggregation.Other tools, however, tend to just use a word without a symbol.This section addresses both the role that social bookmarking, and tagging more specifically, play in the educational setting.Additionally, it provides information about tag approaches and the relationship of expertise to tagging practices.

Social Bookmarking and Tagging in the Educational Setting
There are multiple ways in which social bookmarking might be implemented in a class setting.Social bookmarking tools may be used to support either individual or collaborative bookmarking processes (Kiu & Lim, 2017).Instructors may lead the activity, using social bookmarking tools to collect and share relevant online resources with their students (e.g., Saeed, Yang, & Sinnappan, 2009;Farwell & Waters, 2010).Alternately, learners may be engaged directly in the process of locating, sharing, and tagging resources (e.g., Im & Dennen, 2013;Colwell & Gregory, 2017).Whether actively sharing or passively consuming, learners have been found to be successful with knowledge building through social bookmarking activities (de Carvalho, Furtado, & Furtado, 2015).
Social bookmarking in the academic context is an underresearched area, leaving uncertainties about its overall utility in education despite various proclamations that it is a useful learning tool and activity (Sera, 2015).In one study, students were found to be apprehensive about using a new tool and engaging in social bookmarking (Farwell & Waters, 2010).Many students in this class, in which the instructor bookmarked and shared various resources with students, ended up having a positive experience.However, they struggled with the concept of tagging.In another study, several students did not participate because they considered it the instructor's responsibility to find and share learning resources (Kear, Jones, Holden, & Kurcher, 2016).An exploratory study of student-led social bookmarking in Diigo, student use of and contributions to the group knowledge base varied between sharing and commenting, and a coherent folksonomy emerged toward the end (Im & Dennen, 2013).Although these studies and others focus on disparate learning activities and outcomes, they demonstrate the position that social bookmarking and tagging activities have come to play in the learning experience.
Tagging serves both individual and collective pedagogical purposes.Individually, tagging activities promote self-regulation within the learning process (Cao, Kovachev, Klamma, Jarke, & Lau, 2015), whereas collaborative tagging activities are useful in classrooms because they promote peer engagement around content artifacts, and the act of tagging requires reflection (Bateman, Brooks, Mccalla, & Brusilovsky, 2007).Additionally, students working in the same topical area can learn from each other's processes and paths when they are documented by tagging activities (Klašnja-Milićević, Vesin, & Ivanović, 2018).In other words, tags provide evidence of what materials others have seen and what sense they made out of these materials.Looking to the future, learner-generated tags, shared in a collaborative system, have been suggested as one means of generating personalized recommendations within an e-learning system (Klašnja-Milićević, Ivanović, Vesin, & Budimac, 2018).

Forms of Tagging
For social bookmarking lessons to succeed, students need to be able to work with tags.Most popular social bookmarking tools use tags that are applied at the document level (Bateman et al., 2007).When a collection of tagged resources is amassed, users can sort and search through those resources using the tags.Most tags are nouns indicating the item's content.However, tags are also used to identify type of material (e.g., blog, article), author, personal judgment or sentiment (e.g., stupid, inspirational), self-reference (e.g., my stuff), and task organization (e.g., to read, job search) (Golder & Huberman, 2006).Tags that are general in their approach to classification tend to have higher rates of entropy than specific ones, the latter appearing less frequently in a tag database (Klašnja-Milićević, Ivanović, et al., 2018).
Tagging approaches may be freestyle, allowing users to generate their own tags and collectively develop a folksonomy, or constrained by a dictionary of predefined tags.Freestyle tagging allows knowledge to be represented in unlimited ways, outside the confines of what authoritative figures determine is useful based on social and cultural biases.In addition, freestyle tagging allows topics to develop over time due to increasing information sophistication, levels of distinction, and social trends (Lin & Chen, 2012).However, freestyle tagging is plagued by synonymy (Golder & Huberman, 2006), where multiple tags having similar meanings (e.g., teacher, teachers, educators, and tutor) are applied, and each synonymous tag is linked to different resources.Misspellings and personal naming conventions in a freestyle system further complicate tag searching.In a random sample of tagged items from two social media sites, Guy and Tonkin (2006) found a high rate of user tagging errors.In another study, tag reuse and differentiation in a learning object repository were found to stagnate over time (Zervas, Sampson, & Pelliccione, 2016).
In contrast, tag dictionaries provide standardized naming conventions and reduce the redundancies common in freestyle tagging.However, the recommended terms in a dictionary system may be insufficient to fully capture the richness and diversity of resources that individuals find and share, or the ways in which they may need to classify them.Expert taggers report using both recommended tags from a dictionary along with additional terms in order to sufficiently categorize resources (Panke & Gaiser, 2009).

Accuracy of Novice Tagging
Tags can be generated and applied by experts, authors, machines, or end users.Some of the issues related to tagging include the question of who should generate tags and whether consumers of content (as opposed to authors or experts) are reliable taggers.The belief that experts are more accurate taggers runs sufficiently deep that there is a patent for a computer-based system that will revise a document's tags based on new tagging information, giving the greatest weight to tags generated by experts (Bagwell & Vasudevan, 2017).Expert practices often are considered the benchmark for examining how other people and entities apply tags.
When authors apply keywords to their manuscripts, as is the norm for many academic publications, they tend to focus on main ideas or themes within their manuscript.In a study of enduser tagging practices, the tags of typical users were found to be fewer and more general than the author-selected keywords (Heckner, Mühlbacher, & Wolff, 2008).Another study showed that expert taggers usually apply seven or fewer tags per item (Panke & Gaiser, 2009).When comparing students and experts, Bateman et al. (2007) found that students created more tags overall as well as more unique tags than experts.At the same time, students did not integrate expert tags when those were provided to them.Similarly, another study found that medical experts used fewer and more direct tags than laypeople when cataloging the same images (Phoebe, Wei-Chung, Kai-Ying, Chia-Chi, & Shing, 2016).These studies suggest that experts are more economical than novices in their tagging practices, and that they draw upon a shared knowledge base.
However, when compared to machines, students and text-mining systems were found to be in agreement two thirds of the time (Bateman et al., 2007).This finding corroborates another study of how participants applied descriptive versus assessment tags (Mamykina et al., 2011).The researchers found that participants were inclined to use whatever kind of tags best matched the vocabulary words that they had at hand, suggesting that for novices the tagging process is driven by contextual cues provided directly in the content being tagged.Finally, when tagging material within the context of a class, student tags were found to be germane and useful (Bagwell & Vasudevan, 2017;Gorissen, van Bruggen, & Jochems, 2015).Furthermore, the tags that they applied to instructional videos aligned with those applied to the same videos by fellow students as well as experts.Collectively, these prior studies suggest that students tag differently from experts but that the difference in this context is not necessarily a problem.

Research Questions
In this exploratory study, we developed and tested a learning activity using Diigo, a social bookmarking tool.The following research questions guided this study: • After brief instruction, how accurately can learners apply tags to shared content within a social bookmarking tool?• How do tag accuracy, rates, and types differ based on tag approach?• What were student impressions of tagging and its utility in the class setting?

Participants and Context
Participants were 99 undergraduate students enrolled in six sections of an educational technology class for preservice teachers during the fall 2016 semester (see Table 1).This class met requirements for both the teacher education curriculum and the university's general education computing course.Enrollment was not restricted by major; students from across the university were free to enroll in the class.However, most students (n = 69; 87%) reported plans to work in education after graduation.Data on student age was not collected, but based on past terms we know that students in this class typically range in age from 18-22.The study was approved by the university's Institutional Review Board.Approval was given to access the full archive of tags and tag use in each class's Diigo group.Additionally, 79 students consented to participate in a survey about their experience in this lesson and gave us permission to look specifically at their tag use (see Table 1).
The course instructors had little to no prior experience using Diigo.Those who had used it had done so in a very unstructured manner in another class.The researchers and instructors had a meeting prior to the implementation of the Diigo unit to prepare the instructors to use Diigo, set up their accounts, and discuss how they would teach it to their students.Instructors were assigned to use one of the three tagging conditions at this time.Common instructional materials (e.g., slides and handouts) about Diigo use and tagging were provided to all instructors.
Each class had a separate Diigo group that was used for the unit.These groups were set up by the authors, who shared moderator status with the course instructors.Once the researchers had the Diigo groups appropriately set up and loaded with examples, they did not enter the space again until the unit had concluded and data were ready to be saved.Instructors helped their students set up Diigo accounts and then invited them to join the group.

Diigo Unit and Tag Dictionary
For the Diigo unit, students were instructed to find high-quality online sources (articles, blogs, etc.) based on the topic of the weekly lesson.Then, they assigned a tag (or tags) to describe and organize each source.During some weeks they further highlighted and discussed these sources.Diigo, as a tool, was not the sole focus of the unit; in addition to developing bookmarking and tagging skills, there were topical learning objectives related to the content students were tagging.
Formal instruction on how to use Diigo was provided to students at the outset of the first lesson.Prior studies have found that university students, regardless of level, generally do not understand social bookmarking and tagging activities (Frisch, Jackson, & Murray, 2013;Neier & Zayer, 2015), and they need instruction both about the underlying purpose of social bookmarking as well as how to do it effectively (Taha, Wood, & Cox, 2016).During the first week, students were taught and tasked with generating bookmarks along with tags and descriptions of each item bookmarked.In subsequent lessons, additional Diigo features were taught and used (see Dennen, Cates, & Bagdy, 2017, for a fuller description of the activity).Tagging was a consistent activity across all five weeks.
The tag dictionary was developed by the researchers based on the topics addressed across the five topical lessons in the Diigo unit.The five topics addressed in these lessons were Academic Software and Apps, Web 2.0, Teacher Productivity Software and Apps, Assistive/Adaptive Technology, and Teacher Professional Development.In addition to topical tags related to each lesson, there were also tags to identify the type of resource being bookmarked (e.g., video or article) and the audience or application for the resource (e.g., elementary).Because Diigo limits the number of tags that can be entered into the preset dictionary, the tag dictionary was provided to students as a linked Google Doc.Students in the dictionary groups had access to this Google Doc and were told to copy and paste or type in the tags from the dictionary when they bookmarked items in Diigo.Students in the freestyle-only group could not access the dictionary.

Data Collection and Analysis
Data collection focused primarily on tag archives from the class Diigo groups.Tag dictionaries were downloaded and cleaned.Specifically, instructor-tagged items that had been done as examples for the students needed to be removed.Unfortunately, a technical error resulted in the inability to collect the archives from one of the classes (Section 6).Students were surveyed at the end of the term about their experience using Diigo and their impressions of tags.The survey was conducted online and contained three demographic questions, two questions about prior experience with Diigo and social bookmarking, and seven items about their experience with Diigo in the class (see Table 7 for these items).Additionally, the researchers kept notes based on feedback received from instructors during the unit, and some of the class sessions were observed.
Data analysis focused on counting tagging rates, comparing tags to the dictionary, and finding tag similarities and differences among the course sections and the ways in which and frequencies with which students selected tags.For survey data, closed items were examined for frequency distributions and central tendency, and open items were examined thematically.

Rates of Tagging
Students generally applied tags to bookmarked items as requested within the assignment.However, as shown in Table 2, several items went untagged in each group.The average number of tags per item ranged from three to five for four of the groups, with only one group (Section 2) having both a lower number of tagged items and lower number of tags applied per item.This instructor also had a small class and reported technical difficulties using Diigo.Similarly, this instructor's students commented on technical problems during the survey.Based on some of those comments, we believe that the technical part of the instruction (e.g., how to log in and use Diigo) may have gone awry within this section.We suspect that both the lower rates of tagged items and tag use may reflect the frustrations that class members experienced.

Tag Accuracy
Tag accuracy was defined as the student's ability to apply a tag without error.To accurately apply a tag required understanding the rules of tagging within Diigo.Accuracy does not, in this instance reflect whether or not the tag reflects the content of the resource to which it was applied.As shown in Table 3, tagging accuracy rates were fairly consistent across groups and also were quite high.Among the errors noted were the following: • adding the hashtag symbol (#) to tags, as is the norm on Twitter; • adding other characters or symbols to tags (e.g., "subject: math" and "elementary_math"); • misspelled tags (e.g., "teaachers" rather than "teachers"; "swoftware" rather than "software"); • separated compound word tags (e.g., creating two separate tags-"professional" and "development"-rather than one "professional development"); and • compound word groupings (e.g., creating a tag with two separate words connected, like "professionaldevelopment").
The last two issues listed above, where two-word tags were either combined as a single word or separated into two different tags, were the most commonly seen errors and account for the bulk of the errors in Section 1.These errors suggest that students failed to grasp the appropriate way to enter a multiple-word tag into Diigo, or that they assumed spaces were not possible within tags.Some platforms, like Twitter, use compound-word groupings in tags.In contrast, Diigo allows users to create multiple-word tags by using quotation marks.Not included in the accuracy count, although seen across every group, are instances of tag synonymy due to the appearance of both singular and plural form.The most common instance, appearing in all of the groups, was "teacher" and "teachers."Although we did not see any personal tags with cryptic meaning, there were a few students in the freestyle conditions who used sentiment tags, such as "fun" and "helpful." During our analysis of bookmarked items, which was a manual process, we noted that no course section or condition stood out as being the best or worst at tagging.None of the students had prior experience with Diigo, and only two reported previous experience with social bookmarking.However, there were standout students, both good and bad, within each course section.Each class had exemplary taggers, who had few or no tagging errors.These same students also applied more thoughtful, meaningful tags, whether in a dictionary or freestyle taggers.At the same time, each class had students who made frequent tagging errors or who continuously neglected to tag the items they shared.This was the case in Section 1, which had the highest rate of errors.

Tag Popularity and Dictionary Use
The groups that had the tag dictionary available to them made use of those tags.Only five of the 59 dictionary tags were not applied across the five lessons (see Table 4).We checked the tags used by the freestyle-only groups as well to see whether they naturally used the same tags as were in the dictionary.Their tags matched 42% of the dictionary tags.Notably, students in the freestyle-only groups did not make heavy use of tags for audience/application, nor were they as likely as dictionary groups to classify their items by type of resource.This finding shows that these students were oriented toward thinking about content when tagging but did not independently consider application or resource type.In contrast, although these are useful tags, students may not intuitively and independently consider using them.Either dictionary support or explicit instruction to develop these types of tags may be necessary to increase their prevalence.Table 5 shows the most frequently used tags by course section and denotes which of those tags were included in the tag dictionary.Section 2 again stands out, this time because it is apparent that learners were not following the directions of their condition.Section 2 had a tag dictionary, but only two of the six most highly applied tags in this section appear in the tag dictionary.Technologically, we were unable to prohibit dictionary-only-condition students from applying freestyle tags in Diigo, and students in this class appear to have not heavily used the dictionary.It is possible that they did not refer to the dictionary at all and may not have been aware of it if their instructor did not introduce it as requested; the two dictionary tags that these students did use may appear simply out of coincidence.
In examining the number of times each tag was used across all course sections, single-use tags accounted for a sizable number of tags overall (see Figure 1) and a notable percentage of tags in all but one section (see Table 6).The single-tag rate ranged from 35-40% in the conditions that included freestyle tagging and also in the section that was assigned to use the dictionary but deviated from it.The one section with fewer single-use tags (Section 1) was assigned to the dictionary condition and, based on a review of their tags, relied on the dictionary for their tagging activities.In observations, that particular instructor was rather emphatic during the instruction that students must make use of the tag dictionary.As can be seen in Table 5 and Figure 1, there were a handful of tags (18 in total) that were applied 10 or more times.

Survey Findings
At the end of the course, students completed a brief survey asking a few questions about their experiences using tags (see Table 7).Across all sections students largely agreed (n = 68; 86%) that tags provide a meaningful way to save links, although 20 students (26%) indicated that they felt tagging was at least somewhat a waste of time.To a lesser degree, students indicated that they carefully chose which tags to apply when sharing an item (n = 61; 77%).Most likely to disagree with this statement were students in the dictionary only sections.Similarly, 60 students (76%) felt that it was easy to determine which tags to use, with the dictionary groups again reporting the greatest disagreement.
When asked about using tags to search through resources shared by classmates, 55 students (70%) reported doing it, and 54 students (72%) felt that tags would help them decide which of their classmates' shared resources they should look at.Finally, 58 students (74%) said that tags in general helped them locate relevant resources.In response to this question, students likely considered items shared and tagged by the instructor in addition to items shared and tagged by classmates.Several students (n = 10; 13%) expressed ambivalence on this item, and once again the dictionary-only group stood out as the only dissenters, with six students (24% of the respondents from this condition) reporting that they disagreed.Note.There were 79 survey participants.Two questions were not completed by all participants (*n = 78, **n = 76).

Discussion
The students in this study, like students in similar studies (e.g., Frisch, Jackson, & Murray, 2013;Wood et al., 2014) were social bookmarking novices at the beginning of the term.Even after instruction and five practice sessions, they would still not be considered experts.The struggles that some students experienced with accurate tagging combined with the application of overly general tags by students in freestyle conditions confirms the need for student training to effectively use tools such as Diigo (Wood et al., 2014).Two approaches have been suggested for improving tag literacy: educating users so they can engage in more effective tagging and developing technologybased systems that better support the tag selection process (Guy & Tonkin, 2006).We believe that both of these approaches are necessary and should work in tandem.Although a sophisticated tool that provides tagging and searching assistance is ideal, students will be faced with situations in their lives where they must make tagging and searching decisions without assistance.
The tagging errors that occurred were not surprising.Researchers in other studies similarly found a tendency for some individuals to use phrases for tags or engage in other inefficient procedures (Mamykina et al., 2011), and sloppy tagging practices, such as misspellings and compound-word issues, in folksonomies (Guy & Tonkin, 2006).This finding confirms the need to teach learners how to generate and apply tags and the rationale for stressing the use of succinct keywords rather than phrases.
We anticipated a large number of single-use tags, particularly in the freestyle tagging sections.Like Guy and Tonkins (2006), we had thought the rate might be even higher when students were engaged in freestyle tagging.The great variance in rates of reuse of tags was not surprising.While overall reuse may be low, certain tags may be reused at high rates (Zervas et al., 2016), and the histogram for overall tag use rates (see Figure 1) mimics the one created by Guy and Tonkins (2006).Single-use tags are not necessarily problematic, especially when individuals are beginning to amass tags in a group knowledge base.However, in order to reduce redundant tagging, students could be encouraged to look through existing tags and reuse them to the extent the tags are relevant.
There are many potential explanations for the range of tagging expertise among the students and the various tagging errors that occurred within the class.Although the students did not have prior experience with Diigo, and all but two indicated no prior experience with social bookmarking, we did not ask about prior experience with classification systems in other tools or settings.Students with this type of experience may have been able to transfer this prior knowledge to the lesson tasks and, as a consequence, may have been more successful.Course attendance also may have played a role.We know from instructor reports and our involvement in the Diigo account approval process that some students were absent during the initial lesson, in which tagging was the main focus, and some did not have an account in Diigo until the second or third lesson.This was not a large number of students; although we do not have a precise number, we would estimate it at 5-10% of the students across all sections.Those students may have struggled more than their peers to identify and apply pertinent tags.
In these findings, we see value in both the dictionary and freestyle approaches.The use of a dictionary helped students select and apply descriptors that ultimately would be useful to another person seeking resources on a technology integration topic.Thinking about the activity as an end user rather than as a tag contributor, it is clear that tags such as "K-5" would provide more useful information than "education."Additionally, the tag dictionary appeared to limit the number of single-use tags within the overall collection.However, the dictionary is only of use if it is well created and contains content-, resource-, and audience-specific tags.The tag dictionary also would function better if it were integrated directly into the tool (i.e., users choose tags by clicking on them rather than typing them in, which introduces potential error).
The freestyle approach is helpful because it allows the users to customize the tagging process and include classifications that are personally meaningful and to tag items that are perhaps tangentially relevant to the overall purpose but nonetheless important.Of note was the inclusion and multiple uses of the tag "disabilities" in Section 2. This tag did not appear in the dictionary, but the six applications of this tag suggest that one or more class members had a specific interest in this area.
The dictionary + freestyle approach appears to combine the best of the other two approaches so long as users work in a dictionary-first manner.In other words, the use of a shared dictionary could help reduce the number of single-use tags based on synonymy and increase the use of classifications, such as type or application of resource.At the same time, freestyle tags could be added to extend beyond the scope of the dictionary and to allow for even more specific tagging.Students in one study found tags more useful when looking for general topics than when searching for resources with great specificity (Sinclair & Cardew-Hall, 2008).Tag dictionaries, while they ideally eschew overly general or self-evident tags (e.g., the tag "education" in the context of an education class) may still be a bit general for users seeking resources on narrow topics.
Student survey results show generally positive sentiments toward tagging, although we note that 20 students (20% of the potential participants) declined to participate in the survey.It was not surprising to learn that students found tag application useful but were not highly interested in looking at the contributions of others within the Diigo group.This finding confirms an earlier study of a student communal knowledge base (Wheeler, Yeomans, & Wheeler, 2008).Additionally, while the assignment given to students in this class did require them to comment on shared resources, it did not put students in a strong position of interdependence for compiling and reviewing shared resources.A student could have completed the lessons by simply choosing a resource in Diigo at random and commenting on it.

Limitations
This study has various limitations, representing a combination of lesson design, research design, and tool factors.The students were not graded on the quality of their tagging or their contributions, which may have resulted in some students being unmotivated to fully engage in the activity.Students would not have received formative feedback on their tagging activities unless they explicitly sought it from their instructor, so students who did not fully understand accurate tagging practices from the initial instruction would not have been corrected unless they observed that their peers were tagging differently.Additionally, instructor lesson delivery and facilitation varied despite providing training and instructional materials.Greater control over this element would be desirable in a follow-up study.
Our inability to collect tag archives from one class section was unfortunate and left us without two examples of classes using the combined Dictionary + Freestyle conditions.Because we neither controlled for instructor nor observed all of the class sessions in which Diigo was taught, we are not confident that all classes received equal instruction on Diigo use and tagging.In particular, we believe that the instruction in Section 2 deviated from the plan because of the technical struggles students reported and the degree to which their tagging activities deviated from the assigned condition.
Diigo proved to have a limitation that we did not anticipate when we designed this study.Specifically, the built-in tag dictionary had a limited capacity; only 20 tags could be added to this dictionary.We had wanted to provide the dictionary tags directly in Diigo for the dictionary condition sections so they could select the tags rather than type them in.Although we tested the dictionary feature prior to study design, we did not test it with the full tag dictionary but rather only the tags needed for Lesson 1.When we realized that Diigo was not going to allow us to build a larger dictionary within the tool, we decided to use a Google Doc to provide the dictionary terms instead.This approach, although functional, made dictionary-based tagging a bit cumbersome.Students needed to have two browser windows open: one to view the dictionary and one to use Diigo, so that they could manually type or copy and paste the tags they wished to use from the dictionary.This workaround may have led some students to dismiss using tags, or to stray from using the tags suggested in the dictionary.

Conclusion
In this study, tagging was embedded within a larger instructional activity.An advantage of this approach was its authenticity.Students were not focused solely on tagging, but rather tagging was integrated alongside other complementary activities, such as searching for relevant resources and discussing those resources with peers.The integrated nature of this activity accounts for the relatively low number of tagged items shared by each class; had students been solely engaged in tagging items, they likely would have tagged more items during their class sessions.They also might have spent more time and effort considering each tag.Although metadata specialists may spend time focused solely on item classification tasks, for most people tagging is usually embedded within the workflow of activities that rely on a medley of information literacy skills.
Findings from this study show that most students can become competent taggers with relatively little instruction.Students benefit from dictionary-based tagging systems, which scaffold the tagging process by suggesting tags that are relevant to the course topic and activities.They also benefit from the ability to generate their own tags.These user-created tags, so long as they are sufficiently meaningful to others, add richness and specificity to the knowledge base and may help other learners identify relevant subtopics within the course.To minimize synonymy, learners should be encouraged to develop and follow tagging conventions within the class (e.g., always use singular or root form of a tag) and to check how others have tagged similar items before creating a new tag.
In closing, we believe that the potential of social bookmarking and tagging has yet to be fully understood or exploited in formal learning contexts.However, these skills are important components of information literacy and are used increasingly in professional settings where large quantities of information are being amassed, evaluated, and shared.These skills also are used by professionals seeking to reach a specific audience through the effective use of tags or metadata.More research is needed to systematically explore the best ways to teach these skills to students, how to fully develop and support these skills through learning activities, and how these skills go on to support lifelong learning.Future studies might compare these types of tag dictionaries under experimental settings and extend their use to alternate online learning activities, such as discussion.

Table 1
Overview of Participation and Demographics by Course Section

Table 4
Overview of Tag Dictionary Use by Tag Type

Table 6
Rates of Single-Use Tags

Table 7
Summary of Survey Responses