Examining Interactive and Metacognitive Processes in Student Learning: Findings from a Hybrid Instructional Environment

This study examines the interaction behaviors and metacognitive behaviors of graduate students in the online portion of a flipped classroom. For their time outside the face to face classroom, students were given the choice of two online methods for their interactions -synchronous verbal discussions and asynchronous written discussions. Students were provided a detailed outline for their discussions. Discussions were analyzed and interactive and metacognitive behaviors were categorized and counted. Interaction behaviors and metacognitive behaviors were present in both environments. Synchronous verbal discussions were found to include significantly more interaction behaviors in five of six categories. There was no significant difference in the number of metacognitive behaviors. Students demonstrated the same level of learning behaviors in both environments. Examining Interactive and Metacognitive Processes in Student Learning: Findings from a Hybrid Instructional Environment Feeney, L. & Meyers, S. (2016). Examining Interactive and Metacognitive Processes in Student Learning: Findings from a Hybrid Instructional Environment. Online Learning 20 (3). Introduction In 2006, the U.S. Department of Education conducted a meta-analysis of online learning studies to determine if any trends were evident. This report was updated in 2010. The update states that “When groups of students are learning together online, support mechanisms such as guiding questions generally influence the way students interact, but not the amount they learn” (p. xvi). This statement was the partial inspiration for the research described in this study. The researchers were curious not only about the effect of “guiding questions” on learning, but also the effect of differences in the online environment. The purpose of this study was to compare the interaction and metacognitive factors between students who engaged in verbal communication within small groups with students who worked alone and submitted responses in written format. The impetus for the study was the evolution of the instructional model for the School of Education at Stockton University. It had recently evolved from a fully face to face model to an expanded instructional model including face to face, hybrid, and fully online learning experiences. Graduate students embraced the new models because of freedom from time constraints provided by the online environment as well as opportunities to work in groups with other professionals on a flexible schedule. This component of the learning environment lent itself to students’ abilities to collaborate with peers through technology and then apply the concepts to their workplace settings, (Putman, Ford, & Tancock, 2012). In addition, critical thinking and problem solving are significant to the quality of higher education programs. The authors studied these elements through a comparison of these activities in both synchronous and asynchronous instructional formats (Sengul & Katranci, 2012).


Introduction
In 2006, the U.S. Department of Education conducted a meta-analysis of online learning studies to determine if any trends were evident.This report was updated in 2010.The update states that "When groups of students are learning together online, support mechanisms such as guiding questions generally influence the way students interact, but not the amount they learn" (p.xvi).This statement was the partial inspiration for the research described in this study.The researchers were curious not only about the effect of "guiding questions" on learning, but also the effect of differences in the online environment.The purpose of this study was to compare the interaction and metacognitive factors between students who engaged in verbal communication within small groups with students who worked alone and submitted responses in written format.
The impetus for the study was the evolution of the instructional model for the School of Education at Stockton University.It had recently evolved from a fully face to face model to an expanded instructional model including face to face, hybrid, and fully online learning experiences.Graduate students embraced the new models because of freedom from time constraints provided by the online environment as well as opportunities to work in groups with other professionals on a flexible schedule.This component of the learning environment lent itself to students' abilities to collaborate with peers through technology and then apply the concepts to their workplace settings, (Putman, Ford, & Tancock, 2012).In addition, critical thinking and problem solving are significant to the quality of higher education programs.The authors studied these elements through a comparison of these activities in both synchronous and asynchronous instructional formats (Sengul & Katranci, 2012).

Literature Review Preparing for the Adult Online Learner
When considering the move to online delivery for graduate students, courses must be designed with the adult learner in mind.Characteristics of the adult learner include their desire to determine what they learn, how they acquire and use the knowledge, and ways in which they actively engage in the learning process (Dunlap, Dudak & Konty, 2012).Faculty who teach adult students are facilitators, not the primary providers of course content.They specify the requirements of the course and guide students to complete assignments independently (Kasworm, 2011).Faculty who teach adult students also encourage them to take risks when solving problems and consider alternatives that may not be obvious.This behavior leads to broadened perspectives and enhanced application of the course content (McDougall, 2015).
Adult learners seek knowledge that is relevant to their experiences.They aspire to solve complex problems in the workplace and work to gain the confidence to tackle challenges they face in real world settings (Yoo & Huang, 2013;Harper & Ross, 2011).Motivating factors for adult learners are opportunities for self-management, control of their own learning and sharing their expertise with others so their contributions to the class are of value (Kasworm, 2012;McDougall, 2015;Harper & Ross, 2011).
Although adult learners seek independence and control in the learning environment, they also want support from the faculty.They choose programs that offer small class size, regular communication with faculty and access to technology and research information (Kasworm, 2012).They expect help with the technology needed for the online instructional delivery as well as criteria and expectations for content mastery (Yoo & Huang, 2013).

Choosing a Hybrid Learning Environment
McDougall (2015) and Harper and Ross (2011) studied the importance of adult learners' interactions with faculty members and fellow students.These opportunities might include collaborative discussions, problem analysis activities, considering and responding to others' perspectives and using new knowledge immediately in the workplace (McDougall, 2015;Harper & Ross, 2011).A hybrid model of course delivery addresses these needs with a face to face component and an online component.Regardless of environment, faculty present information and coordinate group interactions among students with case based activities, problem solving and a forum in which to share their own expertise with each other (Barak & Dori, 2009).
Within a hybrid class designed by the authors, there are two interconnected theories of instruction -constructivism and the flipped classroom.Each is an alternative approach to the traditional face to face method of instructional delivery.

Constructivist Theory of Learning
Constructivism refers to the ways in which an individual makes sense of new information presented.Students in receipt of unknown concepts or skills rely on past experience to connect the new with the known (Bas, 2012).Students in constructivist classes have opportunities to collaborate with others as they discuss real world issues and their own experiences.They share information and learn from each other by exploring alternative solutions to problems found in the workplace (Barak & Dori, 2009).Pecore (2013) identifies the common principles that educators follow in the constructivist classroomstudents are encouraged to take responsibility for their learning, to learn about their metacognitive processes, and to understand the complexity of their thinking.
The constructivist learning environment must be conducive to facilitating cooperative participation among students to foster critical thinking.Teachers construct activities based on real life experiences and challenges.Students then collaborate, sharing perspectives to create solutions to problems or evaluate existing situations and propose alternatives where they "think out of the box" (Sengul & Katranci, 2012).Each student contributes to the success of the product with his or her individual task associated with the whole assignment (Sengui & Katranci, 2012).
Faculty in higher education present problems that are content related and students explore complementary solutions based on their own experiences (Pecore, 2013).However, they must be proficient in content, pedagogy and use of technology (Wetzel, Foulger, & Williams, 2008).Assignments, designed to be real world in nature, may be presented in varied formats-visual, computerbased, written, oral, and hands-on demonstrations (Bas, 2012;Putman, Ford, & Tancock, 2012;Park, Kim, & Yu, 2011).

Benefits of the Constructivist Classroom
Research suggests that students benefit from the constructivist instructional model in several ways.They demonstrate their ability to think critically, solve problems, develop arguments, substantiate solutions and acquire knowledge strengthened by interactions with fellow students (Sengul & Katranci, 2012).They take responsibility for personal learning, and demonstrate motivation to achieve success in their academic subjects (Barak & Dori, 2007).
Within the constructivist classroom, faculty guide students as they engage in connecting and organizing new information to their previous knowledge base as they collaborate with others in the class and apply their learning to new constructs, further expanding their learning (Mishra, 2014).The value of the interactions with classmates is the exposure to different viewpoints; acknowledging differences that are essential in today's multicultural world (Putman, Ford & Tancock, 2012;Jong, Chen, Tse, Lee & Lee, 2010).Higher order thinking results when students participate in group activities where they are presented with a problem or issue to discuss and, before a solution is reached, they analyze and synthesize the information, make a connection to a real world situation, and create the resolution (Balaji & Chakrabarti, 2010).
Through this mode of teaching and learning, students are prepared for life after school as they focus on problem solving, maintaining a flexible worldview where all ideas are valued, and helping others apply new information to previous background knowledge (Balaji & Chakrabarti, 2010).Constructivist methods also direct students' learning when they choose topics for assigned course work relevant to their profession (Dunlap, Dudak & Konty, 2012).Another benefit is that students are afforded opportunities to participate in activities that reflect their own metacognitive processes (Pecore, 2013).

Challenges and Limitations of the Constructivist Classroom
The constructivist classroom model poses difficulties with student assessment.The challenge to assessment presents itself primarily with formative assessment measures.The nature of constructivist learning emphasizes process rather than a final product.The dilemma here is the assessment tool that will determine skills mastered and those in need of remediation.Therefore, when faculty assess, they look at the students' active learning in addition to the result of the learning which is a two-pronged process (Mishra, 2014).
Key to the constructivist design, a second assessment matter that must be resolved is the method by which faculty measure progress as students work in cooperative groups.Teachers recognize that students learn differently and, because they are not directly participating in the learning, faculty need a creative approach to non-traditional assessment (Swarnay 2012).Suggestions for assessment include portfolios, reflective exercises, rubrics, and surveys (Misha, 2014;del Moral, Cernea & Villalustre, 2013).
Cooperative learning endeavors in the constructivist classroom require faculty to establish clear guidelines and expectations as well as criteria for assessment.This adds to the time faculty spend teaching and grading (Putman, Ford, & Tancock, 2012).Although research supports group learning as a means to engage students in constructing their own learning, many students prefer working alone.Faculty need to consider the individual preferences of students and might modify the learning environment to accommodate these differences (Breneiser, Monett & Adams, 2012).

The Flipped Classroom
Researchers describe the flipped classroom environment as one in which students access technology for the receipt of new information provided by the teacher outside of the scheduled class.During class time, students engage in activities where they apply new knowledge to their own experiences (Keane, 2014;Roehl, Reddy, & Shannon, 2013).Moving the lecture away from the classroom through a recorded lecture platform does not replace teacher instruction but enhances the presentation and allows students to use the time in class with the teacher to participate in activities related to course content (Strayer, 2012).The primary focus of the flipped classroom is to assist teachers and students in ongoing communication, whether written or oral, as they cooperate in active learning involvement and receive prompt feedback from the teacher and fellow students (Wang & Morgan, 2008;Lo & Monge, 2013;Einfield, 2013).
The graduate students included in this study, are motivated to accomplish tasks, take responsibility for their own learning, and use higher level thought to do so (Lo & Monge, 2014).Keane (2014) notes that reasons for an increase in student academic progress include the flexibility of course delivery and assignment completion, interest in the task, and the support of other students, as a supplement to teacher support.Objectives are aligned with each assignment and the teacher ensures all expectations and assessment criteria are established (Wadley, 2010).

Benefits of the Flipped Classroom
The benefits of the flipped classroom closely resemble those of the constructivist orientation.Students work together in a non-critical, supportive environment and take risks as they work independently (Wang & Morgan, 2008).They become more aware of their own metacognition as they engage in authentic activities related to their current knowledge base (Lo & Monge, 2013) Learning is ongoing and fluid and, as new content is introduced, students embrace it and integrate into real world contexts (Himelo-Silver, Chernobilsky & Jordan, 2008).Students are no longer passive recipients of information, rather they are active in their own learning (Lo & Monge, 2013).Lastly, the flipped classroom provides more time for hands on activities that capture students' interests and motivate them to succeed in their academic subjects (Enfield, 2013).

Challenges and Limitations of the Flipped Classroom
Reports from the literature suggest that students are less satisfied with using technology and the amount of work required before each class session.They expressed concern that students were often reluctant to listen to a recorded lecture and frequently waited until the actual scheduled class (Herreid & Schiller, 2013).This failure to prepare for class in advance left them lacking in the information needed to contribute to class activities.
Another interesting point made by Wang & Morgan (2008) is that students are required to use verbal or written discussion responses to the teacher's lecture questions, but written discussions lacked the spontaneity and the higher order thinking.Faculty also did not embrace the flipped classroom but preferred the traditional course arrangement (Roehl, Reddy, & Shannon, 2013;Lo & Monge, 2013).They indicated that the amount of preparation time and the hours spent before class responding to students' questions and answers was unrealistic when teaching several courses with the flipped classroom format (Francl, 2014).

Examining the Hybrid Environment at Stockton University
The graduate level courses examined in this study employ a hybrid format combining the flipped classroom with constructivist theories.Hutchings and Quinney (2015) describe the flipped classroom as both face to face instruction and distance learning.Students assume responsibility for reading and reviewing course content delivered via technology and participate in activities where they apply the concepts in face to face interactions with classmates.The actual learning takes place in constructivist based theory.Faculty guide students as they make connections between the content and their prior knowledge and construct new knowledge in ways that reflect their individual work environments (del Moral, Cernea & Villalustre, 2012).Class time is used for student/faculty interaction.Instructors clarify concepts, respond to questions and facilitate small group activities where content is applied to real world experiences.
Online discussion forums enable students to meet, with or without faculty presence, as they collaborate on assignments, solve problems, analyze case studies and ask and answer each other's questions.Discussions were conducted using both asynchronous discussion boards and synchronous meeting rooms.Students were given the option of choosing their preferred tool.
Students enrolled in the graduate level special education courses received all instructional materials and resources within the Blackboard learning management system.Faculty uploaded syllabi, exams, rubrics, supplemental readings and pre-recorded lectures, and textbook slide decks.Students selected their own learning activities, given a framework and criteria to follow.They made choices relevant to their classroom experiences.
Reflection is a key component for adult learning and is important in the course format (Barak & Dori, 2009).Students critically analyzed the course content and considered aspects that have significant impact on their prior knowledge.They then worked to synthesize the new information as they connected it with what they already knew.Lastly, students evaluated the value of the new information and used it to enhance current practice.
Students were required to listen and respond to recorded lectures before each class session.They met online in small groups to answer the questions posed in the lecture, share their relevant experiences, and discuss the application of the content to their own classrooms.Students had the option of responding to each other either in a written format through an asynchronous discussion tool within Blackboard or in an online synchronous meeting room using Blackboard Collaborate.Discussions in Collaborate were recorded for faculty review.
The instructor read the written responses, listened to the verbal recordings and replied to all students' comments and questions.Student questions provided direction for further instruction during the following face to face class session.Additional small group work included analyzing and critiquing case studies and supplemental readings that enhanced content and provided additional opportunities for students to think critically and apply the new knowledge to their current teaching positions.

Data Collection
The sample consisted of one hundred twenty-five graduate students enrolled in three courses in the special education program.These courses were EDUC 5331, Behavior Management and Disabilities, EDUC 5320, Survey of Moderate and Severe Disabilities and EDUC 5339, Collaboration and Inclusion.These courses were offered over a two-year period, September 2013 through May 2015.
In addition to the data collected for analysis, the students were surveyed at the conclusion of each semester.They were asked to comment on the format they preferred for discussing course content.There were twelve questions for students to consider regarding the value of interactions with group members versus working alone.
Questions included items such as "Do I generally apply information from the course content to my current teaching practice?","Do members of my group compare and contrast relationships, draw inferences from others' responses to determine relevance of the questions being asked by the instructor as well as the assigned readings?","When working in my group, do the topics elicit emotional responses in me or members of my group?", and "Given the opportunity to respond to questions and assigned readings through verbal communication or written, which do you prefer and explain why?"These responses were not subjected to formal analysis but provide insight into the student experience.Highlights form this survey are presented in the discussion section.
The students selecting synchronous discussion were divided into small groups consisting of two or three members.The students selecting asynchronous discussion remained in a single group in each section.
The synchronous communications sample included forty-four instances of recorded conversations.The asynchronous communications sample included thirty-three instances of online discussion postings.
The artifacts generated by the students -recorded discussions and written discussion postingswere coded and analyzed to determine if synchronous and asynchronous communications exhibited differences.Two major areas of communication were examined: interaction and metacognition.

Data Analysis
As previously mentioned, the synchronous communications sample included forty-four instances of recorded conversations and the asynchronous communications sample included thirty-three instances of online discussion postings.Because of the small sample sizes and the disparity in sample sizes, parametric testing could not be used.To address these issues, the data was tested using the Independent-Samples Mann-Whitney U Test, a nonparametric analysis.This test ranks the data, computes the average rank of the data points, and analyzes the differences in rank, thus addressing the disparity in sample sizes.(George & Mallery, 1999, p. 197).Two-tailed hypothesis testing was selected.There were two null hypotheses: • Incidents of interactions are not significantly different when comparing the synchronous and asynchronous communications.
• Incidents of metacognition are not significantly different when comparing the synchronous and asynchronous environments.

Results
As referenced in the introduction, the United State Department of Education concluded that guiding questions influence interactions but do not affect the amount of learning that occurs.(2010, p. xvi).However, the results in this study challenge that conclusion.Student artifacts in this study were generated based on guiding questions and the data illustrates that learning did take place.
Summarizing the raw data, Table 3 lists the number of instances of interactive behaviors and Table 4 lists the number of instances of metacognitive behaviors occurring in synchronous and asynchronous environments throughout the semester.Because of the disparate group sizes, no conclusions should be drawn from the raw count of behaviors.However, the number of responses to these guiding questions do illustrate that learning was taking place and that it took place at all levels of cognitive complexity.Independent-Samples Mann-Whitney U tests were run at a significance level of .05.For interaction behaviors, the null hypothesis was rejected in five out of the six categories.Table 5 includes details on the statistical findings.For metacognitive behaviors, the null hypothesis was retained for all six categories.Table 6 includes details on the statistical findings.In all cases where the null hypothesis was rejected, the rank of the behaviors exhibited in the synchronous sample significantly exceeded the rank of the behaviors exhibited in the asynchronous sample.Although not statistically significant, in the category "Self-Perception" the rank of the behaviors in the synchronous sample still exceeded the rank of behaviors in the asynchronous sample.Details are included in Table 7.

Discussion
The examination of student learning in this study was performed after the learning experiences were complete.Counts of metacognitive behaviors indicated that learning took place.What is most important about these numbers, however, is that learning is taking place in both synchronous and asynchronous environments at all levels of Bloom's Taxonomy, not just at the lowest levels.
The conclusion presented by in the US Department of Education report was based on a study titled "The effect of using guided questions and collaborative groups for complex problem solving on performance and attitude in a Web-enhanced learning environment" (Suh, 2006).Suh concluded that the effect of the guiding questions was dissipated by the student interactions with each other.However, the findings in this study -which used guiding questions as the basis for student discussion -contradict this conclusion.
The major difference appears to be the population studied.The students in the Suh study were undergraduate, pre-service teachers.The students in this study were graduate students and practicing teachers.The findings of this study imply that student experience in the field may be an important factor in the impact of guiding questions on the learning process.
The surprising result is the rejection of the null hypothesis for the level of metacognitive behaviors.Students in the asynchronous group who had the time to research, reflect, and edit before posting on a discussion board might logically be expected to demonstrate a higher level of metacognitive behaviors than their peers who were engaged in an extemporaneous discussion.However, there was no statistically significant difference between the two groups.
Student comments in the end of semester survey were very revealing.Although more students indicated that the verbal discussion model was their preference, several students favored writing.
Those in favor of synchronous communications sited the immediacy of feedback, • "Collaborate discussion allows us to articulate our experiences as they relate to the course content and we can expand from each other's responses that spark other ideas or stories to share." and observation of theory in action.
• "The Collaborate discussions tend to go hand in hand with some of the strategies we learn about in class, for example, learning from our peers and also offering differentiated strategies in how we complete assignments."• "Learning is a social process and this collaborative approach emphasizes and expands my own learning as I learn together with others." Those in favor of asynchronous communications sited a reluctance to interact synchronously with unfamiliar colleagues, • "I prefer writing my responses or discussing with classmates face to face in class.I find it difficult to communicate with someone online that I do not really know."• "If you have members with a solid work ethic, it could be beneficial having different perspectives however, if you have members that tend to lack that, it can be frustrating." concerns over using an unfamiliar technology, • "I prefer responding in writing because I do not feel comfortable with the technology at this time."and flexible scheduling.
• "I prefer to write my responses because I can work at my own pace and on my own schedule.
It is difficult to plan with three other people our discussion times when we all work and have other obligations."

Limitations
Findings from this study should not be generalized to all populations.Although the results are encouraging, there are inherent limitations: small sample size, narrow population, self-selection, compressed time frame, and the subjective nature of the evaluation.
The sample size, both in number of participants and number of data points, is limited.One hundred twenty-five students were enrolled in the three courses that were analyzed.There were only forty-four synchronous communications in the sample and thirty-three asynchronous communications.
The student work examined in this study was the product of graduate level, practicing teachers.The work product of students in other disciplines and at other levels of education may produce different results.
The student work examined in this study was created by students who chose their modality of communication.Results may vary when participants are not free to choose between synchronous and asynchronous communications.
The student work product was generated over the course of four semesters.A study of student work product from a longer time period may produce different results.
The final area of concern is the instrument and methodology for counting.Qualitative research, in this case the coding of student behaviors, by its very nature is subjective.In this study coding differences were minimized by employing a third party coder.However, a future coder evaluating the same student work product might find different results.Refinements to definitions of interaction behaviors and metacognitive behaviors may improve potential disparities.

Implications for Future Research
The possibilities for future research grow out the limitations of the study.Future research should be done with a larger sample and broader population.For more universal applicability, the population should include undergraduate as well as graduate students and students from disciplines outside of education.Undergraduate students will require more direct instruction in the face to face portion of the flipped environment.Their previous learning environments are often more concentrated on lower levels of metacognition and may not have included reflective practices.
The participants in the study were given the option of selecting synchronous or asynchronous interactions.Adult learners often select a learning environment based on convenience.The results using participants assigned to a learning environment should be examined.
Finally, methods for refining the coder training should be explored.In particular, the coding of the synchronous conversations was challenging.Providing the coders with a transcript will cut coding times and make the process more similar to coding the asynchronous discussion.This will allow the researchers to do reliability and validity testing and will enable other researchers to use the instrument and methodology in future studies.

Table 3 :
A Count of Interactive Behaviors

Table 5 :
Hypothesis Testing Summary -Interaction Behaviors

Table 7 :
Interaction Behaviors in Synchronous versus Asynchronous Environment • "I have more to say about a topic after hearing the opinions of other group members.It also gave me insight into other methods or approaches that can be used with my own students.It helps me solve problems I face in my classroom."freedom of expression, • "When participating in Collaborate discussion, I feel as though I am more inclined to express my thoughts and opinions more freely.When I write, I am more likely to recall facts and restate what the book or article states, but neglect to apply the concepts to my experiences as a special education teacher."• "We are also able to 'vent' about issues in education whether they be in district or at the state level."diversity of opinion and experiences, • "Collaborate discussion has a voice behind it.I am able to understand a classmate's point of view and beliefs.It is important to challenge ourselves by being open minded to other's experiences and the way other (school) districts do things."• "When we complete assignments through writing, we do not have other experiences to think about or reflect on and it does not provide us with as much critical thinking when we simply answer questions."