Online vs. Blended Learning: Differences in Instructional Outcomes and Learner Satisfaction

In an effort to compare learning outcomes and other instructional conditions between online and blended delivery methods, this study was conducted to analyze the differences in instructional and learner factors between two groups of learners exposed to online only and blended delivery formats. Findings, discussion, and implications of differences in learning outcomes between the two delivery formats to improve either the online or blended delivery method were included.

Advances in network and communication technologies have shifted the way we deliver instruction to learners in remote locations.Owing to web enhanced communication systems and newer formats of media, various innovative instructional methods have provided learning solutions meeting the diverse needs of instructors and learners in schools and private organizations.A major concern, however, has been raised in adopting the new technologies because educators and trainers select new technologies not based on learner needs and instructional effectiveness but based on the convenience of the delivery of instructional content (MacDonald & McAteer, 2003).In an effort to meet the diverse needs of learners' learning and improve their performance levels, newer ways to blend traditional instruction with technology mediated instructional methods have emerged.Several research study findings support the positive effect of blended learning on teaching and learning (Bielawski & Metcalf, 2005).
While researchers have previously investigated the differences in learning outcomes and other instructional conditions between face-to-face instruction and online instruction formats, few studies have compared online and blended learning methods to examine differences in learning outcomes and explore mediating mechanism influencing learning.As more colleges and private sector companies adopt online or blended learning formats for the delivery of their courses and training programs, identifying how the two delivery formats are different in their effectiveness for learners' learning and satisfaction has become an important research topic for researchers and practitioners in HRD to better address the teaching and learning issues residing in both delivery formats.

Literature Review
The origin of online instruction is distance education.According to a review related literature, the evolution of distance education can be explained through four generations: (a) printed instruction, (b) early technology in broadcasting systems, (c) online instruction, and (d) web-based teleconferencing (Morabito, Sack, & Bhate, 1999).Online instruction is defined as any form of learning and/or teaching that takes place via computer network (Kearsley, 1997).The advancement of online instruction has opened a new era in distance education and contributed to the expansion of the educational opportunities by reaching people in various geographical locations thereby allowing learners global access to education (Heinich, Molenda, Russell, & Smaldino, 2002).Several researchers have advocated the use of online instruction for instructional effectiveness.Online instruction addresses the issue of time and place constraints on delivering learning experiences to distant learners and allows flexible learning modes so students can control their learning path, pace, and contingencies of instruction (Hannafin, 1984;Steinberg, 1989).For private sector organizations, one of the most significant benefits of online instruction has been just-in-time delivery of training when employees needed learning to effectively address performance problems in the workplace (Berge, 2001).In spite of the many promising features of online instruction, some pitfalls of online instruction have been its limited capability to engage learners in learning events unless the learners were self-motivated and active learners (Daniels & Moore, 2000) and requirement of strong organizational skills in their learning habits for successful learning (Oh & Lim, 2005).A sense of belonging or community is often reported lacking during online learning experiences preventing the development of shared feelings and emotions between learners and instructors.
Researchers have found that these are some of the most important factors influencing learning satisfaction and transfer effectiveness (Sergiovanni, 1994).Creating a sense of presence, a feeling of immediacy, and a broad 39-1 awareness of the real and vivid learning environment to make learners feel very alive, has also been suggested as other issues to overcome in delivering vivid learning experiences to online learners (Fontaine, 2002).
Recently blended instruction has received an increased attention as an alternative delivery method in academic institutions and private companies.The major thrust of blended instruction is to overcome the shortcomings of online instruction and utilize various instructional sequencing and delivery strategies to enhance learner satisfaction while attaining increased learning outcomes.Among the many definitions available, three representative definitions of blended instruction include: (a) a learning method that more than one delivery mode is being used with the objective of optimizing the learning outcome and cost of program delivery (Singh & Reed, 2001), (b) any mix of instructor-led training methods with technology-based learning (Bielawski & Metcalf, 2005), and (c) the mix of traditional and interactive-rich forms of classroom training with any of the innovative technologies such as multimedia, CD-ROM video streaming, virtual classroom, email/conference calls, and online animation/video streaming technology (Thorne, 2003).Regarding specific patterns of blended learning typologies, Singh and Chris (2001) have proposed six combinations of blended instruction: (a) offline and online learning, (b) self-paced, live, and collaborative learning, (c) structured and unstructured learning, (d) custom content with off-the-shelf content, (e) work and learning, and (f) ingredients of the blend synchronous physical formats, synchronous online formats, and self-paced, asynchronous formats.Reasons for using blended instruction include: improved pedagogy, easy access to knowledge, more interaction among learners, personal presence, cost effectiveness, and ease of revision of learning content (Osguthorpe & Graham, 2003).
From the extensive review of related literature about online and blended instruction, many studies revealed the usefulness and effectiveness of each learning delivery format in relation to learner satisfaction and learning outcomes.Few studies, however, empirically tested how the learners in each delivery format are different in terms of learning, instructional satisfaction, and learning involvement and motivation.Additionally, it was identified that seldom studies have been conducted to identify the difference in learners' learning application between online and blended learning environment.Here, the term 'application of learning' refers to the degree to which learners use and apply learned knowledge and skills to their current studies or to current jobs and tasks.Considering the compelling needs to identify evidences of learning effectiveness in both public and private sector organizations, the lack of evaluating learning application outcomes becomes a critical issue for researchers in educational discipline.

Purpose
The purpose of this research study was to identify the differences in instructional and learner factors, students' learning, and application of learning between the two groups of undergraduate students who took a program evaluation course through an online only or blended delivery format at a southeastern university.In order to address this study purpose, several research questions were developed.
1. Do learners in online and blended delivery format show significant increase in learning and learning application between the beginning and end of the course?2. What are the perceived differences in instructional satisfaction, learning, application of learning, and learning motivation and involvement between the learners in blended and online delivery format?3. What are the reasons facilitating or inhibiting the learners' learning and learning application in blended and online delivery format?

Sample
In order to assess learning outcomes based on the learner and instructional variables, a group of HRD undergraduate students were asked to participate in this study.The subjects for the study included 125 students (39 male and 86 female) who took a program evaluation course at a southeastern university.Among the 125 students, 59 students took the course through online delivery format and 69 through blended delivery format using classroom and online instruction.Regarding employment status, 30 students were fulltime students, 59 students had part-time jobs, and 36 students had fulltime jobs.

Instrument and Procedure
The study utilized a multi-method approach that combined close-ended and open-ended questions in an online questionnaire.Linking quantitative and qualitative data in this way enabled confirmation and corroboration of each other through triangulation, provided richer detail, and helped to initiate new lines of thinking (Rossman & Wilson, 1991; see also Miles & Huberman, 1994).The questionnaire was developed to obtain the learners' perceived degree of learning, learning application, and instructional quality of the course.The questions for both the open-ended and closed-ended parts of the questionnaire were written in a language that was familiar to the learners using terminology taught in the course.

39-1
The closed-ended part of the questionnaire used a five point Likert-type scale to measure the perceived degree of learning (1 "do not understand" to 5 "completely understand") and the perceived application of learning (1 "none" to 5 "frequently use") for the eighteen performance objectives of the course throughout the semester.The study utilized a test set to assess actual learning gain before and after each semester.Overall, a reliability alpha was .95 for the learning, .93 for the learning application, and .70 for the test respectively.To collect the pre-and post-survey data, the students were asked to participate in the surveys conducted online at the beginning and at the end of each semester.
The open-ended part of the questionnaire asked questions about the reasons for high or low perceived learning and learning application if each learner responded their perceived learning and learning application of any learning objective at 1 or 2 in the rating scale.The open-ended part of the questionnaire was located at the end of the closedended part of the questionnaire.The open-ended part of the questionnaire also asked the learners' satisfaction with instructional factors such as instructor, learning activities, group works, learning support, and suggestions to improve the course.Including the open-ended questions in the survey was believed to strengthen the study by investigating further information of the learners' insight about the course delivery.

Data Analysis
Quantitative analysis.Basic descriptive statistics were used to analyze the test scores and the perceived degree of learning, application of learning, and instructional quality responded by the learners.Paired t-tests were used to compare population mean scores for the learning increase before and after the course.We also used ANOVA to assess the differences in dependent variables between the comparison groups.
Qualitative analysis.The investigators conducted domain analyses employing content analysis procedures (see Spradley, 1979).These analyses involved sorting through the open-ended responses and identifying themes and patterns that characterized the reasons that promote or hinder the learners' learning and application for the two different delivery formats.After content analysis, cumulative frequencies and percentages for similar types and attributes identified in the domain categories were calculated to determine how often similar types were elicited.This allowed the investigators to determine included terms elicited most frequently and to gain a better understanding about the distribution of beliefs across domain categories.Linking qualitative and quantitative data in this manner helps investigators "see the general drift of the data more easily and rapidly by looking at distributions" (Miles & Huberman, 1996, p. 253).Domain categories and tentative assertions were reviewed by study participants who gave feedback.Peers researchers examined the tentative assertions as well and gave constructive comments.Conducting member checks and peer examination in this manner helps the researchers enhance authenticity and trustworthiness of the findings (Creswell, 1994;Merriam, 1998).

Context of the Course
The course was developed to teach curriculum content on learner and program evaluation for HRD undergraduate students.The course was delivered totally online for two years, and then, through a blended delivery format for another two years utilizing classroom instruction and online delivery methods.Regarding the online learner group, the instructor developed thirteen online learning modules and the workload of one module was equivalent to that of one week's classroom instruction.Four sub-learning sections comprised one learning module.Learning modules provided subject content in learner and program evaluation and various types of media such as texts, graphics, tables, audio, and video clips were used to effectively deliver the learning content to the learners.Several interactive learning activities including online discussions, case study analyses, and online tests and surveys were utilized within the modules to provide the learners with opportunities to apply learned content during learning.All learners were asked to attend the first and last class meeting for course orientation and group project presentation respectively.All learners were also asked to complete individual project to apply learned content, which can later be used as a personal portfolio in program evaluation for future job search.Regarding the blended learner group, half of the instruction was conducted in class and half was delivered through online delivery format.For an effective delivery of the blended learning, the learners were required to attend weekly classroom instruction that the instructor provided presentation on each week's major learning content of the course.After each week's classroom instruction, the learners were required to complete online learning modules to reinforce their classroom learning.The online instruction included various learning activities such as review of more related learning content, links to learning resources, group discussions, and application of learning content through assignments and group and individual projects.

Differences in Learning and Perceived Learning Application Before and After
From the data analysis the learners indicated that they experienced a significant increase in perceived and actual learning as a whole.When further analysis was conducted to assess if both delivery format groups attained significant increase in learning scores, both online and blended learning groups indicated significant increase in their perceived and actual learning (See Table 1).In order to assess differences in the learners' perceived and actual learning, perceived learning retention, and perceived learning application mean scores between the different delivery groups, an ANOVA was conducted.From the analysis, both delivery format groups did not show any differences for the dependent variables in course outcomes.Differences in Learning Motivation, Involvement, and Instructional Quality Regarding the differences in instructional and learner factors based on the two delivery formats, some meaningful differences could be analyzed.First, learners in online delivery format indicated a significantly higher mean score for instructional difficulty level than those in blended delivery format.Second, learners in online delivery format experienced significantly higher workload for their study than those in blended delivery format.On the contrary, learners in online delivery format felt significantly less learning support during study than their count part learners in blended delivery format.

Reasons for High or Low Learning and Application
From the learners' responses of the reasons to facilitate or inhibit perceived learning and application, various reason categories could be developed.In answering the survey questions, the learners were asked to provide the three most influencing reasons supporting or hindering their perceived learning and application.As collectively, the reasons in instructional effectiveness were identified as the most influential factor for learners' learning (online 49.5%, blended 52.4%, total 51%).From the various reasons under instructional effectiveness category, "clear and concise learning content" was found to be the most important reason for high learning for both groups while "review and repetition of learning" was responded more frequently by the blended learner group (17 responses) than the online learner group (4 responses).
Regarding the reasons for low perceived learning, instructional ineffectiveness was also found to be the most important reason category negatively influencing the learners' learning as a whole.Under the instructional ineffectiveness category, some specific reasons quoted by the learners are: "I really didn't completely understand these," "I need more examples to comprehend this concepts," and "Lesson was too complex for me to understand."Other important reason categories that appeared to negatively influence learners' learning include "lack of understanding," "lack of interest," and "not related to my work."Tables 3 and 4 report verbatim reason categories given for high and low perceived learning along with frequencies and percentages for these reasons, respectively.Some reason categories seemed to influence the learners perceived application of learning positively or negatively.Verbatim responses listed in Tables 5 and 6 indicate that the most important reason categories positively influenced learners' perceived learning application include "opportunity to use learning," "applicable learning content to my work," and "personal interest to use learning" while the most important reason categories negatively influenced learners' learning application were "lack of understanding," "not related to my job," "not enough opportunity to use during class," "lack of opportunity to use in my job," and "lack of motivation to apply."One interesting finding between the two comparison groups is the difference in two specific responses in "opportunity to use learning" for high perceived learning application.Learners in blended delivery group responded more frequently to "use learning during class activities, assignments, and for other classes or personal situations" than the learners in online delivery group.In terms of the instructional activities perceived helpful for learning, the learners in both groups indicated similar responses.Among all instructional activities used for the course, learners in both groups perceived group/individual project as the most important learning activity followed by discussion activities, class assignments, review/pre/post tests, case studies, multimedia cases and scenarios, lecture, and online learning modules containing the subject learning content.Table 7 reports the verbatim responses and the frequency and percentage of each learning activity for both learner groups.

Effect of Delivery Format on Learning and Application
From the data analysis, the two learner groups in online and blended delivery formats didn't show any significant differences in the mean scores for perceived and actual learning and perceived and actual learning retention, while all learners, regardless of the delivery options, indicated a significant increase in perceived and actual learning between before and after the course.This finding resembles similar findings with previous studies comparing the learning outcomes of traditional classroom with those of distance education method (Lim, 2002;Relan & Gillani, 1997: Russell, 1999;Wentling & Johnson, 1999).Also, the study findings replicate similar findings that have suggested there is no significant difference in learning application between the traditional classroom instruction and distance education delivery format (Lim, 2002(Lim, , 2004)).From this comparison, the researchers could conclude that instructional delivery format may not affect learners' learning or application of learning to a significant degree.

Instructional Factors and Conditions Affecting Learning and Application
When the researchers conducted further analyses to detect the differences that may exist between the two delivery formats, however, the study revealed several interesting findings.First, learners in online delivery group felt more workload than those in blended delivery group while they claimed less learning support than the learners in blended delivery group during their learning.This finding draws some discussions for the importance of the learners' psychological state in blended and online learning environment since learners may feel unsupported and experience an increased workload if they lack the sense of presence or belonging (McMillan & Chavis, 1986).From a practical perspective, the findings suggest that it becomes an important consideration to embed instructional activities and collaboration opportunities enhancing learners' emotional engagement with peers and instructors in designing online or blended instruction.In order to satisfy learners' sense of presence and belonging during online or blended learning, several suggested instructional strategies include: (a) providing immediate feedback on learners' questions and timely technical support; (b) asking short questions checking the understanding of major learning content at frequent intervals during instruction; (c) sending learners' learning progress report on a regular base to promote learners' motivation for learning achievement; and (d) using humor so the learners feel emotionally refreshed and engaged (Lim, 2002).
Second, blended delivery format seemed to provide clearer instructions to learners than using online delivery format only.In this study, the online learners experienced more challenges and obstacles in achieving similar learning levels than the learners in blended delivery group.They also provided relatively more claims for their lack of understanding as their reasons for low learning than the blended delivery learner group.This finding suggests that blended delivery methods may provide clearer and learner centered instructions than online only delivery method.The issue of instructional clarity in delivering online instruction has been reported from other studies (Lim, 2002(Lim, , 2004)).To resolve this issue, instructors or instructional designers of online instruction are advised to adjust the level of instructional content and apply instructional variation to satisfy learners' learning level.
Among several instructional activities and methods used for both delivery formats in this study, some instructional activities were considered more important than others.For example, group and individual project, discussion activities, and class assignments were noted as most effective learning activities for the learners' learning as a whole.This finding elicits some implications for the importance of learning application, which leads to more learner satisfaction and increased learning regardless of the different instructional formats.That is, learners seemed to value more those learning activities that they could apply learned knowledge and skills to personal situations rather than merely understanding instructed learning content in both online or blended learning environment.To promote more application of learning during blended or online instruction several instructional strategies are deemed effective.Those strategies include: (a) making learning content "applicable" as suggested by Baldwin and Ford (1988); (b) utilizing reflective activities that assist learners to apply their learning to personal situations during instruction (Clark & Taylor, 1992); and (c) embedding a structured learning process composed of a step-by-step guided practice after a segment of instruction, followed-up by individual practice to reinforce the guided practice, and independent practice through class assignments to master learned content in different application context (Lim, 2002).

Future Study and Limitations
This study contributes several meaningful findings to the HRD knowledge base regarding how to design blended or online instruction to better address learners' satisfaction and learning outcomes.Furthermore, this study sought to address the issues of learning application in online and blended learning environment, which is directly connected to the issue of performance improvement in today's organizations.As noted from other research, the nature of the open-ended questions in a survey format may not produce rich, thick description characteristic of one-on-one interviews; however, including open-ended questions helped to strengthen the study by delving further into student insight.
Even though this study revealed several meaningful research findings, the study findings are limited to online and blended learning environments occurring in college settings.To further generalize these findings, future studies using a broader population including private sector organizations are strongly recommended.Regarding the issues related to what constitutes blended instruction, this study utilized one type of blending mix of instructional method using classroom and online instruction.For generalization, future studies are needed to verify how other types of instructional blending influence learners' learning and learning application.

Table 1 .
Difference in Learning Before and After the Course Based on Delivery Format

Table 2 .
Table 2 presents the mean scores, standard error scores, and p values of the instructional and learner factors.Differences in Instructional and Learner Factors Based on Delivery Format

Table 3 .
Reasons for High Learning by Delivery Format

Table 4 .
Reasons for Low Learning by Delivery Format

Table 5 .
Reasons for High Application by Delivery Format

Table 6 .
Reasons for Low Application by Delivery Format

Table 7 .
Helpful Instructional Activities by Delivery Format