Blended Learning from Design to Evaluation: International Case Studies of Evidence-Based Practice

This study compares and contrasts four international faculty development programs for blended learning in order to understand the benefits, challenges, lessons learned, and recommendations from such initiatives. The benefits identified for faculty members, who participated in these programs, were that they became more reflective of their teaching practice and began to make a role adjustment from being a content provider to a designer and facilitator of learning for students. The biggest challenge appeared to be a lack of common institutional definition and understanding of blended learning as well as a lack of time and resources to support faculty in the redesign of their courses. With regards to lessons learned, each program emphasized the need for all institutional stakeholders to be involved in supporting the initiative and that blended learning does not simply imply adding digital technologies to an existing face-to-face course. The key recommendation from this study is that a faculty development program for blended learning needs to be clearly aligned with the institution’s vision and mission.


Introduction
Universities and colleges around the world are increasingly adopting blended approaches to learning and teaching (Cher Ping & Libing, 2016).The EDUCAUSE Learning Initiative (2017) recently identified faculty development for blended and online teaching as the number one key issue for teaching and learning in higher education.A significant proportion of faculty in higher learning institutions have had little formal teaching development and experience.Add to this, the pressure that is being placed on these institutions to increase access to higher education, to improve the quality of student learning, and to control or reduce the rising cost of instruction and it is not difficult to see the challenge and importance of faculty support programs (Kenny, 1998;Twigg, 2003).
In response, many institutions have initiated professional development programs to help the faculty prepare for blended teaching and effectively integrate technology into their teaching practices.These programs usually involve technology training workshops, seminars, summer institutes or project based work with a production team to create a course web site (Murray, 2002).One of the criticisms of these types of faculty development initiatives is that they do not create opportunities for sustained critical reflection and discourse about one's teaching practice.A study by the American Association of Higher Education (Rice, Sorcinelli, & Austin, 2000) suggests that: Faculty want to pursue their work in communities where collaboration is respected and encouraged, where friendships develop between colleagues within and across departments, and where there is time and opportunity for interaction and talk about ideas, one's work, and the institution.(p.13) Faculty have indeed indicated that they want to pursue their work in communities where collaboration is encouraged and respected, however scheduling regular face-to-face meetings is becoming ever more difficult.Increased teaching and research commitments leave faculty with little time for professional development.This study compares and contrasts four international faculty development programs for blended learning.Specifically, with regards to the: 1. Rationale for the blended learning faculty development initiative 2. Benefits 3. Challenges 4. Lessons learned 5. Recommendations

Rationale
The four institutions that participated in this study are the Universidade Federal de São Carlos, Brazil, KTH Royal Institute of Technology, Sweden, North-West University, South Africa, and the University of Ottawa, Canada For the Universidade Federal de São Carlos in Brazil, the main driver for the professional development initiative was to help faculty develop successful approaches to distance and blended teaching.In 2009, SEaD (Secretary General of Distance Education) an academic support organization was created.The purpose of this unit was to execute university policies and support the development of the Open University (established in 2006) and blended approaches to teaching.This organization also develops quality educational standards for distance education and the use of digital technological resources.In addition, the unit actively researches topics and issues related to distance and blended education.
Since its founding in 1827, KTH Royal Institute of Technology in Stockholm has grown to become one of Europe's leading technical and engineering universities, as well as a key centre of intellectual talent and innovation.KTH is Sweden's largest technical research and learning institution and home to students, researchers, and faculty from around the world dedicated to advancing knowledge.As a classical 'brick-and-mortar institution', the rationale for KTH to emphasize a blended approach to learning is to enhance the quality of the courses that are offered at the University.This is accomplished by helping faculty develop new learning activities and assessment methods through the use of digital technologies that can complement or replace the standard course design of lectures and written exams.The blended approach to learning that KTH is adopting is aligned with the institution's vision for 2027, which emphasizes "the virtual and physical learning environment should be equally important" (KTH, 2017).
North-West University (NWU) in South Africa consists of three main campuses with over 64,000 students.Based on a directive from the federal government, the NWU and many other higher education institutions in South Africa have adopted blended learning as a strategy to improve equitable student access and success.Their blended learning strategy has become imperative to accommodate the ever increasing numbers of students and to extend access to new populations of students, alleviate the demand on physical infrastructure, and to enhance the process of teaching and learning for the diverse body of students.New digital technologies provide opportunities for trainers, teachers, and developers to create new learning environments that support and enhance students' cognitive experiences as well as the social environment contributing to student success.
At the University of Ottawa in Canada, blended learning is endorsed by the Board of Governors as a means of reaching objectives set out in their strategic plan Destination 20/20.(University of Ottawa, 2017).In particular, the University's commitment to promote quality teaching on campus, enrich the student learning experience, and facilitate the career development of professors in the area of pedagogy and teaching innovation.
The blended learning initiative, offered through the Teaching and Learning Support Service (TLSS), is steeped in a course design process, which elaborates on the stages of a constructing a course, practices for assessing and evaluating students, proper ways of integrating technology, producing multi-media content, best practices for delivering a course, and strategies for evaluating and improving our teaching.Support is provided in a variety of formats (online modules, blended workshops, consultations, and online resources) to account for the different needs and schedules of faculty.Additionally, incentives are provided in the form of annual grants for blended course development and an excellence award for blended course design.

Benefits
Each of the four international universities involved in this study have identified the benefits of their blended learning initiatives.For example, at the Universidade Federal de São Carlos in Brazil a key benefit has been an increased focus on the teaching-learning process, specifically by faculty members being more explicit and deliberate with their pedagogical approaches to learning in their blended courses.In addition, the online training that faculty have received through the blended learning program has helped to overcome the institutional dichotomy of face-to-face versus distance education.Administrators, faculty, and students are now recognizing the value of online and blended approaches to learning in higher education.
A distinguishing feature of the KTH Royal Institute of Technology is that in order to gain tenure in Sweden faculty members are required to complete ten weeks of courses about teaching and learning in higher education.Two of the courses offered at KTH are specifically about blended learning.One is an introductory course and the other one has a project-based focus (Table 1).The major benefit of these courses is that faculty's perspective about blended learning has started to change.Faculty now view blended learning as a pedagogical approach that can be used in all programs rather than by just the early adopters of technology.Another advantage of introducing blended learning at KTH is that faculty have been triggered to reflect on their current practice as teachers and encouraged to work more on the educational design of their courses.To date, North-West University in South Africa has identified four major benefits of blended courses.First, this mode of delivery provides students with more flexible opportunities for learning by means of on-and off-campus educational experiences.Second, the design of blended courses supports the learning needs of a diverse student body, by involving students in a range of individual and collaborative learning and assessment activities, so that they can engage as learners who progressively assume responsibility for their own learning.Third, blended courses create an enabling learning environment for students that focuses on inquiry-based, active, participative, responsive, and meaningful engagement.Fourth, a blended approach can foster communities of learning outside the formal classroom and promote teaching and learning innovation.
At the University of Ottawa in Canada, the benefits of establishing a blended learning initiative have been identified at the faculty and administrative levels.In particular, the advantages described relate to the ways that perspectives and practices have changed as a consequence of the blended learning initiative.
From a faculty perspective, an introduction to blended learning created new entry points to discuss issues related to teaching and learning.For example, interest or misconceptions about the blended format spurred registration in workshops on the topic.The resulting participation created opportunities for faculty to reflect on and redesign components of their course, experience and evaluate educational technology, connect to faculty with similar interests, learn about different facets of teaching and learning, and become aware of the tools, support, and services available to them on campus.Regardless if a blended course was fully realized by a faculty member through the training process, the incremental application of principles shared in workshops helped create a more comprehensive understanding of course design, teaching, and learning.Incidentally, connecting with the TLSS prompted further participation in workshops, events, and use of consultation services.As a result, the TLSS has been able to identify and collaborate with faculty champions who share their experiences with blended learning and act as agents of change on campus.
From an administrative perspective, support of faculty in converting courses, recognition of blended learning as viable and equivalent to other formats, and promotion of this approach to teaching and learning at the program level has created entry points to evaluate and innovate current teaching practices.Additionally, administrative support demonstrates value placed on faculty member's teaching and learning practice, creates a more diverse course offering that meets students' needs, and helps meet objectives set out by the university's strategic plan.For example, applications to blended learning grants must be approved by directors who often led development efforts; programs that obtain funding for multiple courses use the opportunity to have program level discussions about how to adjust their courses, take part in training as a group, or make use of consultation services.
While others such as Siemens and Dawson (2015) have identified the advantages of blended learning from the student perspective, the blended learning initiative at the University of Ottawa has not directly supported students.However, material was created for faculty members to provide informational, technical, and promotional content to their students.In addition, collaboration with campus services providing direct student support is ongoing.

Challenges
Conversely, a series of challenges associated with blended courses have also been encountered at the four study institutions.The Universidade Federal de São Carlos has identified challenges at the macro, meso, and micro system levels.The macro system focuses on factors external to Universidade Federal de São Carlos.The key challenge at this level is dealing with an unstable Brazilian educational policy and with the modification of the financial parameters in the case of the Open University of Brazil.The meso system refers to challenges internal to Universidade Federal de São Carlos.The key challenge at this level has been to promote a 'new teaching culture' with a focus on student learning and teacher presence in the online environment.Other challenges include translating the 20% reduction of class time for blended courses into action and recognizing distance education and the use of digital technologies as methodological tools.At the micro-level, inside SEaD (Secretary General of Distance Education), numerous challenges have also been encountered.These challenges primarily relate to different theoretical perspectives, objectives, and personal points of view about blended courses.
At KTH Royal Institute of Technology, the major challenge for redesigning courses for blended learning has been time and resources.The institution needs to balance the resources required to support blended courses between the costs for purchases or programming of systems (add-ons) versus the cost for training and support of faculty members.As blended learning is a significant shift in faculty's approach to teaching the institution needs to provide them not only with support and tools for development but also the time to re-design their courses.In addition, faculty members often find it difficult to transition to their new roles as facilitator and designer rather than on just being a content provider in a blended course.
North-West University in South Africa has encountered challenges with the slow adoption of digital technologies by faculty into their teaching practice, developing a shared vision and understanding of blended learning at the institution, student readiness for this approach to learning, and change management issues.With regards to digital technologies, some of the barriers to the adoption of blended learning by faculty are their own adequate or inadequate computer skills, lack of time to prepare new and appropriate teaching and learning materials, students' restricted access to technological resources, and a lack of innovative teaching strategies to address the digital generation of students.Faculty's use of digital technology is often limited to research, academic writing, and communication.Few have advanced experience using technology for teaching activities.Successful digital technology adoption, therefore, depends on the perception and skill level of an individual faculty members.
Another challenge to overcome is a lack of a shared single vision and understanding of blended learning amongst faculty and the academic development staff and management at NWU.A clear institutional framework, shared understanding and goal is needed to achieve successful implementation.It is essential that faculty display a contextually correct understanding of the concept of blended learning in order to perform related activities accordingly.Even the faculty members who indicate that they are comfortable integrating ICT's successfully, often do not have the confidence to engage in blended courses due to a lack of adequate knowledge of blended approaches to teaching.Without a universal definition of blended learning there is no shared language by which the education field can describe the phenomena or address its opportunities and challenges.This complicates the situation and slows down the blended learning adoption rate tremendously.Although faculty have used computers and technology for a number of years in very innovative ways in their classroom teaching, until recently they have not generally used technology to provide students with a true "blend" of instruction that gives them some element of control over their learning.The institution needs to provide the necessary support to use digital technology in such a way that it brings about a fundamental shift in instruction that has the potential to optimize learning for the individual student in ways that traditional instruction never could.
With regards to student readiness for blended courses at NWU, there is a great deal of diversity with regards to location and support.For example, on-campus students have ready access to learning materials, computers, and the Internet whereas distance education students have variable access (e.g., excellent to none).
In terms of change management, the focus should be on creating blended educational environments that include the provision of high-quality learning materials to support interactive learning processes in which students develop their capacity for self-directed learning.
At the University of Ottawa in Canada, much like KTH in Sweden, there exists challenges with the time and resources needed to implement and modify our blended learning initiative.Additionally, while bilingualism is a strength and point of pride of the university, the creation of material in both languages compounds the challenge of time and resources.For example, a recent switch to a new Learning Management System (LMS) requires updating resources, workshops, and re-training staff and professors on the new platform.This in turn takes time away from developing new material on more current topics like multimedia production.
The original target of the University of Ottawa's blended learning initiative was to redesign 1000 blended courses and train 500 professors by the year 2020.However, this timeline has proved to be unrealistic as faculty members develop and integrate components of blended courses slowly and strategically.This slower pace of adoption has allowed faculty to become more familiar with the technology and support available and more comfortable with characteristics of the blended format.As a result, the development of blended courses, as per their definition, lags behind the training of professors.
Finally, while the TLSS experiences great support and enthusiasm for blended courses from some departments, others are slower in their adoption of the format.This is due in part to the diverse needs and agendas across faculties and some prevailing misconceptions about the definition of blended learning.The result is an uneven distribution of courses developed across faculties and continued promotional efforts aimed at informing the campus community about the format.

Lessons Learned
Based on their experiences supporting faculty development for blended learning each of the four institutions provide a series of lessons learned.The Universidade Federal de São Carlos has developed a series of lessons that apply to the macro, meso, and micro systems involved in supporting blended courses.At the macro level, they have attempted to develop patience, resilience, and creativity to deal with Brazil's continual changing educational policies and support for blended and online learning.With regards to the meso level, inside the university, they have learned that change takes time and extensive dialog is required, it cannot be imposed.The institution must realize that faculty members' timelines for change often differ from those mandated by the university.In terms of the micro level, inside SEaD (the unit mandated to support blended courses), there must be an emphasis on constructive and collaborative work, planned activities, sensitive listening, dialog, persistence, and the establishment of links of trust between the SEaD professional staff and faculty members.Garrison (2016) has recently written about how "Thinking and learning in collaborative settings provide an environment where participants engage in critical reflection and discourse.The intended outcomes are not only personal meaning but mutual understanding.Thinking and learning collaboratively is a process of engaging with new ideas, raising questions, and clarifying misunderstandings" (p.vii).
KTH Royal Institute of Technology has learned that change can begin to take place when everyone at the institution values online and blended approaches to teaching and learning.They have also learned that these approaches to teaching can increase faculty's pedagogical knowledge while still emphasizing that conceptual subject matter is important.In addition, they have learned that blended courses are about creating educational environments that help students become selfdirected learners, not just about adding a series of digital technology applications.Sriarunrasmee, Techataweewan, and Mebusayac (2015) studied the connection between blended courses and selfdirected learning and their results revealed that students in the blended courses had higher scores in self-directed learning and communication skills than students in the regular classroom.
The faculty development initiative at KTH has during the recent year been heavily affected by a switch to a new learning management system (LMS).This change has, however, both been negative and positive.On the negative side, just as in the University of Ottawa's case, it has been demanding to do the re-training and updating of course material for the new LMS.That said, the new LMS also serves as a catalyst for faculty to either get started to blend courses or to revise and refine their instructional design when switching platforms.
The two key lessons learned at the North-West University in South Africa are the need to redesign for blended learning and that you cannot blend by simply adding digital technologies.In addition, they have discovered that digital technologies can sometimes be a driver for change.For example, early adopters of technology are also often eager to adopt new teaching strategies for blended courses whereas laggards always need to intentionally redesign for blended teaching approaches.Blended learning is not about simply adding digital technologies to the current curriculum.Face to face teaching practices plus technology does not equal a blended course.Online activities must be created that replace or integrate with face to face class time.This comment has previously been echoed by Picciano (2009) who compares blended learning to mixing paint "when two cans of different colored paints are mixed, the new paint will look different from either of the original colors.In fact, if the new paint is mixed well, neither of the original colors will continue to exist.Similar situations exist in blended learning" (p.10).
At the University of Ottawa in Canada, the lessons learned include working with senior leadership to develop realistic targets for the blended learning program, involving all stakeholders (e.g.students, faculty, administrators, and campus community) in the development and implementation of the initiative, and creating a flexible institutional definition of blended learning.
The initial backing of the Board of Governors, while instrumental to the development of the Blended Learning Initiative, directed their initial focus almost exclusively toward faculty.As a result, other stakeholders in the organization that were directly influenced by blended learning were left out of important decisions.For example, with regards to the funding program for blended course design, an initial inclusion of program directors and financial administrators would have clarified faculty specific processes for transferring funds and approving grants.Similarly, connecting with the registrar's office in advance would have simplified the process of categorizing courses with a blended code.Porter and Graham (2016) have also identified the importance of working collaboratively with senior leadership in order to facilitate faculty adoption of blended learning in higher education.

Recommendations
As previously indicated, faculty development for blended and online teaching is a major issue for teaching and learning in higher education (ELI, 2017).The four institutions involved in this study provide recommendations for effective faculty development support.The Universidade Federal de São Carlos recommends the institutional promotion of student involvement in the process.They also recommend a greater variety of support for blended courses in order to meet the needs of different faculty groups and programs at the university.In addition, they recommend 'staying the course" with blended learning.Institutional change takes time and is a dynamic process that requires continuous review and involvement of different members of the academic community to establish short, medium and long term goals for the blended learning initiative.In terms of institutional adoption and implementation Porter et al. (2014) also emphasize that change takes time and that ongoing support is required from senior administration in order to ensure that blended learning does not become perceived as another short term "fad" in higher education.KTH Royal Institute of Technology emphasizes that blended learning should be a strategic initiative aligned with the mission and vision of the institution.First, senior leadership must 'walk the talk' by providing the necessary time and resources for faculty to develop blended approaches.Second, a recommendation is to build the faculty development initiative on a solid research-based foundation.As faculty members at KTH are also researchers, they appreciate that the professional development courses and workshops offered are grounded in current research.For them, this is a sign of quality and indicates that blended courses are a valid and reliable approach to teaching and learning in higher education.Third, a success factor for KTH has been the reorganization of research, education, development, and technology support services into one single unit.As a result of this, these four support areas have become more integrated in supporting the design, development, implementation, and evaluation of blended courses.Picciano et al. (2016) have written extensively about the importance of evidence-based practice and the need for ongoing blended learning research.
The key phrase echoed throughout the literature regarding implementation of blended courses is 'buy-in'.It is also the case at North-West University in South Africa for successful implementation.Time should be set aside to communicate with, to engage, and to actively involve faculty and students so that they are fully aware of the blended learning initiative.They recommend a three-step strategy for the successful implementation of blended learning; communicate, learn, and collaborate.In Stage One, with regards to communication; they suggest supporting formal and informal educational opportunities that prepare interested parties for blended learning initiatives.Based on their experience they believe that blended learning is too broad and complex to explain in a brief presentation.They recommend preparing a series of modular activities that are delivered in a series of workshops to help faculty and students understand concepts, models, and planning processes required to succeed with blended learning.
Stage Two, the learning phase, involves offering workshops that help faculty gain a deeper understanding of the different components of blended courses.For example, workshops on curriculum development, digital technology integration (e.g., LMS, Google Applications), and learning interface development (e.g., developing learning materials).Some of these workshops might also be appropriate for students.
Stage Three involves collaboration.Faculty members do not have to 'reinvent the wheel' and create their own blended course models.Opportunities can be created for the sharing of best practices by working with other faculty members on blended courses in virtual environments and hosting inter-campus events.These types of activities will help to build a body of knowledge from all participating campuses and faculties that is contextual to the institution.
At the University of Ottawa, the TLSS recommends identifying the objectives of the blended learning initiative early and to ground it in the context of the institution.Ensure that a range of stakeholders are involved in the process to understand the needs and potential apprehensions towards blended learning.The adoption of blended learning can require a culture change and, as a result, it is important to build momentum, have buy-in from members of the community, and to collaborate with champion faculty on campus.With regards to faculty development, the TLSS recommends providing a variety of support opportunities (online, blended, face-to-face, consultations, etc.) and clarifying the technical support and tools available for the development and implementation of blended courses.

Conclusion
This study compared and contrasted four international faculty development programs for blended learning in order to understand the rationale, benefits, challenges, lessons learned, and recommendations from such initiatives.With regards to rationale, all of the programs were designed to help faculty members develop successful approaches to blended teaching that would increase student access and success with learning opportunities in higher education.In addition, initiatives at KTH Royal Institute of Technology in Sweden (Vision 2027) and the University of Ottawa in Canada (Destination 20/20) were directly aligned with the vision and mission of the institutions.
The benefits identified for faculty members, who participated in these programs, were that they became more reflective of their teaching practice and began to make a role adjustment from being a content provider to a designer and facilitator of learning for students.The observation was made at the Universidade Federal de São Carlos that faculty who participated in the blended learning program increased their focus on the teaching-learning process by being more explicit and deliberate about their pedagogical approach to learning.At KTH in Sweden the higher education courses about blended learning have acted as a trigger for faculty to reflect on their current teaching practice and encouraged them to work more on the educational design of their courses.NWU in South Africa indicated that the blended learning initiative has become a catalyst for promoting teaching and learning innovation at their institution.And, the University of Ottawa stated that the introduction of blended learning to faculty members has created new entry points to discuss issues related to teaching and learning.
In terms of challenges, all four programs emphasized the lack of time and resources to support faculty through the complete cycle of designing, developing, implementing and evaluating blended courses.Another key challenge identified was the lack of a common institutional definition and understanding of blended learning.The faculty development team at NWU in South Africa states that without a universal definition of blended learning there is no shared language by which the education field can describe the phenomena or address its opportunities and challenges.
Two key lessons that emerged from comparing and contrasting the four international blended learning programs were the need for all institutional stakeholders to be involved in supporting the initiative and that blended learning does not simply imply adding digital technologies to an existing face-to-face course.For example, the University of Ottawa stressed the importance of working with senior leadership to develop realistic targets for the blended learning program and involving all stakeholders (e.g.students, faculty, administrators, and campus community) in the development and implementation of the initiative.KTH indicated that change can begin to take place when everyone at the institution values online and blended approaches to teaching and learning but the Universidade Federal de São Carlos emphasized that change takes time and extensive dialog is required, it cannot be imposed.Again, NWU stated the importance of developing a common institutional definition as many faculty believe that blended learning is simply about adding digital technologies to the current curriculum.They stress that face to face teaching practices plus technology does not equal a blended course.Online activities must be created that replace or integrate with face to face class time.
The key recommendation from this study is that a faculty development program for blended learning must be clearly aligned with the institution's vision and mission.KTH emphasizes that blended learning should be an institutional strategic initiative.Both the NWU of South Africa and The University of Ottawa echo this comment and recommend identifying the objectives of the blended learning initiative early and to ground it in the context of the institution.The Universidade Federal de São Carlos recommends 'staying the course' with blended learning.Their experience suggests that institutional change takes time and is a dynamic process that requires continuous review and involvement of different members of the academic community to establish short, medium and long-term goals for the blended learning initiative.Finally, NWU emphasizes the need for those responsible for the blended learning program to continually communicate, learn, and collaborate with others in the university.

Table 1 .
KTH courses in teaching and learning in higher education for faculty members