Developing a Quality Improvement Process to Optimize Faculty Success

As part of a major shift to embed quality improvement processes within a School of Nursing at a medium-sized Midwestern university, a faculty enrichment program using a Plan-Do-Act-Study design was implemented. A central focus for the program was the development and maintenance of an online faculty resource center identified as “My Faculty Center.” Nursing faculty used the tools in My Faculty Center to assess their baseline skills in the areas of teaching & learning best practices, use of learning management system, and technology. The first goal (Goal 1) of the systematic faculty development program was to measure the impact of the Faculty Enrichment program on overall faculty development with regards to technology. The second goal (Goal 2) was to evaluate the relationship, if any, between faculty development and student evaluations. To meet Goal 2, data from IDEA end-of-course evaluations (IDEA, n.d.) and the skills assessments were correlated. Results indicated that the instructor’s desire to learn more about teaching and learning best practices was positively correlated with students providing higher scores on progress on relevant objectives, perceiving the instructor as an excellent teacher, and the course summary scores. When students rated an instructor as an excellent teacher, that instructor selfreported a higher level of applying skills in using Desire-to-Learn (D2L). Negative correlations were found between the instructor’s self-reported skills with presentation and multimedia tools and the students’ perception of progress toward objectives, excellence of course, and overall course score. Based on the initial results, future professional development will optimize adequate training on the Learning Management System (i.e. D2L), for all faculty, and emphasize the effective use of technology in the classroom. Developing a Quality Improvement Process to Optimize Faculty Success Merillat, L., & Schelbmeir, M., (2016). Developing a Quality Improvement Process to Optimize Faculty Success. Online Learning 20 (3) 159 172. Introduction Many nursing departments and schools have a disconnect between the increasing age of the nursing faculty workforce and the increased demands to improve and enhance the use of technology in nursing courses. The plethora of teaching modalities requires an assessment of faculty skills and comfort with technology in both the traditional face-to-face courses as well as those courses taught completely online. The administrative conundrum is to figure out how to assist both junior and senior nursing faculty with additional technology-related skills and to understand the impact of the use of new technological advances on the students’ assessment of course delivery. One strategy would be to utilize a quality improvement model that methodically approaches the problem of faculty needs regarding technology. Structuring a systematic process would include data collection on every individual faculty member with regards to knowledge and skills related to technology within the design and delivery of the course, and then using the data to develop both individual and group learning opportunities to enhance existing technology-related teaching skills. Literature Review Like many disciplines, faculty members in the School of Nursing are experts in their content areas, but may have no formal preparation as educators. A meta-analysis of faculty development initiatives (Steinert, Mann, Centeno, Dolmans, Spencer, Gelula, & Prideaux, 2006) found that participants benefited from, and were satisfied with, faculty development programs. Key features of effective faculty development included: • Use of experiential learning • Provision of feedback • Effective peer and colleague relationships • Well-designed interventions following principles of teaching and learning • Diversity of educational methods within single interventions In a seminal article, Wilkerson and Irby (1998) outline a comprehensive faculty development program that integrates professional, instructional, leadership, and organizational development. The authors propose that faculty members should be oriented to their university and their various faculty roles. All faculty members should have access to teaching and learning workshops. Faculty members should be encouraged to grow as leaders in their fields. To be effective, faculty members must work within their organization’s policies and procedures. In describing learner-centered professional development, Lewis (2000) recognized that professional development should involve instructors “in identifying what they need to learn and in developing learning experiences in which they will be involved” (p. 3). Faculty development should be continuous and ongoing. Instructors in blended or online courses must develop new skills to be effective educators. Fuller (2006) identified a list of 23 “practice priorities” or skills important to effective teaching in several different nursing programs. Many of these skills must be taught or demonstrated to be learned well. Arah, Hoekstra, Bos, & Lombarts (2011) developed two instruments for evaluating instruction in clinical experiences. Robinia (2008) found that nurse educators with strong online teaching efficacy beliefs valued instructional expert support, peer/mentor support, and preparatory courses and seminars for online teaching. One approach to providing preparatory courses is to embed them within an institution’s learning management system (LMS). Baker (2003) found that an Online Technology Teaching Model (OTTM) that embedded technology and pedagogical training in an LMS was effective. Faculty reported that strategies perceived as useful, engaging, and learner-centered would be used to implement innovative technology tools within their courses. Badawood, Steenkamp, & Al-Werfalli (2013) describe a structured Developing a Quality Improvement Process to Optimize Faculty Success Merillat, L., & Schelbmeir, M., (2016). Developing a Quality Improvement Process to Optimize Faculty Success. Online Learning 20 (3) 159 172. approach for improved use of the LMS. The acronym OASA stands for Opening, Analyzing, Stimulation, and Achieving Processes. With OASA, a transformation of faculty capability occurs as faculty members move from Level 1 (awareness) through to Level 5 (use of the LMS to enhance teaching). Merillat (2011) found that faculty members that used online Learning Centers embedded in Blackboard, whether alone or followed by face-to-face training, use more Blackboard features in their web-only courses. Wagner & Hulen (2015) reported on the benefits of having faculty members work with instructional designers. Instructional designers can assist by streamlining the design and development of the course as well as supplying fresh ideas to engage students. For many reasons, an instructional designer may not be available to work with faculty members as they develop their courses. Like other skills, instructional design skills can be taught. One approach is to embed this training in the LMS as well. For nursing, it’s important that faculty members not only be versed in educational technology, but also in the emerging technologies related to nursing (NLN Board of Governors, 2015). These emerging technologies include mobile health initiatives, cloud computing, virtual health care, and informatics. For a systematic, comprehensive approach to faculty development, skill development with these technologies must be considered as well. Lewis (2000) stated that faculty development “should be connected to a comprehensive change process focused on improving student learning” (p. 3). The Faculty Enrichment program at the School of Nursing seeks to make this connection.


Introduction
Many nursing departments and schools have a disconnect between the increasing age of the nursing faculty workforce and the increased demands to improve and enhance the use of technology in nursing courses.The plethora of teaching modalities requires an assessment of faculty skills and comfort with technology in both the traditional face-to-face courses as well as those courses taught completely online.The administrative conundrum is to figure out how to assist both junior and senior nursing faculty with additional technology-related skills and to understand the impact of the use of new technological advances on the students' assessment of course delivery.One strategy would be to utilize a quality improvement model that methodically approaches the problem of faculty needs regarding technology.Structuring a systematic process would include data collection on every individual faculty member with regards to knowledge and skills related to technology within the design and delivery of the course, and then using the data to develop both individual and group learning opportunities to enhance existing technology-related teaching skills.

Literature Review
Like many disciplines, faculty members in the School of Nursing are experts in their content areas, but may have no formal preparation as educators.A meta-analysis of faculty development initiatives (Steinert, Mann, Centeno, Dolmans, Spencer, Gelula, & Prideaux, 2006) found that participants benefited from, and were satisfied with, faculty development programs.Key features of effective faculty development included: All faculty members should have access to teaching and learning workshops.Faculty members should be encouraged to grow as leaders in their fields.To be effective, faculty members must work within their organization's policies and procedures.In describing learner-centered professional development, Lewis (2000) recognized that professional development should involve instructors "in identifying what they need to learn and in developing learning experiences in which they will be involved" (p.3).Faculty development should be continuous and ongoing.
Instructors in blended or online courses must develop new skills to be effective educators.Fuller (2006) identified a list of 23 "practice priorities" or skills important to effective teaching in several different nursing programs.Many of these skills must be taught or demonstrated to be learned well.Arah, Hoekstra, Bos, & Lombarts (2011) developed two instruments for evaluating instruction in clinical experiences.Robinia (2008) found that nurse educators with strong online teaching efficacy beliefs valued instructional expert support, peer/mentor support, and preparatory courses and seminars for online teaching.One approach to providing preparatory courses is to embed them within an institution's learning management system (LMS).Baker (2003) found that an Online Technology Teaching Model (OTTM) that embedded technology and pedagogical training in an LMS was effective.Faculty reported that strategies perceived as useful, engaging, and learner-centered would be used to implement innovative technology tools within their courses.Badawood, Steenkamp, & Al-Werfalli (2013) describe a structured approach for improved use of the LMS.The acronym OASA stands for Opening, Analyzing, Stimulation, and Achieving Processes.With OASA, a transformation of faculty capability occurs as faculty members move from Level 1 (awareness) through to Level 5 (use of the LMS to enhance teaching).Merillat (2011) found that faculty members that used online Learning Centers embedded in Blackboard, whether alone or followed by face-to-face training, use more Blackboard features in their web-only courses.Wagner & Hulen (2015) reported on the benefits of having faculty members work with instructional designers.Instructional designers can assist by streamlining the design and development of the course as well as supplying fresh ideas to engage students.For many reasons, an instructional designer may not be available to work with faculty members as they develop their courses.Like other skills, instructional design skills can be taught.One approach is to embed this training in the LMS as well.
For nursing, it's important that faculty members not only be versed in educational technology, but also in the emerging technologies related to nursing (NLN Board of Governors, 2015).These emerging technologies include mobile health initiatives, cloud computing, virtual health care, and informatics.For a systematic, comprehensive approach to faculty development, skill development with these technologies must be considered as well.Lewis (2000) stated that faculty development "should be connected to a comprehensive change process focused on improving student learning" (p.3).The Faculty Enrichment program at the School of Nursing seeks to make this connection.

Methodology
As part of a major shift to embed quality improvement processes within the organizational structure of a School of Nursing, a systematic faculty enrichment program was developed.The systematic plan used a Plan-Do-Study-Act design (How to Improve, n.d.).The initial step was to create metrics for assessing baseline skills (P) in the areas of teaching and learning best practices, learning management system skills (Desire-to-Learn, D2L), application of technology skills, and application of scholarship skills.Learning opportunities based upon this baseline assessment data were identified and targeted, and customized professional development was provided to the faculty (D).The next step in this quality improvement process was to correlate baseline assessment data with student course evaluations (S).Results from the correlational analyses were integrated into the existing faculty enrichment program (A) to enhance both faculty and student satisfaction with the teaching and learning process.
The chief outcomes from the Faculty Enrichment program focus on four key areas: • Increased use and efficacy of technology for professional and educational purposes • Improved implementation of teaching & learning best practices • Increased evidence of scholarship among faculty members • Increased evidence of advanced nursing concepts and innovation among faculty members Activities of the Faculty Enrichment program include both internal and external opportunities.A central focus for the program has been the development and maintenance of an online faculty resource center, My Faculty Center (Figure 1).This faculty portal is supported by the Faculty Support Center (http://www.facultysupport.com)and is accessible to faculty through the university's learning management system (LMS).

Participants
Data was compiled for 68 courses from the Spring 2014 semester: 61 face-to-face and 7 online or practicum courses.The nursing courses were taught by 27 different nursing faculty at both the undergraduate (51 courses) and graduate levels (16 courses).All of the faculty members included in this analysis were full or part-time faculty.Adjuncts were not included in this initial analysis because they did not complete all of the assessment instruments during this time period.The end-of-course evaluations were administered to 1759 students with 1392 responding.

Assessments and Measures
Correlational analysis was conducted between four course evaluation summary scores with the sub-scales from three of our assessment instruments.The four-course evaluation summary scores from IDEA evaluations were for:

Results
After correcting for Type I error using the Benjamini & Hochberg procedure (McDonald, 2015) with a false discovery rate of .1,new p values were calculated to determine levels of significance.Overall, when students rated an instructor as an excellent teacher, that instructor self-reported a higher level of applying skills in these areas with a medium effect size (Becker, 2000): In examining the other IDEA summary scores, negative correlations were found.In reporting progress on relevant objectives, students scored the course lower while the instructors self-reported a higher level of skill using presentation tools, r(66) = -.294,p < .02,and a higher level of skill using multimedia tools, r(66) = -.370,p < .01 with a medium effect size (Becker, 2000).Similar correlations were found when students rated the course as an excellent course.Students scored the course lower while the instructors self-reported a higher level of skill using presentation tools, r(66) = -.321,p < .02,and a higher level of skill using multimedia tools, r(66) = -.353,p < .01 with a medium effect size (Becker, 2000).
When evaluating the summary score for the course, courses with a lower score were negatively correlated when instructors self-reported a higher level of skill using multimedia tools, r(66) = -.314,p < .01 with a medium effect size (Becker, 2000).
When completing the checklists, faculty members were asked if they would like more information on specific items.The instructor's desire to learn more about teaching and learning best practices was positively correlated with higher student scores on progress on relevant objectives, r(64) = .318,p < .01,perceiving the instructor as an excellent teacher, r(64) = .267,p < .05,and the summary course score, r(64) = .321,p < .01 with a medium effect size (Becker, 2000).
The desire to learn more is not limited to teaching and learning best practices.There is a very strong correlation between the desire to learn more about teaching and learning best practices and to learn more about D2L, r(64) = .722,p < .01 and other technology tools, r(64) = .712,p < .01 with a large to very large effect size (Becker, 2000).
In most areas, the faculty scored above 70% on the Teaching & Learning Skills Checklist (Figure 6).By comparison, they scored less than 2.5 on a scale of 4 in each of the Technology Skills Checklist areas (Figure 7).

Discussion
The results highlight the importance for all faculty members to be adequately trained on an institution's learning management system, even if they only teach traditional face-to-face courses.The LMS is a primary communications channel for many instructors, and students rely heavily on this medium.Email can be cumbersome and is often overlooked by students who may be inundated with university "spam."The LMS can provide a stable, consistent platform for communications.The LMS can be used to supplement content delivery so more time can spent on the all-important interaction between professors and students (Bowen, 2014).
It's important to note that there can be a disparity in the skills level of full-time faculty and adjuncts in the use of an LMS (Figure 6).Repeated use of a tool doesn't guarantee that users will become more sophisticated in its use.Blackboard users tend to use only the most basic features and to use the class management features rather than to use Blackboard as an instructional tool (Woods, Baker, & Hopper, 2004;Malikowski, Thompson, & Thies, 2007;Merillat, 2008).Adjunct faculty members may not have access to the same level of orientation, training, and support as full-time faculty.
The negative correlations between presentation and multimedia skills with students' perception of progress towards goals and perceived quality of the course raise some interesting questions.For many years, educators have been operating under the assumption that improved presentation and multimedia can increase student interest and understanding of course material (Nowaczyk, Santos, & Patton, 1998;Gatlin-Watts, Arn, & Kordsmeier, 1999;Tsung Juang, W., 2010).At the School of Nursing, several factors may work against this positive outcome.Faculty members may know how to use presentation and multimedia tools, but not how to use them effectively in the classroom.For example, • Instructors rely too heavily on presentation tools such as PowerPoint (Bowen, 2006).
• When instructors use these tools, they might not be following best practices (Apperson, Laws, & Scepansky, 2008).• Instructors use these tools at the expense of student interaction (Bowen, 2006).
Analysis showed a positive correlation between an instructor's desire to learn more about teaching and learning best practices and students' perceptions of progress toward objectives, excellence of teacher, and the overall course score.An instructor's attitude toward the learning experience can have an impact.Alsharif & Qi (2014) found that instructor enthusiasm was most highly correlated with student intrinsic motivation and vitality.The desire to learn can be infectious.Those instructors who expressed a desire to learn about teaching and learning best practices also expressed a desire to learn more about D2L and other technology tools.

Conclusion
Based on these results, professional development will be optimized to ensure that all faculty members-especially new faculty and adjuncts-will be adequately trained to use the learning management system (D2L).Self-paced modules, D2L Bootcamp and D2L Advanced Topics have been added to My Faculty Center.Professional development needs to emphasize the effective use of technology in the classroom, and not simply the use of technology.
The initial results of the systematic implementation of the Faculty Enrichment Program are encouraging, and faculty members demonstrate progress in nearly all areas.The next step of our evaluation includes a similar analysis for exclusively online courses to determine if any variation exists within this subset versus the whole, as well as to examine differences between and among graduate and undergraduate courses and instructors.
experiential learning • Provision of feedback • Effective peer and colleague relationships • Well-designed interventions following principles of teaching and learning • Diversity of educational methods within single interventions In a seminal article, Wilkerson and Irby (1998) outline a comprehensive faculty development program that integrates professional, instructional, leadership, and organizational development.The authors propose that faculty members should be oriented to their university and their various faculty roles.

Figure 1 -
Figure 1 -My Faculty Center Faculty members use the tools in My Faculty Center to assess their baseline skills in the areas of teaching and learning best practices, use of D2L, technology, and scholarship.On an annual basis, each faculty member meets with the School of Nursing's instructional designer.The assessments are reviewed, updated, and an individualized faculty development plan is created (Figure 2).Resources are automatically populated for specific goals.Priorities are established and follow-up sessions with the instructional designer are scheduled.

Figure 2 -
Figure 2-Example of Individualized Faculty Development Plan Faculty members work towards their goals individually, through one-on-one tutorials with an instructional designer and workshops.Time is allocated during each monthly faculty meeting to address topics of interest to the majority of attendees.Periodically, e-mails with attached resources are sent with

•
Progress towards relevant objectives • Perception of instructor as an excellent teacher • Perception of course as an excellent course • Overall summary score The three assessment instruments and sub-scales included: • Teaching & Learning Skills Checklist (Figure 3) -97 items, 7 sub-scales o Technology Skills & Interest Checklist (Figure 5) -45 items, 10 sub-scales o Basic Applications o Communication Tools o Class Management Tools o Presentation Tools o Interaction Tools o Collaboration Tools o Multimedia Tools o Reflection Tools o Research Tools o Nursing Applications This analysis did not include correlations with the Scholarship Skills Checklist because the sample size was too small.