Maintaining Academic Performance and Student Satisfaction during the Remote Transition of a Nursing Obstetrics Course to Online Instruction

This study explores the effect of rapidly transitioning an in-person pre-licensure nursing specialty course to remote instruction during the COVID-19 pandemic. The redesign included the following learning technologies: live and recorded whiteboard lectures with Socratic-style questioning, electronic audience response systems, remote simulations, and virtual unfolding case studies to replace didactic and clinical learning experiences. Quantitative results indicate that learning quality was sustained, with no significant difference in students’ course performance or satisfaction. These specific technologies can be used in any discipline-specific course during emergency remote instruction to promote essential student-to-student and faculty-to-student interactivity.


Maintaining Academic Performance and Student Satisfaction during the Remote Transition of a Nursing Obstetrics Course to Online Instruction
In March of 2020, in-person education at many universities was halted due to COVID-19 (Gomez, Azadi, & Magid, 2020). Nursing is a uniquely manual profession that historically has depended heavily on in-person instruction environments to enhance critical thinking, hands-on skills development, and social integration unique to the profession. Therefore, fully online integration in pre-licensure nursing courses can be a complex and challenging process (Jackson et al., 2020;Morin, 2020). Some studies have demonstrated the academic success of remote learning programs in complex coursework disciplines, such as medicine (Bertsch et al., 2006;Callas et al., 2010). The purpose of this study is to describe the transition to required emergency remote instruction in a pre-licensure nursing course, the methods used, and evaluation of student satisfaction and academic performance between in-person and remote learning environments.

Context of the Study
The institution within this study was a public, academic medical science center in the southern United States. The course, Care of the Childbearing Family, is a required third-year prelicensure nursing course for the Bachelor of Science in Nursing (BSN) degree. The course is 7.5 weeks and customarily offered twice every spring semester (to Cohort 1 in early spring, and Cohort 2 in late spring), with two didactic class days and one 12-hour clinical experience per week. The course focuses on women's health, the childbearing experience, maternal care, and neonatal care; learning outcomes focus on student knowledge of nursing interventions for women and their families during the prenatal, antenatal, and postnatal periods. Enrollment within this course consists of typically 50-60 students per cohort. As a third-year course, the attrition rate in traditional in-person formats is estimated at 3-5% within a given cohort. The course grading scale for Care of the Childbearing Family is more rigorous when compared to general education courses. A 75% exam average is required to pass the course, and the course is a prerequisite for advancing through the undergraduate program. This policy is in place to help prepare students for the rigorous National Council Licensure Examination-Registered Nurse (NCLEX-RN) required to obtain RN licensure in the United States.

Transition to Emergency Remote Instruction
The conversion to remote instruction due to COVID-19 occurred during the first week of the second spring cohort. The transition posed several challenges related to the facilitation of learning complex maternal and neonatal concepts in the didactic setting, and the need to replace direct client clinical learning experiences. Table 1 provides a brief description of teaching methods, and how they differed between Cohort 1 and Cohort 2 due to the emergency transition to remote instruction. We then provide a more detailed description of how didactic and clinical instruction differed between the two cohorts.

Didactic Instruction
Didactic instruction within the nursing sciences typically focuses on lecture interwoven with interactive learning activities, which help establish a community of learning and inquiry (Arbour et al., 2015;Kasperiuniene et al., 2016). In particular, Socratic-style probing questions aid students with critical thinking through discussion, dialogue, and better comprehension of complex concepts (Dewald, 2020). While Cohort 1 received all instruction in-person, Cohort 2 received lectures synchronously through the learning management system (LMS). Faculty chose a synchronous modality to maintain a sense of presence with students and prevent students from feeling isolated. To maintain the interactive nature of didactic instruction, faculty used virtual whiteboards and electronic audience response systems.
During in-person lecture, instructors relied on dry-erase boards to support Socratic questioning and demonstrate the complex analytical thought processes and decision-making required of the practicing nurse. To sustain these processes in the synchronous online environment, faculty used virtual whiteboards. For example, a Collaborate Ultra whiteboard was used to teach the clinical outcomes of infants of a diabetic mother ( Figure 1). A faculty member displayed the topic "infant of diabetic mothers" and asked students to describe the six known significant complications. Using Socratic-style probing questions, the students described the six major complications and appropriate interventions. STEM-based disciplines have been slow to incorporate interactive learning techniques in lecture-heavy didactic coursework (Freeman et al., 2014). Engagement techniques, such as Socratic questioning, can easily be sustained in a virtual, synchronous setting.

Screenshot of Virtual Whiteboard in LMS.
Faculty also used an electronic audience response system (EARS), which allow students to be polled on specific questions in real-time and can be used synchronously or asynchronously and in either face-to-face or remote formats. Studies of EARS in higher education suggest these tools can enhance student engagement, create beneficial anonymity for answers, and provide instant feedback (Wood & Shirazi, 2020). Integrations through the LMS conference software allowed faculty to add multiple-choice or true/false questions for students to answer anonymously. Student responses helped guide the virtual class meeting discussion in synchronous formats, by helping faculty identify if most online learners had grasped a concept before moving on to the next learning objective.

Clinical Instruction
Clinical instruction typically includes in-hospital patient care experiences. Cohort 1 received clinical learning experiences in-person through a combination of simulation experiences and on-site acute care hospital settings. Cohort 2 received remote simulation and various methods for alternative remote clinical experiences approved by the state and national nursing boards (Campaign for Action, 2020). Clinical time was implemented primarily through virtual Assessment Technologies Institute (ATI) resources, in conjunction with some novel approaches discussed below.

Remote Simulation
Simulation has been well demonstrated in the literature to improve student satisfaction and educational development (Cant & Cooper, 2017). This course typically performs simulated training for the high-stakes, low frequency, emergent obstetric condition, postpartum hemorrhage. One of the simulation's main goals was to help students effectively apply communication techniques to promote positive maternal and neonatal outcomes through evidence-based interventions. Simulation allows the students to prepare and practice for an emergent clinical situation without having the mother and baby's life at risk.
The traditional in-person postpartum simulation included a brief introduction to a patient scenario and real-time decision making using a patient mannequin. This patient interaction was video recorded to aid in participant evaluation. In traditional simulation scenarios, students are assigned the role of direct care provider or observer. Research shows that both roles increase student knowledge and clinical reasoning skills (Alexander, 2020;Rogers, Baker, & Franklin, 2020). In Cohort 1, three students were randomly assigned the role of direct care provider and five as observers. After completing the simulation, the students watched the video recording and debriefed to discuss what went well and what could be improved. This debriefing component of simulation is regarded as paramount to ensuring students meet learning outcomes (Alexander, 2020;Rogers, Baker, & Franklin, 2020), and research suggests that debriefing after viewing a recorded video simulation enhances student technical skills, self-confidence, and self-reflection (Yeun, Chon, & An, 2020).
The short time frame for planning the COVID-19 transition to a remote platform was extremely atypical for the nursing education field, as the creation of simulation scenarios requires careful planning using a systematic and purposeful approach (INACSL Standards Committee, 2016). The emergency transition required faculty to appropriately prioritize elements of the course that could be effectively transitioned to remote environments. Due to the need for a rapid transition, our team used pre-recorded simulations from the Cohort 1 in-person class sections to implement the postpartum remote simulation experience for Cohort 2. Archived recordings were readily available upon obtaining photo and media release permissions. Students observed the remote simulations during synchronous class sessions, with the instructor facilitating the simulation with strategic pauses at key moments to discuss essential points related to communication and evidencebased interventional strategies. For example, the instructor paused the recording during the patient's initial assessment to emphasize critical nursing interventions and clinical reasoning related explicitly to postpartum assessment. Debriefing occurred synchronously after viewing the recorded video simulation.

Specialty-based Unfolding Case Studies
To replace hospital clinical time for Cohort 2, faculty developed a three-part unfolding case study about high-risk care for the newborn. Unfolding case studies describe a clinical case that unfolds in the same way it might present in a real-life patient experience and allow students to receive, process, and troubleshoot information in clinical situations. Unfolding case studies have been shown to advance clinical reasoning and judgment in nursing students in obstetric-discipline courses (Arbour et al., 2015). The high-risk newborn topic was strategic because many pathologic neonatal problems and interventions are applicable in pediatric and adult populations.
The case study's three parts unfolded as follows: clinical history, clinical presentation, and clinical nursing interventions for an acutely ill neonatal patient. One of this particular case study's main learning objectives was to recognize and identify abnormal assessment and lab findings in the high-risk newborn. For this objective, faculty preferred implementation in an asynchronous method so that students could learn how to receive and interpret information and practice selfdirected decision-making autonomously. Faculty gave in-depth, individualized feedback to each student to scaffold based on individual knowledge gaps (see Figure 2). Faculty submitted their case study for peer review by the Quality and Safety Education for Nursing (QSEN, 2020), who approved the study's rigor and overall quality. Although the content would appear drastically different, this method could be applied in various online educational programs.

Procedures
To assess the potential impact of the remote redesign on student performance and student satisfaction, course faculty conducted a formal research study exempted by the institution's review board (IRB). Data were collected from two spring 2020 class sections of the Care of the Childbearing course: Cohort 1 (n = 59) who received traditional in-person education, and Cohort 2 (n = 55), who were transitioned to emergency remote instruction. Course faculty between the two cohorts were the same. The student composition of the two courses seemed qualitatively similar, although we could not retrospectively collect racial/ethnic or demographic data and therefore cannot account for these characteristics in our analysis.

Performance Measures
To measure student performance, we compared student performance on an existing cumulative final exam between Cohort 1 and Cohort 2. Students in Cohort 2 were not proctored during the final exam, but their time was limited to 1.5 minutes per question in order to minimize the possibility for academic integrity infractions. Additionally, students were required to sign an academic honor code before testing. The Kuder-Richardson Formula 20 (KR-20) is the measurement of an examination's overall reliability and was compared between the two cohorts on the final exam.

Satisfaction Measures
End-of-course evaluations were used to compare student satisfaction and contained 18 items measured using a 5-point Likert scale. End-of-course evaluations were implemented through a de-identified, secure link initiated from the internal university registrar. Example evaluation questions included course organization, communication, learning objectives, evidence-based practice, promotion of critical thinking skills, and overall course quality.

Data Management
Aggregate exam data were exported from the exam software for score analyses and KR-20 values. The final exam consisted of 60 multiple choice and select all that apply (SATA) items and were analyzed using an unpaired t-test. De-identified end-of-course evaluations were also exported for analysis using a Mann-Whitney U test due to abnormally distributed student evaluation data. Data were analyzed in aggregate for mean/median, p-value, z-score and U using IBM SPSS version 26 (Armonk, NY). Significance was determined using p < 0.05 for both student performance and student satisfaction measures.

Student Satisfaction
Additionally, student satisfaction from course evaluations showed no statistically significant difference between the in-person Cohort 1 (Md = 4.53, n = 57) and the emergency remote Cohort 2 (Md = 4.70, n = 53), U = 58.50, z = -2.26, p < 0.24, r = .04. In order to understand whether there were any particular items driving the descriptively (but not significantly) higher score for the remote section, we also examined differences between Cohorts 1 and 2 in terms of each item. Only two individual questions had statistically significant differences between the cohorts and received higher scores in Cohort 2: the questions "The textbook(s) enhanced my understanding of the course content" (p < 0.01) and "Assignments enhanced understanding of the course content" (p < 0.02).

Lessons Learned and Conclusion
Given the emergent and complex nature of the transition to remote instruction in this course, we feel the student performance and satisfaction were exemplary. Most notably, there was no significant difference in overall student academic performance or student satisfaction after the course transitioned to remote instruction. The one exception included two items measuring satisfaction that demonstrated the textbook and assignments enhanced satisfaction with learning in the remote environment. It may be that the textbook and assignments became more important to the students' in the remote learning setting.
This study identified novel conduits for maintaining active student participation among remote-based learners. Many virtual conference platforms allow for useful tools, such as electronic audience response systems, live recording sessions for repeated viewing, and students' ability to speak via their microphones and web cameras. This infrastructure emulates as much of the traditional in-person classroom as possible to engage students, especially concerning responding to unique student questions. It is crucial to simulate the in-person environment while allowing for the unique opportunities only available in a remote setting, and in particular, to create a collaborative, informal, and student-centered experience (Martin, Ahlgrim-Delzell, & Budhrani, 2017).
Our case study may not be generalizable to all types of students and contexts. The remote learning environment may have been better received because students in our program are typically Generation Z and millennials (Lipomi, 2020). Our study's results are also limited by the inability to obtain a proctoring service for remote testing. Providing proctored testing, leveraging larger sample sizes, and accounting for demographic variances between cohorts would strengthen this study's findings.
Given these limitations, we feel this study provides valuable real-time feedback on student performance and satisfaction in an emergent transition to remote learning. The content provided is primarily applicable to nursing and medical disciplines; however, many of these methods could be readily applied to other disciplines with content alterations. Numerous published studies describe remote education in the COVID-19 pandemic ranging from medicine, English as a second language, and even remote choir education (Baxter, 2020;Eren & Oztug, 2020;Gomez, Azadi, & Magid, 2020). These methods could be applied throughout a breadth of disciplines. We hope the performance and satisfaction data adds to what is known about remote education and successful novel methods.