Using Telehealth to Enhance Pediatric Psychiatric Clinical Simulation: Rising to Meet the COVID-19 Challenge

Securing clinical locations for Bachelor of Nursing Science (BSN) programs is always a complex challenge, particularly when clinical experiences are focused on vulnerable populations such as pediatric psychiatric patients. To provide students with a meaningful clinical experience during the first weeks of the COVID-19 pandemic, we developed a telehealth simulation centered on a pediatric psychiatric patient with unfolding adolescent depression and suicidal ideation. This case study uses qualitative data analysis of student debriefing sessions as well as a pre/posttest design to explore the effectiveness of the simulation. Results suggest that simulations can be useful for presenting unique patient scenarios focused on therapeutic communication in psychiatric assessment, particularly for vulnerable patient populations. Quantitative data analysis showed BSN students improved their understanding of communication needs of adolescents and retained their knowledge over time; qualitative comments showed that students felt the simulation was a valuable learning experience and achieved key learning outcomes.


Context
The Bachelor of Science in Nursing (BSN) program serving as the context of this case study is located in a moderate sized academic health sciences center in the southern United States, where prospective students undergo a selective admission process that includes a standardized preadmission exam and interview. "Nursing Care of Children," a required pediatrics course for BSN students, contains theory and practicum (clinical) learning components. Throughout the course students apply theory-based ethics and values in the clinical setting while providing familycentered care congruent with age, stage of growth and development, and health status. Prior to spring 2020, the course was customarily taught in-person, and its student performance and retention was usually very strong, even though students often commented on the difficulty of course exams.
A central objective of the pediatric nursing course is for students to apply therapeutic communication techniques when addressing the mental health needs of adolescent patients (minors) in the care of a family member. Such skills are not easily acquired through didactic coursework alone and need to be practiced under the protected supervision of those with psychiatric expertise. Locating suitable clinical sites for psychiatric education is challenging, especially when working with pediatric patients. Previous clinical contracts at mental health units were often unsatisfactory to both students and faculty, in large part because behavioral health services have yet to be uniformly integrated into primary care, and available sites are limited to acute psychiatric care facilities. Welcoming student observers in such settings can be understandably difficult for providers who are necessarily concerned with the physical and emotional welfare of both patients and students.
In investigating potential alternatives or complements to these experiences, program faculty turned to recent studies t suggested that in-person and online simulations can provide effective experiential learning in psychiatric education (Doolen et al, 2014: Mitra & Fluyau, 2020Witt et al, 2018). Thus, in the fall of 2019-before COVID-19-faculty began developing an original standardized patient simulation to support the course objective on assessing mental health. The Adolescent Depression with Suicidal Ideation (ADSI) simulation provided a two-part scenario in which an adolescent is depressed, suicidal, and cutting herself with a plan to commit suicide. The scenario simulated an encounter requiring the BSN student to conduct a series of timesensitive actions to maintain the safety of the adolescent, including assessment with the Patient Health Questionnaire (PHQ9) depression screening tool and communication techniques, recognizing a medical emergency, and applying a patient handoff technique known as Situation-Background-Assessment-Recommendation (SBAR). SBAR is the preferred method for providerto-provider communication when transferring patient care. The simulation was collaboratively developed by a faculty member with expertise in mental health and the college of nursing simulation center coordinator, an expert in simulation development, facilitation, and debriefing. Simulation design was guided by the standards of best practice: simulation design (INACSL Standards Committee, 2016). Knowles and Kolb's theories of adult and experiential learning frame the simulation (Knowles, 1970;Kolb, 2014). Reflective debriefing was framed by the Theory of Deliberate Practice and executed using the Debriefing with Good Judgement model (Ericsson, 2015). The simulation was fully developed as a face-to-face (F2F) clinical learning experience and scheduled to pilot on April 3, 2020.

The Pivot to Remote Learning
In spring 2020, 60 BSN students enrolled in the F2F pediatric Nursing Care of Children course. The two-part unfolding ADSI was to take place in a university simulation center using a mother-daughter actor team and an APRN student to model communication and behavior techniques. In March 2020, safety restrictions due to COVID-19 required pivoting to an online simulated telehealth format. Faculty had two weeks to prioritize concerns, modify, and adjust.

Concerns
Faculty were worried that converting the simulation to an online format would impede achievement of learning outcomes. The original in-person simulation was designed to organically unfold, so as to immerse students in a scenario depicting a withdrawn 15-year-old girl whose mother has brought her into the clinic "because something just isn't right!" Producing an illusion of realism is especially important in simulations focused on recognizing warning signs and responding appropriately to wide-ranging stages of emotional distress and/or mental illness. Faculty were concerned that subtle nuances in body language, facial expressions, and speech would be difficult to discern in an online setting and prevent students from responding to patient cues with appropriate therapeutic communication techniques. Faculty were also concerned that the sense of social support and emotional safety, normally derived from physically present peers and faculty, would be imperceptible to roleplaying students in the online setting. Despite studies supporting efficacy of simulations in online settings (Harris et al., 2020) and positive results on student engagement (Rogers et al., 2020;Cantrell, 2017), faculty remained cognizant of the fact that few simulations focused on assessing mental health and even fewer on assessing mental health in a telehealth capacity.
Doubt regarding technology and connectivity were equally high. Faculty were not only unskilled in implementing online simulations, but new to converting didactic content via the Collaborate Ultra function of BlackBoard (Bb). Poor and dropped internet connections are consistent problems in our rural state. Would students be able to stay logged into the classroom long enough to hear full lectures and learn the content? Would engagement be compromised if connectivity lapsed? What would happen if we began the simulation and suddenly lost connectivity? The college of nursing simulation center coordinator had some experience adapting simulations to be remotely accessed and was able to apply her telehealth technology infrastructure when coordinating simulated patients (SP) and participants into one virtual room.

Online Rollout
Prereadings were assigned one week before the ADSI simulation with instructions to have the SBAR guidelines and PHQ9 screening tool instructions on-hand. Telehealth modules were distributed 24 hours in advance with instructions to join the event via Bb 15 minutes before start time to check for audio and camera connectivity. Because all students were new to simulation in general, a virtual orientation introduced basics and ground rules. The prebriefing introduced the simulation purpose, scenario, communication and SBAR techniques, and delineation of roles.
The ADSI simulation used the Bb platform and was deployed four times over two days with faculty dividing 60 BSN students into four groups. Each group was assigned to one of the four sessions. Students played the part of clinic nurses in pairs and logged-in from their preferred remote individual location. The pediatric APRN faculty and her daughter acted the roles of "Mom" and 15-year-old Jamie (Figure 1). The APRN student assumed the role of pediatric psychiatric clinic APRN. Faculty serving as content experts, chatroom and breakout room monitors, and the facilitator and debriefer, were located in the simulation center and distanced per COVID guidelines. The graduate student roleplaying as APRN and the mother-daughter standardized patient actors all participated from their homes. An online learning technology expert from the university's Office of Educational Development agreed to be on stand-by to resolve potential electronic disturbances. The chatroom monitor recorded attendance and observed active engagement of all attendees. Four virtual break-out rooms were created to allow the nurse pairs to intermittently step 'out' of the scenario to brainstorm and plan the course of care (this step normally takes place in an adjacent room). Students in the role of active observers were instructed to be visible at all times to provide valuable insights and support to their peers.
Simulation Scenario 1 focused on adolescent psychiatric assessment with and without Mom present and interpreting findings of the PHQ9 depression-screening tool. Faculty observers used a checklist to chart student progress in the following actions and behaviors: use of eye contact, empathy, clarifying questions, nonjudgmental conversation; ask mom to step out of the room; ask patient questions about self-harm, sexual activity, and substance abuse; identify signs of self-harm; recognize medical emergency; inform patient that some information will be shared with mom to help keep patient safe; bring mom back into the room; provide mom and patient appropriate summary and introduce plan to talk to provider.
Scenario 2 prompted roleplaying nursing students to use the SBAR handoff technique to transfer patient care (in this case to the APRN). Desired actions and behaviors on the faculty checklist for the second scenario included: use of eye contact, empathy, nonjudgmental conversation when reporting findings to provider; report findings in SBAR format; report PHQ9 findings; do not leave patient alone in exam room; and provide an appropriate summary and plan to Mom and patient.

Figure 1.
Screenshots from a Live Simulation.

Something's not right and we've got to get to the bottom of this. She's just not herself.
1b: Student role-playing as nurse I don't know if we're allowed to do this…but at some point, are we allowed to ask mom to leave the room?

Methods
We used four data sources to evaluate the simulation, including two quantitative and two qualitative sources gathered from all N = 60 students. First, the Post Simulation Learning Assessment Quantitative (PSLA) is routinely administered by the simulation center and was implemented after the simulation debrief session. The survey includes eleven five-point Likert scale items which ask students to rate their confidence in knowledge application; decision-making, reasoning, and communications skills; and perceived value of the debrief session. Second, faculty developed a five-question Adolescent Depression knowledge pre/posttest emulating national licensure exam questions. This test was implemented pre-and postsimulation to measure knowledge acquisition. PSLA and knowledge items were analyzed using descriptive statistics, paired t tests, and the Mann-Whitney U Test.
In addition to the two quantitative measures, we gathered qualitative information from two sources. First, the PSLA included two open-ended questions that asked students to share what they valued most and for ideas on improvement. Second, after the scenarios, the students debriefed their experience with faculty. The open-ended questions and simulation debrief transcripts were analyzed by three independent investigators trained in qualitative methods to identify themes. The investigators separately hand coded the data into thematic categories and then collaborated for theme consensus.

Results
In this section, we first provide insights gathered from the qualitative data, and then provide the quantitative results. Student comments in both the debrief sessions and PSLA open-ended responses indicate evidence of strong observation and reflection, the second of Kolb's (2014) four stages in experiential learning. Some students self-identified learning gaps and recognized a need for practice improvement, for example: "I wish we had asked her more about her self-harm, I just wasn't sure how to do it"; "We shouldn't have left her alone"; or "I need to get smoother at going through the history questions." Further analysis of qualitative data revealed five themes: communication, scenario, emotions, content, and modality. Communication was a vital component of the simulation because it introduced the students to the importance of effective communication with the adolescent population. Some of the student comments confirmed they benefitted from "watching the assessment and hearing the SBAR handoff" and they observed "how to help make an adolescent feel safe to open up." The students also acknowledged that the simulation prepared them to interview an adolescent privately to ensure integrity of assessment. Students reported that the scenario of the simulation "felt hands on and in person" and provided a "real time experience of a difficult situation." They also felt as if the "simulation experiences were so incredible and realistic." Many strong emotions were evoked from the students' experience; for example, "all of this is more than I can handle"; "I felt as if I were actually in clinic providing nursing care for the patient"; and "I need a moment. Can I step away from the video because I'm on the verge of tears?" The comments related to the content of this simulation allowed the faculty to see where the students gained insight into the mental health of an adolescent and gained confidence as a provider in approaching an adolescent through a difficult experience. For example, one student commented, "She is trusting you to see her pain when she shows you her cuts"-a perception that the student felt was important and yet "wasn't in the [class] readings." Responses surrounding the theme of modality indicate effectiveness of online modality for clinical learning experience, with limitations noted: "[it was] difficult to read the patient body language and facial expressions"; "I didn't like that it was on the computer, but I understand that is what we had to do"; "the online mental health simulation was beyond excellent." Results of the PSLA assessment, which asked students to rate their perceptions on learning and self-confidence performing select skills, showed that more than 95% agreed or strongly agreed with all 11 items. Faculty were concerned the online modality would reduce the sense of social presence integral to simulated learning, and thus were particularly pleased to see positive ratings on the following three items: "this learning experience developed my communication skills" (97% of students agreed); "the facilitator(s) made me feel at ease during the debriefing" (100% agreed); and "this experience developed my decision-making skills" (97% agreed).
Data from the depression knowledge pre/posttest showed an overall pre-post improvement from 70% to 77%; results for each specific item are shown in Table 1. Students showed a marked increase in knowledge in terms of the ability to recognize symptoms of depression, assess lethality of plans for self-harm, and handle immediate care. Results for knowledge about transitional care remained relatively unchanged. The item focused on assessing the adolescent in private was a difficult elect all that apply' question which required students to select B and C, but no other distractors. Prior to the simulation, 90% chose B and 97% chose C. After the simulation, 100% of the students chose both B and C. While the proportion who chose B significantly improved (at p < 0.05), the proportion who chose incorrect distractors also increased, resulting in lower post-test scores overall. Faculty noted this as an area to clarify further during class. During the simulation, students had to ask the mother to leave the room to assess the patient privately yielding surprising and contradictory answers from the initial assessment with the mother present. Following this experience, students were able to recognize the importance of "Asking the client history questions in private" resulting in a significant increase (p = 0.013).

Conclusions
The online simulation experience provided faculty and students an opportunity to complete valuable course and program objectives related to mental health in an online environment. Quantitative data analysis showed knowledge gains, increased understanding of communication needs of adolescents, and overall satisfaction with the simulation. Faculty recognized the value of analyzing the data both quantitatively and qualitatively. Qualitative comments showed that students felt this was a valuable learning experience, with excellent achievement of learning outcomes related to pediatrics and pediatric mental health objectives. Persevering in a virtual capacity allowed faculty to identify critical content areas requiring further clarification and to offer individualized intervention to students both during the simulation and extended learning time in the didactic classroom.
There were a few limitations and many lessons learned by faculty and students. The first limitation related to poor internet connectivity, which could be overcome by pre-testing participant connections several days in advance of the simulation. Faculty will need to identify multiple alternate sources of connectivity to support students and prevent educational disadvantage. A few students perceived difficulty reading facial expressions and body language, which could be mitigated by adjusting the placement of patient actors and/or camera. A few students expressed a sense of detachment when assuming the observer role and a desire to participate in the clinical role (nurse). This limitation could be addressed by reducing simulation group size and hosting additional sessions. Lastly, faculty will likely modify the knowledge test for clarity and consider implementing interrater reliability during the analysis.
Our experience suggests online simulations with standardized patients are an effective way to present unique patient scenarios focused on therapeutic communication in psychiatric assessment. Future research is needed on the utility of online simulation for other unique, vulnerable patient populations and especially for teaching psychiatric mental health conditions.