Introduction

Background

The history of Liberian Physician Assistants dates to 1965 with the realization of limited physician personnel; there was a need for clinic personnel trained in clinical diagnosis, performing basic procedures, and treating and preventing disease. A joint venture between the Liberian government, WHO, and UNICEF established the first physician assistant (PA) program at the Tubman National Institute of Medical Arts (TNIMA) in Monrovia, Liberia [1]. PA’s were initially clinically limited and in few numbers until the two civil wars (between 1980 to 2003) and the Ebola outbreak in 2013, resulting in many medical providers dying or fleeing the country, leaving less than 50 physicians to serve over 4.3 million people. Physician Assistants were mobilized to every corner of the country and elevated to provide a wider range of services, including births. Today, many Physician Assistants continue to be the sole providers of medical and obstetrical care for the entire village.

According to Challoner and Forget (2011), the kaleidoscope of political unrest and disease outbreaks led to supply chain issues for medical education supplies such as books, teaching equipment, laboratory equipment, and materials needed in training [2].  According to Stoor-Burning and Jee (2021), with medical education resources in such short supply, much of the supplies were diverted to medical and nursing schools, leaving physician assistants with ineffective or outdated books and equipment [3]. Of the many medical educational materials the PA programs could request, the plight of their smallest patients was the catalyst for the call to action of their American colleagues.  Liberia has been ranked among the world’s most challenged regarding the neonatal mortality rate of 30/1,000 live births [4]. According to the Healthy Neonatal Network (2023), the Newborn Mortality Reduction Strategy, ushered by the Liberian House of Representatives in 2019, has shown promise to reduce neonatal death rates through education to physicians and nurse midwives but has not improved education resources for Physician Assistants. The motivation of this project is to improve the educational gap in low-resource neonatal death and disability through the Helping Babies Breathe (HBB) program for Certified PAs and PA students. HBB is a train-the-trainer program developed by the American Academy of Pediatrics in collaboration with WHO, the National Institute of Child Health and Development (NICHD), and the United States Agency for International Development (USAID). It is designed to aid birth attendants in recognizing the signs and symptoms of asphyxia, the leading cause of death immediately after birth. The course guides the learner through the most common reversible causes of asphyxia while using minimal supplies and equipment, a frequent barrier in developing countries and austere environments.

Methods Participants

The targeted study participants were Physician Assistants (PA) or PA students actively enrolled in one of the three accredited programs in Liberia (TNIMA, Baptist College of Missionary Physician Assistants, and Cuttington University). The participant demographics were limited to sex and a student’s school of origin. The study did not reveal any identification on the participants to protect their confidentiality, including any questions about academic standing, their area of residence, or any information regarding their socioeconomic status. No school was chosen to participate in the activity. All PAs were recruited through the LINPAA, and PA students were recruited through a WhatsApp PA student networking chat group to ensure they were voluntarily participating, not assigned to participate by their professors. Participants arranged their transportation to and from the classes.

Participants were provided a meal, an American Association of Pediatrics NeoNatalie Newborn Simulator Complete kit, which comes with:

• NeoNatalie™ newborn simulator with umbilical cord
• 2 Umbilical ties
• 2 Sheets to simulate towels
• Head cap
• Newborn bag-mask device
• Newborn bulb suction,
• Training stethoscope
• Transport storage bag
• Certificate of completion

(https://www.laerdalglobalhealth.com) (Appendix 1).

Appendix 1: Image of Helping Babies Breathe Kit (Credit: American Academy of Pediatrics).

Most of the twenty kits were donated for the project by the authors. Due to the limited number of HBB kits available, they were all collected at the end of each class. Then each school was gifted five HBB kits (three schools total - Tubman National Institute of Medical Arts [TINMA], Cuttington University, and Baptist Missionary). The remaining kits were gifted to the Liberia National Physician Assistants Association (LINPAA) and our host facility JFK Hospital to be used for ongoing education programs.

Inclusion Criteria

PA and PA students who were adults (at and above 18 years of age at the time of the class) and who were able-bodied to perform HBB skills with accommodations if needed.

Exclusion Criteria

Any participant under 18 years of age and with an inability to perform HBB skills was asked not to participate. Participants who were not Physician Assistants were not offered participation in the study but could participate in all other aspects of the community benefit project.

Sampling Methodology

Non-probability sampling through convenience sampling was used through known contacts within the community, social media platforms, and word-of-mouth.

Research Design

The quasi-experimental research design for quantitative comparative analysis was the most effective method for analysis of HBB post-course didactic and clinical testing to compare the traditional course with the experimental virtual-led course.  Participants were informed that two HBB educational sessions were occurring on day #1 (morning and afternoon) and one session was occurring on day #2 (morning only).  Participants were assigned to either the traditional course or the virtual-led course based on the time they arrived.  The traditional course occurred on day #1 in the morning, and the virtual-led course occurred on day #1 in the afternoon.  The course on day #2 was a traditional course with Master Trainers present to assist the newly minted trainers with educational techniques and refresher of HBB methodology.  The instrumentation was dictated by the American Academy of Pediatrics (AAP) Helping Babies Breathe provided throughout the training materials and final course evaluation.

Tactics to Minimize Validity Threats

Several tactics were integrated to minimize research bias or threats to internal validity.

Instrumentation bias was controlled through the utilization of AAP-dedicated HBB education tools. Environmental factors were controlled through a team consisting of HBB Master Trainers that were provided during in-person, virtual, and mentored training sessions. Selection bias was minimized as participants did not need to sign up formally. Volunteers only needed to arrive at the prescheduled event based on the location, time, and dates that were advertised via social media and convenience sampling methods through an interest of learning HBB. Financial bias was mitigated, as the course was free of charge, all supplies needed to participate were donated and provided, and meals were provided at no cost.

Description of the Helping Babies Breathe Program

According to Singhal et al. (2020), the Helping Babies Breathe (HBB) curriculum has been designed as a pictorial, portable, skills-based pedagogy focused on basic techniques that can stimulate a neonate experiencing insufficient or absent breathing, reducing neonatal anemia through delayed umbilical cord cutting, reduction of infections through promoting of using clean tools and environments, maintaining newborn body temperature, and encouragement of early breastfeeding [5].  The course progressed in a prescribed and precise manner, following the HBB learning objectives in a stepwise fashion, with each learned skill becoming the foundation for the next.  The course revolves around easy-to-remember color schemes - Green (healthy), Yellow (needs some intervention), and Red (needs emergent intervention), all occurring within the Golden Minute after birth.  The facilitator guides the student through each color with a knowledge and skills check between each section to find gaps in knowledge before progressing to the next section.

Upon completing all the prescribed steps, the student undergoes the HBB-designed OSCE cases that test the critical thinking and clinical skills learned, followed by one of the two optional written knowledge test versions (A or B).  Remediation occurs in real-time to encourage learning, and a written re-test, if necessary, is performed with the previously unused test version. The student is provided with all the equipment and learning materials used during the class and encouraged to share the knowledge they learned with others - to continue the cycle of the Train-the-Trainer model.

The program arranged at JFK Hospital in Monrovia, Liberia, was designed to assess all aspects of the HBB model over two days. On Day#1, there were two courses offered.  First, the traditional inperson course was performed.  A virtually-led traditional course followed the traditional course - the same course as the traditional in-person, yet the only instructor was virtual without the use of any in-person instructors. While the in-person instructors were on stand-by for technical support only, they did not provide any support for teaching or remediation. 

Virtual Helping Babies Breathe (HBB) Course Overview Preparation for Virtual HBB Course

The preparation for the course included testing the presentation platforms Zoom and Google Meet. Our assumption is based on significant experiences with Zoom for our mock courses. The equipment needed for our course was the HBB kit contents, an HBB flip chart, and a copy of the HBB booklet (American Academy of Pediatrics. The Internet was supplied by personal cell phone hotspots powered by local phone data plan sim cards. An HD USB web camera supplied video and audio, along with a microphone attached to a personal laptop.

The primary researchers practiced the sessions virtually using Zoom without any problems. They added a television to split the screen between the HBB participants and the teachings of the course to mimic an in-person course. Weaknesses in our preparation include not testing Zoom or Google Workplace internationally. We learned that testing with low-bandwidth internet capacities and having a backup plan and presentation platform are imperative.

Presentation of the Virtual HBB Course

Testing was conducted on the Zoom (n.d.) video platform, and we encountered multiple problems with the video and audio portions. We tested Google Meet (n.d.) as an alternative, which functioned much more predictably throughout the course in the low internet speed offered at the JFK Hospital in Monrovia, Liberia. Via Google Meet (n.d.), introductions, an overview of the day, and the learning objectives were reviewed. A short discussion was also reviewed on what the participants wished to learn from the course and their local cultural capacities and needs.

The course explicitly followed the American Academy of Pediatrics (2021) guidelines step-by-step, with return demonstrations on each skill taught. The participants followed the pictorials in the HBB Booklet and the flip chart, which are replications of one another. The virtual instructor coached and mentored all participants, and peer review and coaching were encouraged and employed throughout the course.

Each participant had an HBB training kit, and a review of all the equipment was completed virtually. Significant time was spent on the skill development of proper bag-valve-mask ventilation as it is the core learning skill of the course.

The virtual instructor validated the skill mastery with each course segment, and a teach-the-teacher component was added at the end of each skill. Frequent referencing back to the pictorials in the book or flipchart for visual reinforcement was incorporated throughout the virtual course.

All the students successfully completed the written test. The practicum test, the OSCE-Station B from the American Academy of Pediatrics (2021), was provided to each participant. The most complex scenarios, numbers 6 and 7, were chosen randomly from the case scenarios at the end of the book for practicum testing.

No names were ascribed to the participants as they were randomly identified by coding to protect their confidentiality. We reviewed the course and celebrated the success of all the participants. Retrospectively, we could have tested and adjusted Zoom (2021) to perform better in a low-bandwidth region. Google Meets (n.d) worked in this environment without any appreciable lags or drops during the entire duration of the virtual course, allowing for uninterrupted learning. This reiterates the critical importance of a thorough quality and performance check of any technology being used remotely.

The null hypothesis to be tested was that there was no significant difference in outcomes between the traditional inperson course and the virtually led course. 

Mentor In-Person HBB Course

In the spirit of a Train-the-Trainer curriculum, an additional version of the traditional curriculum was performed to determine if the students from Day #1 become effective instructors for the students on Day #2, as this aspect is often overlooked in research. All the students from Day #1 were invited to return the next day to instruct the Day#2 cohort.  The in-person Master Trainers monitored to see if the in-person traditional cohort were better trainers than the virtually led trainers.  With the understanding that the HBB curriculum focuses on learning the skills and steps and less on becoming an instructor, we saw all the students, regardless of format, were vague on instruction and remediation techniques more commonly found in Master Trainers.  It was evident on Day #2 that the newly minted instructors, regardless of in-person or virtual session, had gaps in their ability to train others.  This required the Master Trainers, professors from Tubman National Institute of Medical Arts (TNIMA), and leadership from the Liberia National Physician Assistant Association (LINPAA) on location to develop an ad hoc curriculum for mentorship-type training during their Train-the-Trainer session, which invoked our mutual collegiality and collaboration.  The result of the mentorship-based program was higher confidence in the newly minted instructors, and all better understood the HBB curriculum.  Statistics showed the students from Day #2 were twice as likely to successfully pass their written tests and outperform their traditional and virtual predecessors in both OSCE and critical thinking challenges.

The collaboratively developed ad hoc curriculum utilized Master Trainers as mentors to the newly minted instructors. The Master Trainers provided demonstrations and techniques for teaching each learning objective. The new trainer would then instruct the new student.  Once the student appeared confident with the skills and concepts from the first objective, we would all proceed to the next, then so on until completed.  The new instructors would provide the OSCE and written test.  The Master Trainer would only observe during testing and interject if inaccurate information was provided, which allowed the new instructor to increase their confidence and refine their remediation skills.

Results

To investigate differences in student performance between training groups, mixed-effects modeling was used. Mixed-effects models are used to estimate the influence of input factors on an outcome (fixed effects) while accounting for grouping factors (random effects) that violate assumptions of independence between data points typically required for fitting linear models. All analyses were performed in R [6]. Traditional assumptions for mixed linear models were checked and no large violations were identified.

First, mixed-effects logistic regression was used to model the relationship between student training groups and multiple-choice question outcomes. Logistic regression is used when the outcome variable is binary. Each question was treated as an independent data point and served as the dependent variable. There were 18 questions and 38 students, leading to 684 total observations. The student group was treated as a fixed effect. Question number and student ID were treated as random effects. Table 1 contains the results of this analysis.

Table 1: Results of mixed-effects modeling for multiple choice question outcomes between all groups.

Based on the results of this analysis, there is not enough evidence to reject the hypothesis that student outcomes are equal between the pre-mentored Virtual and In-Person groups. A low p-value indicates that In-Person-Mentoring may result in better student outcomes, though more evidence is required to achieve the necessary statistical power. There was relatively little variance in outcomes between students. There was more variance in outcomes between questions. Model estimates can be interpreted by taking the exponential of each estimate to get the odds ratio for the outcome measure. For example, the odds of a correctly answered question for the In-PersonMentored group is estimated to be exp (0.69) or ~2 times greater than in the In-Person traditional course group. Post-hoc multiple comparisons using Tukey’s correction were applied to investigate pairwise differences between group outcomes. Table 2 contains these results:

Table 2: Pairwise differences by student group for model estimates using Tukey’s method. Values are column group – row group (p-value).

While controlling for family-wise error due to multiple comparisons, there is not enough evidence to suggest there are significant differences in outcomes between groups. The In-Person-Mentored group may outperform non-mentored groups, but more data is required to draw strong conclusions.

Next, mixed-effects logistic regression was used to model the relationship between mentor type and multiple-choice question outcomes. Logistic regression is used again. Each question was treated as an independent data point and served as the dependent variable. There were 18 questions and 15 students (mentored students only), leading to 270 observations. Mentor type was treated as a fixed effect. Question number and student ID were treated as random effects. Table 3 contains the results of this analysis.

Note: There was no significant association between mentor type and student outcomes.

Table 3: Results of mixed-effects modeling for multiple choice question outcomes by mentor type.

Discussion

Resources needed for medical education are critical for consistent learning for the next generation of medical providers.  The Physician Assistant programs throughout Liberia are critically needed to fill the health provider gaps throughout the country. They are instrumental in the first moments of life after birth to reverse brain damage or death from unrecognized breathing difficulties or lack of breathing (asphyxia).  The Helping Babies Breathe (HBB) program was developed to give resource-constrained providers and birth attendants the necessary tools to recognize asphyxia and provide appropriate interventions to reverse or improve breathing well enough to get the neonate to a higher level of care.  Liberia remains challenged in neonatal survivability and courses, like HBB, can make a difference if adequate resources and education can be provided. 

Our project provided HBB training kits and three training options to improve the train-the-trainer model.  Both the traditional and virtual courses offered on Day #1 were effective at providing the information needed to fulfill the course requirements successfully, pass the prescribed written test and OSCE, and demonstrate the critical thinking needed to recognize and manage neonatal breathing difficulty or asphyxia that can occur immediately after delivery.