Introduction

The COVID-19 pandemic mandated changes in healthcare services to mitigate the pandemic’s effect and ensure continuous patients care. The rapid implementation of virtual clinics was one of the major digital responses to the pandemic. Virtual clinics were used in several centers before the COVID-19 pandemic to overcome barriers to optimal patient care and the overwhelming of the healthcare system. The virtual clinic service was initially dedicated to particular patients groups, such as patients with limited mobility, no access to transportation, and those living away from the healthcare service [1,2].

The outpatient visits during the pandemic had a high potential for disease transmission among patients and healthcare providers; therefore, the rapid transition to virtual clinics aimed to decrease disease transmission. Many patients had their first experience with virtual clinics during the pandemic, and patient satisfaction was a major concern [3]. Patients’ satisfaction with the virtual clinic service during the pandemic varied widely in the literature [4,5], and the patient-physician relationship is difficult to be built remotely [6]. Additionally, optimal implementation of virtual clinics could reduce the cost of care [4]; however, a study from the United Kingdom reported that the virtual clinic was not as cost-effective as the standard of care in patients with urinary incontinence because of the high re-attendance rate [7].

 Virtual clinics have been recently introduced to Saudi Arabia [8]. The quality of virtual clinics in different settings should be thoroughly evaluated before their generalization to ensure the continuation of high-quality service. Therefore, we aimed to assess patients’ satisfaction with the virtual clinic service during the COVID-19 pandemic in a primary care center and identify factors affecting it.

Patients and Methods

We conducted a cross-sectional questionnaire-based study from May 2020 to May 2021. The study included adult patients aged 18 years or above who attended the virtual clinic at Prince Sultan Primary Health center- in Riyadh- Saudi Arabia. After setting appointments with the patients, the virtual clinic visits were conducted by phone. The patients had the mental and physical ability to read and respond to the questionnaire. The questionnaire was distributed in Arabic to the patients who used the virtual clinic services during the study period. Additionally, we conducted telephone interviews with those who have already used the service, with the same questions as the written questionnaire. The double response was prevented by confirming the patients’ Medical Record Numbers (MRN) and national ID. There were no incentives to participate in the study, and the patients gave their consent to participate in the survey before responding to the questions. The local ethical committee approved the study before patients’ enrolment.

The calculated sample size was 489 patients, given that 20000 patients attended the virtual clinic, and the expected response rate was 65% with a 5% margin of error and 95% confidence interval, and an expected satisfaction rate of 70%.

A structured questionnaire was used to collect the baseline data and measure the patients’ experiences. After the literature review, we designed the questionnaire and performed face validation by two consultants; then, we conducted a pilot study. Baseline data included gender, age, and education level. Additionally, we collected data about the referral pattern to the virtual clinic and how the patients knew about this service.

The questionnaire used to assess the patients’ experience and satisfaction contained 13 questions with 5 Likert scales answers (strongly agree, agree, neutral, disagree, strongly disagree). The questionnaire evaluated if the patients received instructions before referral and if they received the call on time. The patients were asked if the physician introduced themselves before discussing the medical issue and if the patients understood their medical condition clearly, and had their questions answered during the virtual encounter. The patients were also asked if the clinic saved time and money and if they were willing to use the virtual clinic in the future and recommend it to their relatives. We explored the patients’ preference to use virtual or physical clinics in the future. The questionnaire included questions if the virtual clinic was a reason to receive timely care and whether the patients were comfortable and satisfied with the virtual clinic service.

The internal consistency of the questionnaire (reliability): The internal consistency (reliability) of the 13 items of the questionnaire was assessed using Cronbach’s alpha and showed high reliability (coefficient=0.95).

Data were presented as frequencies and percentages. Bar charts were used to graph the data. The internal consistency of the survey was tested using Cronbach’s alpha (a value of 0.70 or higher indicated good reliability of the questionnaire). We used logistic regression analysis to evaluate factors affecting patients’ satisfaction and reported odds ratios. We included all baseline variables in the multivariable logistic regression model. The outcome was patients’ satisfaction (patients who answered agreed and strongly agreed if they were completely satisfied with the service). Model calibration was tested using Hosmer-Lemeshow goodness of fit (p=0.73) and model discrimination with the area under the curve (AUC=0.79). All statistical analyses were performed using STATA 16 (Stata Corp- College Station- TX- USA). A p-value of less than 0.05 was considered statistically significant.

Results

Baseline data

The study included 507 patients; 245 (48.32%) were males and 262 (51.68%) females. Most of our participants’ age was from 18 to 29 years (n=174; 34.32%) (Figure 1). The participants’ educational level was intermediate or lower (n=41), high school (n=229), university (n=214) and postgraduate education (n=23) (Figure 2). Referral to the virtual clinic was according to the patients’ request (n=314, 61.93%) in most cases. The patients knew about the virtual clinic through a phone call from the hospital (n=247), previous experience with the virtual clinic (n=171), hospital flyers (n=80) and advertisement (n=9) (Table 1).

Patients' experience with the virtual clinic

Most patients strongly agreed that they received clear instructions before referral to the virtual clinic (n=293) and the same number of patients stated that they received a call from the virtual clinic on time.

The patients agreed that the physicians introduced themselves and the virtual clinic service and asked if they were calling at a suitable time before enquiring about their medical conditions. The virtual clinic was helpful for the explanation of the medical conditions clearly to most patients (agreed=29.98%, strongly agreed=60.75%) and for answering their questions (agreed=28.99%, strongly agreed=64.30%).

Most patients stated that the virtual clinic saved time and money compared to their physical clinic visits (agreed=24.11%, strongly agreed=66.01%). Almost 79% of the patients agreed/ strongly agreed to use the virtual clinic in future visits, and 73% would recommend it to their relatives if suitable.

Sixty percent of the patients agreed/ strongly agreed that the virtual clinic could replace the physical clinic, and 66% agreed to have their regular follow-up in the virtual clinic. Eighty-seven percent of the patients agreed/ strongly agreed that the virtual clinic was the reason to receive timely care during the COVID-19 pandemic. The virtual clinic also reassured the patients about their health conditions during the pandemic (agreed=25.84%, strongly agreed=61.75%). Eighty-six percent of the patients agreed/ strongly agreed that they were completely satisfied with the virtual clinic services (Table 2).

Factors affecting patients' satisfaction with the virtual clinic

Patients who answered agreed or strongly agreed to be completely satisfied with the virtual clinic services were merged into one group. Factors affecting their satisfaction were evaluated using logistic regression analysis. Satisfaction was significantly lower in patients above 70 years and those not referred to the virtual clinic upon their request. No other factors affected patients' satisfaction (Table 3).

Discussion

Providing remote healthcare services with the same quality and efficacy as in-person services is a goal for healthcare providers and patients. Telemedicine is replacing in-person service gradually, and it can become the standard of care in the future. The remote service can be provided through text messages, telephone calls, or video encounters. Telemedicine aims to offer services to patients unable to access medical services or those living in remote areas. The COVID-19 pandemic has accelerated the healthcare transformation in some countries [9]. During the pandemic, many patients were exposed to the new experience of remote medical services without prior preparation. The rapid transition to remote healthcare services to overcome the hardships met during the pandemic raised questions about the quality of the services provided and patients’ satisfaction.

We performed a cross-sectional study to evaluate the patients’ experience with the virtual clinic service in a primary healthcare center during the COVID-19 pandemic. The study included 507 patients; 86% were completely satisfied with the virtual clinic services. Satisfaction was significantly lower in patients above 70 years and those not referred to the virtual clinic upon their request. In a randomized trial, Healy and coworkers reported good satisfaction and a preference for a follow-up with the virtual clinic after surgical procedures compared to the actual clinic services [10]. Alharbi and associates reported a lower level of satisfaction (68%) among patients who used virtual clinics in different disciplines during the pandemic than in our study [11]. They reported a higher satisfaction rate in males and younger patients [11].

Moreover, O’Reilly and associates reported a 58% satisfaction rate among users of the virtual oncology clinic [5]. On the other hand, Santoro and colleagues reported a 6.7/7 satisfaction score in a multidisciplinary Down syndrome clinic [12]. The high satisfaction rate reported in the Santoro study is similar to Alharbi and associates, who found better satisfaction in the pediatric virtual clinic [11]. Another study reported a 93% satisfaction rate with the pediatric virtual clinic service during the pandemic [13]. The difference in satisfaction rates in different studies could be attributed to the different patients’ characteristics who used the virtual clinic; additionally, it could be related to the specialty of the virtual clinic. Our study included patients who used the primary healthcare service only. The variation in satisfaction rate reported in other studies among different specialties should be considered when implementing the virtual clinic service.

We found that the level of education did not affect satisfaction, contrary to what was reported by Alharbi and associates [11]. Abdulwahab and colleagues found that satisfaction was affected by gender, internet service quality, and specialty [14]. After the pandemic, factors affecting satisfaction should be considered when implementing the virtual clinic service. Moreover, proper education about the virtual clinic services could improve satisfaction; we found that patients referred to the virtual clinic based on their request had higher satisfaction. Another study emphasized the importance of face-to-face encounters before referring patients to the virtual clinic [15].

An important consideration when generalizing the virtual clinic service after the pandemic is the cost-effective benefits of this service. Theoretically, the virtual clinic could decrease the cost of healthcare services by reducing the utilization of resources and limiting transportation. However, the quality of the services may have a negative impact on the cost if not provided effectively. Few studies had evaluated the cost-effectiveness of telemedicine; none of them were randomized. In a systematic review, Mistry reported that the cost-effectiveness of telemedicine over conventional methods was not conclusive [16]. A similar finding was reported by Whitten and associates [17] in their review of 24 articles about the cost of delivering telemedicine. On the other hand, other studies reported potential cost-effectiveness advantages of telemedicine [18].

In addition to patients’ satisfaction, several studies have evaluated the effect of the virtual clinic service on the clinical outcome. Kolesnik and associates compared patients’ preparation for mitral valve surgery using in-person and virtual clinics during the COVID-19 pandemic. They found no differences in the ventilation time, hospital stay, and 30-day readmission between the two approaches. Additionally, they reported a 91% satisfaction rate with the virtual clinic service [19]. Al Ammari and coworkers reported equal efficacy of virtual clinics for managing anticoagulation during the pandemic compared to physical clinic visits [20].

In summary, our study showed that most patients were satisfied with the virtual clinic services. When planning future services, we should consider factors affecting satisfaction, such as the patients’ age and referral pattern. Pre-referral face-to-face education could help to improve the service provided.

Study Limitations

Several limitations should be considered when interpreting the results of this study. The study presented a single-center experience, and generalization of the results may not be feasible in all settings. Additionally, the study was performed in a primary healthcare center, and its applicability to all medical specialties needs further studies. However, the restricted inclusion of one virtual clinic could be advantageous by creating a homogeneous population. The study did not explore all factors that could have affected the satisfaction with the service.

Conclusion

The patients were generally satisfied with the virtual clinic services and logistics. Satisfaction was lower in patients older than 70 years and those who did not choose to be referred to the virtual clinic. Those two factors may be considered when planning future virtual clinic services.

Figures

Tables

References

1. McGrail KM, Ahuja MA, Leaver CA (2017) Virtual Visits and PatientCentered Care: Results of a Patient Survey and Observational Study. J Med Internet Res 19: e177.
2. Makhni MC, Riew GJ, Sumathipala MG (2020) Telemedicine in Orthopaedic Surgery: Challenges and Opportunities. J Bone Joint Surg Am 102: 1109-1115.
3. Mohammed HT, Hyseni L, Bui V, Gerritsen B, Fuller K, et al. (2021) Exploring the use and challenges of implementing virtual visits during COVID-19 in primary care and lessons for sustained use. PLoS One 16: e0253665.
4. Snoswell CL, North JB, Caffery LJ (2020) Economic Advantages of Telehealth and Virtual Health Practitioners: Return on Investment JMIR Perioper Med 3: e15688.
5. O’Reilly D, Carroll H, Lucas M, Sui J, Al Sendi M, et al. (2021) Virtual oncology clinics during the COVID-19 pandemic. Ir J Med Sci 190: 1295-1301.
6. Hanna GM, Fishman I, Edwards DA, Shen S, Kram C, et al. (2016) Development and Patient Satisfaction of a New Telemedicine Service for Pain Management at Massachusetts General Hospital to the Island of Martha’s Vineyard. Pain Med 17: 1658-1663.
7. Jones G, Brennan V, Jacques R, Wood H, Dixon S, et al. (2018) Evaluating the impact of a “virtual clinic” on patient experience, personal and provider costs of care in urinary incontinence: A randomised controlled trial. PLoS One 13: e0189174.
8. Hassounah M, Raheel H, Alhefzi M (2020) Digital Response During the COVID-19 Pandemic in Saudi Arabia. J Med Internet Res 22: e19338.
9. Bhaskar S, Bradley S, Chattu VK, Adisesh A, Nurtazina A, et al. (2020) Telemedicine Across the Globe-Position Paper From the COVID-19 Pandemic Health System Resilience PROGRAM (REPROGRAM) International Consortium (Part 1). Front Public Health 8: 556720.
10. Healy P, McCrone L, Tully R, Flannery E, Flynn A, et al. (2019) Virtual outpatient clinic as an alternative to an actual clinic visit after surgical discharge: a randomised controlled trial. BMJ Qual Saf 28: 24-31.
11. Alharbi KG, Aldosari MN, Alhassan AM, Alshallal KA, Altamimi AM, et al. (2021) Patient satisfaction with virtual clinic during Coronavirus disease (COVID-19) pandemic in primary healthcare, Riyadh, Saudi J Family Community Med 28: 48-54.
12. Santoro SL, Donelan K, Haugen K, Oreskovic NM, Torres A, et al. (2021) Transition to virtual clinic: Experience in a multidisciplinary clinic for Down syndrome. Am J Med Genet C Semin Med Genet 187: 70-82.
13. Geng-Ramos G, Taneja R, Challa C, Vazquez-Colon C, Cronin J, et al. (2022) Telemedicine for the pediatric preoperative assessment during the COVID-19 pandemic: Evaluating patient and provider satisfaction. Perioperative Care and Operating Room Management 27: 100252.
14. Abdulwahab SA, Zedan HS (2021) Factors Affecting Patient Perceptions and Satisfaction with Telemedicine in Outpatient Clinics. J Patient Exp 8: 23743735211063780.
15. Holyk T, Pawlovich J, Ross C, Hooper A (2017) The role of telehealth in improving continuity of care: The Carrier Sekani Family Services primary care model. BCMJ 59: 459-464.
16. Mistry H (2012) Systematic review of studies of the cost-effectiveness of telemedicine and telecare. Changes in the economic evidence over twenty years. J Telemed Telecare 18: 1-6.
17. Whitten PS, Mair FS, Haycox A, May CR, Williams TL, et al. (2002) Systematic review of cost effectiveness studies of telemedicine BMJ 324: 1434-1437.
18. de la Torre-Díez I, López-Coronado M, Vaca C, Aguado JS, de Castro C (2015) Cost-utility and cost-effectiveness studies of telemedicine, electronic, and mobile health systems in the literature: a systematic Telemed J E Health 21: 81-85.
19. Kolesnik I, Holmes SD, Quinn RW, Koenigsberg F, Gammie JS (2022) Effectiveness of telemedicine in a mitral valve center of excellence. J Card Surg.
20. Al Ammari M, AlThiab K, AlJohani M, Sultana K, Maklhafi N, et al. (2021) Tele-pharmacy Anticoagulation Clinic During COVID-19 Pandemic: Patient Outcomes. Front Pharmacol 12: 652482.