Introduction

Without doubt, there are many obstacles faced by surgeons in undertaking research. This has been well described with the similarity of issues often leading to a consistent recommendation regarding the critical need for a unified culture and strong institutional support to enable surgeons and their departments to develop sustainable research programs [1-3]. Indeed after coming to a crossroad regarding how surgical research could be enhanced at the local teaching hospital level, an institution wide approach was considered pivotal in developing a new model and way forward [4]. Established in early 2014, the strategy involved creating a novel hospital-based surgical academic institute with the goal to be the primary hub of surgical research and education at the hospital through the promotion, support and development of academic surgical departments [4]. Recognising that research success is rare outside a supportive environment, [1] the focus was to utilise enthusiastic leaders from surgical departments with strong academic output to support the majority of departments who were less academically active and to embed an organizational wide approach to providing infrastructure and research resources to ultimately increase their output [4]. Following an initial needs assessment to establish the baseline of available research resources and activity, [5] a wide range of strategies were implemented by the institute covering leadership and governance, research support groups and funding and resource initiatives. The detail of these strategies have been previously described [4].

Having now been in place for six years, it is timely to evaluate the early results of this model, to guide it’s further development and to inform other institutions which may be considering how best to support academic surgery [6]. Therefore, the aim of this study was to evaluate the model by investigating the change in surgical research activities and availability of dedicated research resources that has taken place during this time.

Materials and Methods

This study was a prospective evaluation of the surgical academic institute based at a tertiary referral hospital in Sydney, Australia. The baseline surgical research activity has been described previously [5]. The evaluation involved 16 surgical departments within Royal Prince Alfred Hospital (see list in Table 1). It should be noted that since the audit undertaken in 2014, due to its highly multi-disciplinary nature, ‘trauma’ has been removed as a strictly surgical department resulting in there being 16 and not 17 surgical departments in this evaluation. Two main outcomes were assessed, including measures of (i) surgical research activity between 2014 and 2019 and (ii) surgical research resources between 2014 and 2018.

Surgical Research Activity

Measures of surgical research activities included the number of journal articles published, research databases established and supervision of Higher Research Degree (HRD) students reported within each year.

Journal Articles: Defined as the number of peer-reviewed scientific articles published or accepted for publication. This information was gathered from publications that included a hospital surgeon as an author and were manually extracted from medical databases including PubMed, ResearchGate and Google Scholar. These were cross-referenced with publications presented by surgical departments and with annual reports, individual surgeon curriculum vitae’s and discussion with department Research Leads to verify possible omissions. Publications were only counted once and those accepted for publication were only within the final year of analysis in 2019. Co-publications between surgical departments were included in the individual count for each department but only attributed once in the overall total count.

Research Databases: Defined as the number of research databases created within REDCap (Research Electronic Data Capture) by each surgical department. In 2015 the implementation of REDCap, [7] which is a secure web-based application for building and managing research databases developed by Vanderbilt University, was supported and made available to all surgical departments to utilize free of charge for their research. A manual extract of all surgical research projects established within the system was provided by the local REDCap administrator. The number of surgical research databases within each department was collated and used as a quasi-indicator of active research projects. Duplicated databases, test only and quality improvement projects were excluded from the evaluation.

Student Supervision: Defined as the number of HRD students enrolled and being supervised in each year. This was not an accumulative count. Data was provided from the University of Sydney for 2017, 2018 and 2019, and included all enrolled students being supervised by a surgeon or faculty of the Institute who were undertaking a HRD such as the Doctor of Philosophy, Master of Philosophy, Doctor of Medicine (MD), and Master of Surgery.

Surgical Research Resources

Defined as the number of research resources available within the 16 surgical departments in 2014 and 2018. The summative score was designed and implemented prior to when the Institute commenced in 2014 and again in the fifth year (2018) with the surgical resources evaluated listed in Table 3. This information was collated from a structured interview with the Head of each surgical department in 2014 and 2018 who were asked to report whether a resource was or wasn’t in place (yes or no). All resources were weighted equally. Departments were arbitrarily defined as being adequately ‘research resourced’ if they reported ≥80% of the resource elements in place. The number and proportion of research resources available were calculated by surgical department and as a total for the hospital.

Analysis

All statistical analyses were performed using SPSS software version 25 (SPSS Inc., Chicago, IL USA). Regression analysis was used to identify potential linear trends for all surgical departments on the number of journal articles published per year (2014 to 2019) and number of new REDCap research databases created per year (2015 to 2019). To compare the number of HRD students between 2017 and 2019 we used the chi-square test. The number of available research resources between 2014 and 2018 were compared within each surgical department using the Chi-squared test or the Fisher’s test (small samples). A P value less than 0.05 was considered statistically significant.

Results

Surgical Research Activities

Journal Articles: During the study period, a total of 1037 unique journal articles were published. There was marked variation across the 16 surgical departments including how they progressed over the five year period with the number of articles published in any one year ranging from 1 to 51. Overall, there was a positive trend between 2014 and 2019, however, a statistical significance was not achieved (p=0.093). The melanoma & surgical oncology department published the most articles overall (n=214). The orthopaedics and neurosurgery departments demonstrated a significantly positive trend over the five years (p=0.049 and p=0.012, respectively); whereas, other surgical departments presented no significant changes in the number of publications over the years (P values ranging from 0.059 to 0.919) (Table 1).

Research Databases: From its introduction in 2015, 140 new research databases were created in REDCap by surgical departments to support research projects with a significant overall annual increase observed over the five-year period (p=0.020). Over the five years period, the colorectal surgery department (including the Surgical Outcomes Research Centre) created the most databases overall (n=41) (Table 2).

Student Supervision: A total of 80 students supervised by a surgeon or Institute affiliate were undertaking a HRD in 2017, 114 in 2018 and 100 in 2019. The majority of students in 2017 and 2018 were undertaking a Doctor of Medicine (61% and 40%, respectively), while in 2019 most of the students were undertaking a Doctor of Philosophy (39%; p<0.001).

Surgical Research Resources: Across the 16 surgical departments, 74% of the research resources or infrastructure were in place in 2018 compared to 53% in 2014 (p<0.001). Marked variation was observed across the surgical departments. Eight departments (50%) in 2018 were defined as being suitably ‘research resourced’ compared to five (31%) in 2014. The departments of urology (p=0.019), benign gynaecology (p=0.003) and plastic & reconstructive (p=0.029) presented a significant increase in surgical research resources between 2014 and 2018 (Table 3). Within the 10 research resources investigated, a significant increase was observed in the number of clinical research leads (p=0.007), research database (p=0.049), internal funding (p=0.028) and external research collaborations (p=0.033) in place (Table 3).

Discussion

The evaluation of a novel surgical academic institute based at a tertiary referral hospital in Sydney, Australia was able to determine the level of surgical research activity and availability of research resources that occurred during the six-year period of its operation from 2014 to 2019. While the experience of each surgical department was different, overall the model can be deemed a success given there has been enhanced surgical research activity demonstrated by an increasing trend in publications, research databases and students supervised, and a greater availability of research resources across the 10 domains investigated. Departments that significantly strengthened their academic standing during this time include benign gynaecology, orthopaedics and upper gastrointestinal. The academically strong departments recognised in 2014 including cardiothoracic, colorectal, head and neck surgery and melanoma & surgical oncology, continued this success in 2019.

In examining what aspects were successful, a number of key lessons emerged. Firstly, the resources most essential for a department to invigorate their research programs were the appointment of a dedicated research officer or database manager, and the creation of a research database. Although this is self-evident and hardly groundbreaking, these fundamental elements are in keeping with previous recommendations made [1,8]and were certainly pivotal for the successful departments outlined above. The Institute played a key role in supporting departments to use different funding approaches to secure their core research staff salaries including use of a unique funding model supported by the surgeons themselves and in being creative in how they could access different funds [4]. The Institute also managed these research staff on a day-to-day basis, which provided structure, consistency and a collegiate environment across surgical departments. The implementation of REDCap as previously outlined [4], was a game-changer in terms of giving departments access to a system free of charge that was designed to be used for research and was well supported across the institution with shared learning taking place across departments. Another element that was critical was the investment in academic leadership. From 2014 to 2018, all surgical departments had identified a ‘clinical research lead’ who was engaged in the Institute. Utilising a ‘hub and spoke’ model, this enabled the Institute to actively work with each department and support their research programs. Of the 16 departments, the Institute was able to secure financial remuneration for over half [9], to quarantine time for their clinical research lead to coordinate the research for their department. This also served to re-invigorate support for academic surgeons by the University. With various novel funding models utilised, two retiring Professors of Surgery (2.0 full-time equivalent, Level E) in 2014 have been replaced with 13 x 0.5 full-time equivalent academic surgeons across many departments including three Level E, eight Level D and two Level C. Finally, the successful integration of medical students into the Institute has been pivotal for providing early support and mentorship to those with an early interest in academic surgery. This has been achieved through the introduction of the MD program at the University and subsequent supervision of many student research projects (121 in total), along with the Sydney University Surgical Society (SUSS) becoming physically based within the Institute premises. It is noted that due to changes in both the supervisor requirements and the scope of research projects that students are able to undertake, there has been a steady decrease in the number of MD students requiring supervision by a surgeon or faculty of the Institute observed recently, and this will require new strategies to ensure the strong engagement with medical students continues.

In terms of what didn’t work and the ongoing challenges experienced by the Institute and surgical departments, a number of lessons also emerged. Overall limited inroads were made into the field of laboratory research. In 2018, only half (n=8) of the surgical departments had an identified laboratory lead and had accessed wet laboratories for undertaking research. This is likely a reflection of the demands of current clinical practice whereby surgeons don’t have the time and resources needed to access and maintain a wet laboratory. It has been suggested that expectations regarding surgeon’s involvement in basic research need to shift and a more pragmatic model should involve a smaller number of surgeons trained in basic and translational research collaborating with a larger number of surgical scientists who undertake the studies [8]. It is evident the Institute needs to consider how this research resource and activity is supported and measured going forward. In addition, when the Institute was first created it was envisaged that a novel measure titled the Clinical Trials Index (CTI) would be widely implemented. This would capture the proportion of a department’s patients who were subsequently enrolled in a clinical trial or prospective study [4]. This has not occurred with only three (19%) of the surgical departments utilising the measure in 2018. In hindsight this was an ambitious strategy given the majority of surgical departments were in the early stages of establishing their research programs and developing prospective studies. Furthermore, it was anticipated a new research specific component of the hospital’s eMR would be implemented that would automate collection of the measure however this did not take place.

Finally, a number of surgical departments remained static in their research development including ophthalmology, vascular and plastics. Each department is different and have differing reasons on qualitative assessment and discussion about their needs. These include recent changes in consultant staffing, large departments with small individual time commitments to the one institution as well as an inability of consultants to accept one surgeon as the research lead.

In undertaking this evaluation, a range of metrics were used to determine the success of the model. These were selected due to the availability of data and because they are widely accepted measures of academic output [9]. The most effective way to assess the academic contribution of a surgical department is complex and debatable depending on the focus of the review with different metrics utilised to emphasise different levels of success [6,9,10]. Certainly the metrics chosen for this evaluation could be viewed as a rather narrow measure of surgical research and may have missed a number of critical components of a department’s research, such as innovation, research impact or embedding research into clinical programs [9], which has been a priority of the Institute. Over the period examined there has been some advances in this area with the Institute involved in leading a number of unique programs. This includes the implementation of a public robotic-assisted surgery program within a comprehensive research framework [11], the use of 3D printing in surgical planning and the establishment of 3D facilities based at clinical sites [12], the creation of a new specialized centre for providing cytoreductive surgery and Hyperthermic Intraperitoneal Chemotherapy (HIPEC) utilising an external mentored model [13], as well as the initial development of novel research programs within facial and uterine transplant.

Similarly, although prior to the existence of the Institute, there were no research metrics being collected across surgical departments, and little in the literature to compare it to, it may be misguided to assume academic progress by each department as having to be linear. The pathway from initial research idea through to translation is long and nuanced, and it is not unreasonable to anticipate there being peaks and troughs in how a department rates academically when considered on an annual basis and within only the first five years of the institute model. There were a number of limitations with this study including that it is based on the single experience of a metropolitan based large tertiary referral hospital that is co-located on a University campus. As such the novel academic model implemented may not translate successfully to other centres particularly those that are smaller or rurally based. The evaluation of research activity was based on manual extraction of available data and so may have resulted in missed data due to human error. Similarly, the availability of research resources was based on data self-reported by the surgical Heads of Department and was not verified by an independent source. However, this was equally assessed both before and after. Finally, there was no control group to compare the results against.

In contemplating future endeavors, there is potential for this model to be considered by other tertiary referral hospitals in major capital centres, particularly across Australia and New Zealand, where there is the volume of both staff and clinical workload to support the academics activities and to generate a unified culture. There are currently two other similar models being established in tertiary referral hospitals in Sydney with support and guidance being provided from this unit and with the strong support of their own hospital administration and University, which are critical components of the model.

In conclusion, this evaluation of a novel surgical academic institute based at a tertiary referral hospital in Sydney, Australia was able to determine promising trends in surgical research activity and the availability of research resources during the initial six-year period of its operation from 2014 to 2019. Longer term outcomes of research translation impacting on clinical procedures will be proactively observed and monitored.

Tables

References

1. Watson DI (2008) Surgical research: lost in translation? ANZ journal of surgery 78: 358.
2. Santilli SM (2008) Current issues facing academic surgery departments: stakeholders’ views. Academic Medicine 83: 66-73.
3. Ceelen WP (2014) Clinical research in surgery: threats and opportunities. European Surgical Research 53: 95-107.
4. McBride KE, Solomon MJ, Bannon PG, et al. (2017) Enhancing surgical research at the hospital level: a new model. ANZ Journal of Surgery 87: 431-432.
5. McBride KE, Young JM, Bannon PG, Solomon MJ (2017) Assessing surgical research at the teaching hospital level. ANZ Journal of Surgery 87: 70-75.
6. Zargar‐Shoshtari K, Hill A (2017) Surgical research in the public hospital. ANZ Journal of Surgery 87: 428-429.
7. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, et al. (2009) Research electronic data capture (REDCap)-a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics 42: 377-381.
8. Smythe WR (2010) The future of academic surgery. Academic Medicine 85: 768-774.
9. Souba WW, Wilmore DW (2000) Judging surgical research: how should we evaluate performance and measure value? Annals of Surgery 232: 32.
10. Ashrafian H, Rao C, Darzi A, Athanasiou T (2009) Benchmarking in surgical research. The Lancet 374: 1045-1047.
11. McBride KE, Steffens D, Solomon MJ, Anderson T, Young JM, et al. (2019) Research as the gatekeeper: introduction ofrobotic-assisted surgery into the public sector. Australian Health Review 2019.
12. Mukherjee P, Cheng K (2019) 3D printing and virtual surgical planning in a difficult Bonebridge case. Virtual and Physical Prototyping 14: 53-58.
13. Ansari N, Brown KG, McBride KE, Steffens D, Koh CE, et al. (2019) Accelerating the learning curve in cytoreductive surgery and hyperthermic intraperitoneal chemotherapy using an external mentor model. ANZ journal of surgery 2019.