Introduction

In the fall of 2019, a new administrative team was appointed to the Graduate Nursing Program at a rural university in the Southeast United States (U.S.). The Graduate Nursing Programs included: (a) Master of Science in Population Health (MSN), (b) MSN in Population Health with a concentration in Nursing Education, (c) MSN in Population Health with a concentration in Emergency Management, (d) Post-Master’s Doctor of Nursing Practice (DNP), and a (e) Bachelor-Doctor of Nursing Practice (BSN-DNP) Family Nurse Practitioner (FNP) track, and a

(f) Bachelor-Doctor of Nursing Practice (BSN-DNP) AdultGerontology Acute Care Nurse Practitioner (AGACNP) track. The newly hired administrative team members included a Director of Online and Graduate Nursing Programs and Coordinators for both MSN and DNP programs.

Accreditation is a process widely used to ensure quality in higher education institutions. Nursing programs engage in the accreditation process with the intention of advancing academic standards and enhancing quality. Accreditation serves as a Continuous Quality Improvement (CQI) system which encourages the establishment of trust in higher education through means of authentication. The systematic process promotes achievement of performance standards [1].

The Commision on Collegiate Nursing Education (CCNE) is an accreditation agency, which provides a voluntary process in which nursing programs can seek accreditation in the U.S. and its territories. CCNE provides a peer review process which validates excellence and integrity of nursing programs in accordance with established criteria, standards, and procedures for evaluation [2,3]. The administrative team members discussed current understanding of Standard IV, program effectiveness, of the guidelines established by CCNE.

The team developed a goal to achieve the standards articulated by CCNE and specifically concentrated on the current processes, policies, and outcome data management practices. The team prioritized the collection of evidence and supporting documentation needed to demonstrate compliance with Standard IV, program effectiveness, when writing the required CCNE selfstudy and in providing supportive evidence in a virtual resource room during a CCNE onsite visit.

CCNE defines program outcomes as individual and aggregate data which are specific, measurable, and observable ([4] p. 26, para. 4). Furthermore, the data must be analyzed and used for ongoing program improvement with clearly articulated levels of achievement for students, faculty, and the program. CCNE’s Standard IV, program effectiveness, focuses on assessment and achievement of program outcomes. Moreover, the standard addresses the attainment of established mission, goals, and program outcomes. Program outcomes include student outcomes, faculty outcomes, and other outcomes as identified by CCNE ([4] p. 19, para. 1).

The team worked collaborately as they embarked on the journey of Zero to Hero to complete a baseline assessment of the current policies and processes for collecting internal and external outcome data. To facilitate the baseline assessment, current policies and processes were audited which assisted the team in identifying the problem, establishing benchmarks, setting goals, and creating a quality improvement initiative for collecting, analyzing, and tracking of program outcomes. A gap analysis was used and proved beneficial for the purpose of determining deficiencies between current and desired state. The current state of internal and external data collection was compared to Standard IV key elements which are essential in achieving the desired state of CCNE readiness [5].

Internal data are collected in the graduate nursing department and at the university level by the Office of Institutional Research and Effectiveness. The internal data consists of: (a) program learning outcomes (PLoS), (b) first destination survey, (c) alumni survey of recent graduates, (d) employer survey, and (e) graduate student exit questionnaire/interviews. The internal data is shared at the university level with Institutional Research and Effectiveness (IRE) and used for CQI. The collaborative relationship promoted informed, data-driven decisions and ensured a systematic method to ongoing CQI [5].

External data collected by graduate nursing included key elements from Standard IV. According to CCNE, there are several key elements associated with Standard IV which include (a) a systematic process is used to determine program effectiveness; (b) program completion rates demonstrate program effectiveness; (c) licensure pass rates demonstrate program effectiveness; (d) certification pass rates demonstrate program effectiveness; (e) employment rates demonstrate program effectiveness; (f) data regarding completion, licensure, certification, and employment rates are used, as appropriate, to foster ongoing program improvement; (g) aggregate faculty outcomes demonstrate program effectiveness; (h) aggregate faculty outcome data are analyzed and used, as appropriate, to foster ongoing program improvement; (i) program outcomes demonstrate program effectiveness; and (j) program outcome data are used, as appropriate, to foster ongoing program improvement (CCNE Standards for Accreditation of Baccalaureate and Graduate Nursing Programs, 2018, pp. 19-23). The team used CQI as it is a vital component of the accreditation process.

The CQI concepts guided the ongoing process which was geared toward improvement of the current systems, operations, program outcomes, and attainment of educational accreditation standards. Through the CQI process, improvements were made, the effect of the improvements were evaluated, then the cycle was repeated until the desired outcome was achieved [6]. CCNE, as the chosen accreditation body by the School of Nursing (SON) does not stipulate how data must be obtained, tracked, or analyzed. Appropriate policies, processes, tracking, analyzing, and storage of outcome data became a priority goal for the team. Identified gaps in the current processes were addressed by the team using an innovative approach to the management of outcomes. Brainstorming and creative strategies were explored and developed using Lewin’s Theory of Change.

According to Butts and Rich [7], Lewin’s Change Theory was first introduced and used in the 1940s in industry. The model includes unfreezing, moving, and refreezing. Unfreezing is the act of destabilizing old behaviors in order to unlearn or discard them. Moving allows the change to more acceptable behaviors. Refreezing involves achieving a state of balance. Using Lewin’s theoretical model, the team implemented planned change to bring about improvements in existing policies, processes, and systems. Unfreezing was demonstrated with destabilization of the old behaviors from status quo toward planned change. Movement involved actions to develop new policies, processes, and systems with a shift in behaviors toward a change. The final stage, refreezing, involved a new state of stability and establishment of new policies, processes, and systems [7].

A “Heroic Effort” for Collection, Documentation and Tracking of Program Outcomes

The team developed diverse strategies to achieve CCNE’s Standard IV guidelines. The approach included several steps such as the development of a policy related to tracking of outcomes which included the implementation of an Outcomes and Assessment Committee (OAC), creation of tracking forms reflective of Standard IV’s criteria, and establishment of a structured repository within the University Learning Management System (LMS) for storage of the developed templates and completed outcome data.

The first step was development of a policy which included the establishment of an informed, data-driven decision making committee, development of templates, and validating, securing, organizing, and maintaining program data/outcomes [8]. The aforementioned step assisted the team members in being successful change agents. Olshansky (as cited in Mason et al., 2021) indicated policies serve to provide transparency, offer guidance, and define processes to improve systems within an organization (p. 503) [9]. According to Stoich, a growing number of school systems in the U.S. and internationally have created policies which establish teams dedicated to improving processes and systems. The development and implementation of OAC is an example of a committee, developed by a policy, which is dedicated to improving processes and systems. The committee is inclusive of administrators, program coordinators and faculty members with shared decision making related to processes in the collection and analysis of program outcomes [10].

According to Aburizaizah (2022), as the mandate for excellence in education rises, the demand for CQI increases. CQI is at the forefront globally and a pursuit among higher education institutions. CQI iniatives are inspiring academic leaders to examine the mission, vision, goals, objectives, and academic practices at the program level (Aburizaizah, 2022). OAC, a subcommittee within the graduate nursing program, was intended to discuss, plan, and reflect on the current assessment process, and to develop methods to achieve the desired state. The committee focused on a comprehensive plan which articulated processes within the policy that addressed collecting, analyzing, securing, organizing, and maintaining outcomes for use in continued program improvement. Therefore, the committee ensured program-level outcomes were measured to verify that students are meeting the established benchmarks necessary for accreditation.

Raszewski et al. (2021) indicated that with substantial growth in Doctor of Nursing Practice (DNP) programs, data management practices in nursing programs must be adopted. Data management practices include validating, securing, organizing, maintaining, and storing data. Data storage was emphasized through the use of repositories (Raszewski et al., 2021). To achieve this goal, templates and a repository were developed. Template forms were created by the committee for collection, documentation, and trending of program outcome data. Outcome templates related to program outcomes included end of program, end of program exit interviews, residency/practicum outcomes, program completion rates, program pass rates, admission/ enrollment data, official certification pass rates, and alumni/employer surveys. Templates were also developed focusing on faculty, inclusive of evidence of expertise and documentation of academic/experiential qualifications. Other templates focused on faculty outcomes related to mean teaching effectiveness scores, five-year trended data of mean teaching effectiveness scores, documentation of graduate faculty expertise, including degree specialization, specialty course work, and the required multi-state licensure.

Raszewski et al. (2021) found there was a lack of a systematic and program level approach to data collection and storage in nursing programs. Furthermore, when included, it is limited. Additionally, emphasis was placed on the ability of administrators to preserve data. Perserving data referred to backing up and storing data for ease of access and use in the future. Finally, data sharing through the use of repositories was highlighted (Raszewski et al., 2021). Therefore, an electronic OAC repository was developed and implemented using the University designated LMS for the uploading of outcome templates and completed outcome tracking forms each semester. To facilitate the entire lifecycle of data management practices, the program coordinators were designated as data management stewards, who were responsible for completing and storing the outcome templates in the newly established OAC repository (Raszewski et al., 2021). The program outcomes were analyzed/synthesized, trended, and appropriately documented for tracking purposes.

Conclusion

A baseline assessment revealed a gap between the current state (“Zero”) compared to the desired state (“Hero”) of being in continuous compliance with Standard IV. Effective data management can be achieved through the creation of a robust data management system along with policies and processes (Raszewski et al., 2021). Use of the developed policy and processes such as OAC, templates/forms, and use of the LMS repository proved to be successful as a method to reach CCNE standards. Raszewski and colleagues (2021) further suggested best practice guidelines included a culture of data management and an infrastructure being built for storage solutions via use of a repository. Currently, all graduate nursing programs have appropriate outcome documents within the designated repository and the data are used effectively for CQI purposes. The structured process for collecting, analyzing and tracking data promoted graduate faculty understanding of program outcomes and highlighted accountability in ensuring compliance with Standard IV. In summary, the journey from “Zero to Hero” proved to be highly effective as faculty and administrators are fully engaged in the CQI process.

References

1. Aburizaizah SJ (2022) The role of quality assurance in Saudi higher education institutions. International Journal of Educational Research Open 3: 100127.
2. Association of Colleges of Nursing (2023) Mission and purposes. Commission on Collegiate Nursing Education Accreditation Process/ Who We Are.
3. Acevedo-De-los-Rios A, Rondinel-Oviedo DR (2022) Impact, added value and relevance of an accreditation process on quality assurance in architectural higher education. Quality in Higher Education 28: 186204.
4. Commission on Collegiate Nursing Education (2018). Standards for accreditation of baccaularate and graduate nursing programs. American Association of Colleges of Nursing.
5. Galura S, Warshawsky N (2022) Initial evaluation of a Doctor of Nursing Practice – Executive track program: The development of a three-year process to implement the new AACN Essentials. J Prof Nurs 42: 276-280.
6. O’Donnell B, Gupta V (2023) Continuous Quality Improvement. NIH National Library of Medicine. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing.
7. Butts JB, Rich KL (2018) Philosophies and theories for advanced nursing practice (3^rd Edition.). Jones and Bartlett Learning.
8. Raszewski R, Goben AH, Bergren MD, Jones K, Ryan C, et al. (2021) A survey of current practices in data management education in nursing doctoral programs. Journal of Professional Nursing 37: 155-162.
9. Olshansky EF (2021) Policy and politics in nursing academia. In Policy & politics in nursing and healthcare. (8^th Edition.). Elsevier.
10. Stosich EL (2021) Are we an advisory board or a decision making entity?: Teachers’ involvement in decision making in instructional leadership teams. Leadership and Policy in Schools.