1.       Abstract

 Quality is a common point of discussion in the preclinical evaluation process of new drug candidates, however this can be a very subjective topic. While there is no universally accepted definition of “Quality”, under the US Food & Drug Administration’s Good Laboratory Practice Guidelines it is most often used to describe study performance, scientific validity, reliability and rigor, and last but not least-documentation. It can be a challenge for the CRO industry to fully appreciate the practical constraints of operating in a typical preclinical research setting as it relates to quality and quality management systems. This overview highlights some of the basic issues defining “Quality” with respect to GLP-compliant research generated as part of the drug development process.

 Professor Max Baur [1], the Vice-President for Science and Research at the University of Bonn, is quoted as stating, “The definition of quality is no guarantee for excellence, but surely a strategic measure on the way to reach excellence.” “Quality Research” most commonly refers to the scientific process encompassing all aspects of study design, data collection, and reporting; in particular, it pertains to the judgment regarding the match between the methods and questions, selection of subjects, measurement of outcomes, and protection against systematic bias, nonsystematic bias, and inferential error [2-4]. Principles and standards for the parameters related to quality research design and conduct were a primary impetus for the development of the Food, Drug & Cosmetics Act (21 U.S.C. Chapter 13, et seq.), it’s Good Laboratory Practice Guidelines (21 CFR §58) and subsequently to the US Environmental Protection Agency’s Toxic Substances Control Act of 1976 (15 U.S.C. §2601 et seq. ) as well as the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA; 7 U.S.C. §136 et seq.) which gave the EPA the authority to require reporting, record-keeping and testing requirements, and restrictions relating to chemical substances and/or mixtures as well as pesticides. Under a 2002 revision, Section 408 of the Federal Food, Drug, and Cosmetic Act (FFDCA) authorizes EPA to set tolerances, or maximum residue limits, for pesticide residues on foods (21 U.S.C. §301 et seq). Under these federal statutory requirements, the participants of industry and academia have established concepts of quality research commonly found in texts, reports, essays, and guides to research design and methodology. Comparing research methods that are primarily designed to gather qualitative data to those that are primarily designed to gather quantitative data, parallel assessments for quality can be framed in terms of credibility (parallels with internal validity), transferability (parallels with external validity), dependability (parallels with reliability), and confirmability (parallels with objectivity) [2,5].

 First and fore most every Contract Research Organization (CRO) must develop processes that establish, maintain, and develop a “Culture of Quality”. According to this culture is one in which everyone in the facility [6], not just the Quality Assurance Unit is responsible for the quality of any GxP-regulated studies. A central feature of all CROs is that each employee or team of operational staff is both a customer of and supplier to other workers in the facility: they collectively form a chain of internal customers and suppliers. It is the responsibility, therefore, of each operational unit to ensure the quality of their own work. The emphasis is on ensuring that things are ‘done right the first time’. When they are not completed appropriately then the process that has led to an unsatisfactory output is analyzed in order that corrections can be made in the process to ensure that the problem does not arise again (i.e. CAPA, Lean Six Sigma, etc.). A company with a true and invested quality culture would not need to rely on checks of the final report, as any errors would have been corrected well before that point in time. Reliance on checks on the final report has the potential to shift responsibility away from those involved in the initial stages of data generation and documentation. According to “Quality” scientific research is a process (checklist) [7], comprising several steps that attempt to ensure the credibility, applicability, consistency and neutrality of the results. The checklist defines a standard, which is agreed by experts, and may vary depending on whether one is using a quantitative or qualitative approach. The criteria to assess “Quality” includes: the internal validity of the results (context, sample size, power calculation), external validity (ecological generalizability, verified predicted relationships, etc.), reliability (consistency if replicated), replicability (can others reproduce the results?), and objectivity [4,5,7,8]. Within a CRO, standardi6zed processes are established and the Test Facility Management encourages every researcher to follow them with rare exceptions.

 The National Research Council and others (NIDDR) [5,9-11] have described standards that shape scientific understanding and that are frequently used to frame the discourse on “Quality Research”. This has led to the term scientifically based research being used in some settings to address research quality. Frequently mentioned standards for assessing the quality of research include the following:

·         Pose a significant, important question that can be investigated empirically and that contributes to the knowledge base.

·         Link study objectives to relevant theory and regulatory goals.          

·         Apply protocol methods that best address the research questions of interest.

·         Base research on clear chains of inferential reasoning supported and justified by a complete coverage of the relevant literature.

·         Provide the necessary information to reproduce or replicate the study from established standards of reporting (i.e., clear, cogent, complete).

·         Ensure the study design, methods, and procedures are sufficiently transparent and ensure an independent, balanced, and objective approach to data interpretation.

·         Provide sufficient description of the sample, the intervention, and any comparison groups.

·         Use appropriate and reliable conceptualization and measurement of variables.

·         Evaluate alternative explanations for any findings.

·         Assess the possible impact of systematic bias.

·         Submit research to a peer-review process. Adhere to quality standards for reporting (i.e., clear, cogent, complete)

 While there is no consensus on a specific set or algorithm of standards that will ensure quality research, the more research studies are aligned with or respond to these 11 principles, the higher the quality of the research [4,12]. This suggests that achieving only one or two standards is typically insufficient to assert “Quality”. For example, some suggest that while standards such as peer review and standardized reporting are important benchmarks, research should not be judged solely by whether or not it is published in the leading journals [2]; in fact, the majority of preclinical research data for a given drug candidate are never published.

 According to the American Board of Internal Medicine the research environment for training investigators must be comprehensive and include: [13] 

·         Adequate funding by the Test Facility Management for research to include appropriate institutional commitment for technical/research staff training.

·         A critical mass of productive researchers who can serve as mentors and professional colleagues to new staff is needed. While acknowledged that the CRO is not an academic institution, the “Productive Researcher” can be defined by the number of publications in peer-reviewed journals, the quality of the journals in which research is published, frequency of citation of scientific work in the literature. Other valid source that can be used to assess the relative productivity of the study director is the participation in professional organizations, such as the Society of Toxicology (SOT), Society of Toxicological Pathologists (STP), American College of Toxicology (ACT), as well as the Safety Pharmacology Society.

·         Training includes a broad curriculum with well-defined goals and objectives. New staff should have educational experiences including formal course work applicable to their preparation for research careers unless this didactic experience has already occurred as part of an advanced degree or related work experiences.

·         Inclusion of the following 5 criteria should be included as part of a quality research group at a CRO:

·         Critical interpretation of the applicable regulatory guidelines that help to understand research methods, including design of standard regulatory-required safety studies

·         Training in basic data analysis and the intended use of appropriate biostatistics and/or medical informatics.

·         Understanding professional and research ethics.

·         Monitoring ongoing research projects within the facility (protocol, data management, quality control, computer packages, clinical trials, global and regional regulations, dissemination of information).

·         Regularly scheduled “In House” training sessions

Harvey & Stenseker (2008) [14] have proposed that a responsive quality culture is governed primarily by the external demands of the study Sponsor and the respective regulatory agency (FDA, EPA, EMeA, etc.). As such the Test Facility takes a positive approach to opportunities and seeks to share good practice, but tends to view quality-related activities and strategies as a solution to externally-driven problems or challenges. A reactive quality culture is driven primarily by compliance and accountability that seeks opportunities for reward. The Test Facility must establish norms, good internal practices and quality that are embedded as part of daily practice and professional conduct. Although Harvey and Stensaker recognized that most institutions will embody a number of these characteristics they argue that these differential orientations will result in very different approaches to quality activities [14].

In an attempt to characterize or define “Quality” research under statutory control of federal regulating agencies we propose the following:

·         The quality of science conducted at a “Work for Hire” facility (i.e. CRO) is based on implications and recommendations of Study Directors with applicable background, education, and experience. The science conducted should be logical, warranted by the limited amount of data that may be available at the time of study development, and explained thoroughly, with appropriate caveats.

·         The research team derives implications and may develop recommendations based on the research findings. A high-quality study thoroughly explores the implications of its findings. It examines where new knowledge and old knowledge are congruent and where they are not; it examines whether existing theories and conceptual frameworks have been strengthened or must be modified.

·         Per the GLPs the study director has overall responsibility for the technical conduct of the study, as well as for the interpretation, analysis, documentation and reporting of results, and represents the single point of study control. As the “Single point of control”, a recommended course of action is a highly accountable step for Study Directors.

·         The actions of study Sponsors and Regulatory Reviewers, relying on the data from those studies, may affect the resources or well-being of many millions of individuals-even entire nations or regions. The Study Director cannot take this duty lightly.

·         The Study Director recommendations must follow logically from a study’s findings and implications and be strongly supported by them.

·         The Study Director’s recommendation for the course of action must include caveats to help ensure that it is not applied to inappropriate cases or with unrealistic expectations.

·         The data documentation at the testing facility must be accurate, understandable, clearly structured, and temperate in tone.

·         Documentation standards should be applied to paper-based documents, graphic and tabular presentations of data since under regulatory standards the way in which research is documented is critical.

·         Accuracy is particularly salient as a prima facie indicator of the quality and credibility of research. In instances where there is significant variability or uncertainty, it is important for the Study Director to indicate the confidence with which one should regard the accuracy of what is presented, as well as understand whether the variability is expected.

·         High-quality documentation should make a study understandable to its intended audiences. The report text should be straightforward and precise with necessary technical terms defined and all acronyms defined.

·         The study report must augment textual descriptions with graphical or pictorial elements to help explain complex and novel ideas, and should include appropriate references to related issues in the literature appearing in peer-reviewed scientific journals.

·         High-quality study reports must be temperate or neutral in tone. It should be neither so flat as to appear unengaged, nor so emotional as to appear partisan or biased. Almost all CRO study reports are relevant to two intersecting sets of discussions: one among Sponsor and Sponsor Monitors, and another among regulatory decision makers (i.e. FDA, EPA). A temperate tone is best suited to both communities, as well as to wider audiences, such as the general public.

·         The contracted study should be compelling, useful, and relevant to stakeholders and decision makers. A high-quality study makes an impression and compels respectful attention; it cannot be ignored or dismissed by those working in the area it addresses.

·         The contract laboratory conducts regulatory-based research that focuses on real-world problems, practical issues, and regulatory-required study designs. Therefore, a high-quality study must be not only interesting but also useful. It must contribute to the understanding of the safety profile of a therapeutic as a means to gauge possible risks to humans.

·         A CRO must establish widely respected operational standards that remain independent of Sponsor pressures. The Study Director is involved in frequent interactions with scientists and Sponsors, who are recognized as an important source of expertise; but because the CRO conducts its research under regulatory control and review, a high-quality study must also be independent of, and relevant to, other stakeholders in the policy domain [15].

·         Interaction with a broad set of stakeholders can help to ensure the relevance of the research and the practicality of its recommendations. Independence refers to intellectual independence, not financial independence.

·         In part to obviate any inference that its research may be biased by its relationship to Sponsors, the CRO must strictly adhere to GLP compliance. By administrative policy the laboratory must institute strong policies and mechanisms to ensure intellectual independence. The CRO must have a rigorous research quality assurance process, its Study Directors commit themselves to seeking and using critical assessments of their work in all phases, and the CRO must include routine peer review in the conduct of the study.

·         All CROs face the challenge of addressing both scientific perspectives and regulatory perspectives. The research laboratory seeks balance among these two competing perspectives by treating both fairly, portraying them accurately, and weighting them according to merit. Quality science is not dictated by regulations but rather the regulatory agencies provide the requisite support systems and processes by which quality science is advanced. 

Good science, sound science is not guaranteed by the presence or actions of a Quality Assurance Unit (QAU) or data quality review teams. Those internal groups serve as an important support mechanism to the established scientific methodologies, valid and reliable laboratory systems and equipment, and the structured foundations established through the protocol development and study conduct stages. No single aspect of study conduct controls “Quality”. Scientific quality is established or demonstrated by the quality of the data and study report. The QAU or data quality teams at the CRO provide an objective, independent, internal peer-review that helps to establish, for the Study Director, that the protocol and operational processes that were conducted outside the direct observation of the Study Director were followed. The real demonstration of “Quality” is in the final product of study conduct-the “Deliverable”-the written final report. Without a cogent, complete and accurate report of study conduct, data analysis, and a trained Study Director’s interpretation, the statutory mandate of “Study Replication” is not possible. The integration of all departments and personnel involved in contractual study conduct generally comes through the Report Writing Department, working closely with the assigned Study Director. The efforts of the Study Director and operational staff conducting preclinical research will not be meaningful if the final report does not meet regulatory standards of excellence. A well-written report based on verifiable study conduct data serves best to ensure “Quality” within any research environment, but this is most critical in the CRO producing data on behalf of its Sponsors and the regulatory agencies charged with assuring safety and efficacy of the products it approves for the marketplace [16]. 

The conceptual framework that guides the pharmaceutical industry includes all aspects of the research conduct as well as the development of self-improvement strategies for preclinical safety, efficacy, toxicity, and policy management. In the end, the industry must police itself to be considered valuable to the research process. As technological improvements evolve, the conceptual relationships of organizational systems relevant to preclinical screening of novel therapeutics will be tested, and new terminology and testing protocols (strategies) will emerge or evolve from existing processes and those processes will be expected to more precisely drive the industry forward to address new scientific needs. Although there are some core principles established by drug regulating agencies (FDA, EPA, DEA, etc.) that are applied across the pharmaceutical industry as a whole, the conceptual frame-of-reference for new drug approvals demands a broader scope of scientific excellence from the industry itself which must provide information for improvement strategies that work best in the complex, adaptive drug discovery and development arena.