Aim of the article is to help researchers carrying out studies in the field of dentistry with sound methodology which will give more accurate and precise result. Appropriate precaution has to be taken to prevent loss of time, money and other resources and most importantly poor quality of the study which will affect the concreteness of the result.
Conclusion: Validity and reliability are 2 important aspects of any measurement for accuracy. Validity refers to accurately measuring what it claims to measure while reliability is an assessment of reproducibility and consistency of a measurement or instrument. Validity and reliability are threatened by different forms of bias. There is a proper technique to achieve validity and reliability. Precautions should be taken to ensure sound methodology.The article highlights different aspects of validity.
Keywords:Validity; Accuracy; Research
Source of Funding
The study was self-funded by the authors and their institution or No external funding, apart from the support of the author’s institution, was available for this study.
Conflict of interest
Dr. /Prof. Sandhya Jain have no conflict of interest.
Scientific Rationale for Study
To equip clinician and researchers with an understanding of basic steps or quality results. Relevant dental examples are also provided in the article for easy understanding.
This is a review article in which Reliability and validity which are two different aspect of accuracy are discussed in detail. This article helps in providing proper knowledge about methodology used to check the validity and accuracy of the result obtained from any study.
Clinical study is done with sound methodology without any kind of bias will yield accurately precise results. Studies are only useful if they are based on methodological appropriateness.
Accuracy is the closeness of any assessment to its actual value. Validity and reliability are 2 important aspects of any measurement for accuracy.Validity refers to accurately measuring what it claims to measure while reliability is an assessment of reproducibility and consistency of a measurement or instrument. For any instrument to be valid it should measure what it intends to measure.E.g.:- If an instrument is used to measure caries, it should measure caries and not the enamel hyperplasia.
If a machine weighs 40 Kg instead of 50 Kg weight, the validity of instrument is questionable(Table 1). But if instrument measures 40 Kg repeatedly it is a reliable instrument.
There are certain precautions we should take into considerations while performing study. If we do not take these precautions it will incorporate the bias and will adversely affect the quality of the result.
Types of Validity
External validity is about generalization. If a study produces similar results among different kinds of populations, the result is said to be generalized. If the population behaves differently (lack of population validity) or environment influences behaviour (lack of ecological validity) the result cannot be generalized. Study done at high altitude may be different from that done at low altitude. Similarly, maturational time of adolescents may be different for different countries for various reasons.
Internal validity ensures good experimental design to produce accurate results. Appropriate construct and content of the questionnaire, randomization, blinding, matching ensure sound methodological concept for concrete results. Results need to be interpreted in an objective and critical way, before assessing their implications and drawing conclusions.
Types of Internal Validity
- Content validity
- Criteria validity.
- Face validity (Logical validity) and Apparent validity
Face validity (Logical validity) and apparent validity
Face validity should be judged by subjective assessment and relevance of the questionnaire to the participants, face validity is determined by a review of the items and not through the use of statistical analysis.
Face validity is a measure of how representative a research project is ‘at face value’ and either appears to be a good project.
For example: if we have to collect the sample of handicapping malocclusion and if there are cases with normal occlusion and mild malocclusion are also included in the study we can visualize and say that normal occlusion and mild malocclusion are not fit for the study.
Similarly, if we have to collect the sample of aggressive periodontitis and there comes a case of marginal gingivitis or periodontitis,we can easily differentiate between them for face validity.
Whereas in Apparent validity, question is generated in consultation with opinion of experts and subjects themselves. All questions should be logical.
It consists of demonstrating that the measure is related to other similar measures of the similar characteristics and not related to other characteristics. It defines how well a test or experiment measures up-to its claim and degree to which instrument reflects the concepts, how well the instrument is constructed.
Convergent validity was conducted by correlating scores between Dental Fear Scale (DFS), total score and global question (do you fear going to dental office). Convergent validity is a subtype of construct validity, it determines whether 2 similar constructs corresponds with each other or not while discriminant validity shows differentiation between 2 dissimilar groups that can be easily differentiated. Convergent validity shows high correlation whereas discriminant validity shows low correlation coefficient.
Convergent validity of Dental Fear Scale (DFS) was calculated by correlating scores between global question and Dental Fear Scale (DFS) (Table 2).
This type of validity tests concepts that have no relationship to each other.
Dental fear scale shows discriminant validity between children with negative and no negative dental experience at childhood. (Table 3).
Most dimensions of concept are included to evaluate itsmaterial/advice is sought from experts, pilot study/factorial analysis. Content validity ratio refers to extent to which a measure represents all facets of a given construct. An element of subjectivity to determine content validity, in which a degree of agreement is requires about what a particular personality trait such as extraversion represents. The gain of a high content validity prevented by a disagreement about a personality trait.
In order to assess the content validity of Caries Assessment Spectrum And Treatment (CAST) Index in Indian population, twenty Subject Matter Experts (five each from Conservative, Paediatrics, Public health and Oral medicine) were asked to determine if each of codes (0-9) of cast index were essential or non essential and also justify the reason for it by writing their comments for each code(Table 4).
Formula for Content Validity Ratio (CVR),
CVR= (ne-N/2) / (N/2)
ne-no of dentist with post-graduate degree qualification indicating “essential”.
N – Total no. of dentist with post-graduate degree qualification.
(0.5 and above -Acceptable; 0.4 and below – Not acceptable;CVR score 1 is highly acceptable). On the basis of CVR score code 1, 4, 9 were found to be non essential for Indian population.
It is a best measure of validity.
It assess whether a test reflects certain set of abilities like concurrent validity and predictive validity.
It compares result of two things (new and old test) at same time. Comparing newly designed test with already existing gold standard at the same time for concurrent validity to check whether new test is valid or not.
It is a measure of how well a test predicts. It involves testing, subjects and comparing them at some point in future. For example, if a student appearing in medical entrance excels we can predict that he will perform better in future also.
Validation through ROC (Receivers Operating Curve) / Sensitivity and Specificity
Receiver Operating Characteristic(ROC)
It is a graphical plot that represents performance of a binary classifier system as its discrimination threshold is varied. This curve plots the True Positive Rate (TPR) against the False Positive Rate (FPR). The true-positive rate is also known as sensitivity, recall or probability of detection whereas false-positive rate is also known as the probability of false alarm orfall-outand can be calculated as (1 -specificity).
Receiver operating characteristic analysis provides tools which select possibly optimal models. From the cost context or the class distribution it also discards suboptimal ones independently (and prior to specifying). In a direct and natural way this analysis is related to cost/benefit analysis of diagnostic decision making.
Sensitivity measures the proportion of positives that are correctly identified (e.g., the condition which correctly identifies the percentage of sick people who are having disease).
Specificity measures the proportion of negatives that are correctly identified as such (e.g., the measurement of healthy people who are correctly identified as not having the condition).
To assess the sagittal relationship between maxilla and mandible with accuracy and reproducibility a new cephalometric measurement introduced, named the W angle.
Receiver Operating Characteristic Curve use to examine the sensitivity and specificity of W angle to discriminate between 3 different skeletal pattern group(Class I, Class II, Class III).
ROC showed that W angle less than 510has 96% sensitivity and 90% specificity for discriminating the class II group from class I group. A w angle greater than 560 has 95% sensitivity and 98% specificity for discriminating the class III group from class I group (Table 5).
Therefore ROC shows that;
Between class I & class II is 510
Between class I & class III is 560
Two versions of Community periodontal Index and Treatment Need(CPITN) partial and full version were validated using ROC Curves in Brazilian population .
Reliability does not depend on validity while validity depends on reliability also. It should weigh accurately to be a valid instrument. Methodology of any research should be full proof to give accurate result. Any kind of bias in the study will distort the findings. If selection bias is not prevented by allocation concealment /randomization, while allocating patients to different study groups the result will be altered. Similarly, if there is no blinding and the investigator / analyser know about the groups he is measuring there may be bias affecting the final result. Thus method and methodology both should be valid to produce accurate result. This will reduce systemic error.
Accuracy is lack of errors. For any result to be accurate it should be free from any kind of bias and at the same time instrument / method should be a gold standard method or a valid method. Similarly random error should be minimised to increase the precision. Precision is opposite to random error. Thus both validity and reliability are important aspect of accuracy. Systemic errors are caused by different forms of bias affecting the mean value. Random errors do not affect the mean value but affect reliability.
Validity and reliability are threatened by different forms of bias . For example – Selection bias, recall bias and non-response bias etc.
For equal distribution of confounding factors there should be perfect matching between experimental and control group. Proper selection method (Randomization) should be followed to allocate patients to each group. The method of allocation should not be known to the researcher (Allocation concealment). Blinding of subject, investigator and analyser helps in controlling different forms of bias. E.g., Behaviour bias. Appropriate steps should be taken to avoid recall bias, non response bias etc. All these precautions ensure sound methodology(Table6).
There is a proper technique to achieve validity and reliability .
To perform any research study often we have to create valid and reliable questionnaire / test. There are certain guidelines to carefully construct and develop its contents so that most dimensions of concept are included by seeking advice from experts[11-13]. It should be well constructed to reflect the concept. Its results should correlate with gold standard method. High Positive correlation shows good convergent validity. Similarly discriminate validity shows low correlation between 2 different groups with theoretically different concepts.
A question developed in a different country may not be a valid measure in our country. Just because questionnaire has been piloted or used in previous studies does not mean it is valid or reliable. Questionnaire should be cross culturally adopted and validated.
Qualities of any research depends on various factors like validity of instrument, intra and inter observer reliability, presence of known and unknown confounding factors, presence of bias etc.
To enhance quality of the study, researchers should take precautions in designing studies to reduce bias, which can be achieved through perfect matching, blinding and adapting appropriate selection method.
Questionnaire should have proper content validity and should adapt according to the need of study.Sound and concrete results canonly be produced with appropriate method and methodology.
- Jain S, Jain S, Punyani PR, JainD (2015) Basics of interpreting results. International Dental & Medical Journal of Advanced Research 1: 1-4.
- Saunders B, Trapp R(1998)Basics and Clinical Biostatistics. (2nd edition), United states of America: Prentice-Hall International Inc, USA. Pg No:58-59.
- Phansopkar S, Hegde-Shetiya S, Devadiga A, Agrawal D, MahuliA, et al. (2015) Face and Content Validation of Caries Assessment Spectrum and Treatment Index among Few Subject Matter Experts in India.International Journal of Dental Health Concerns 1: 13-18.
- Jain S, Sharma N, Jain D(2015) Basic fundaments of designing a quality research. Journal of Advanced Medical and Dental Sciences Research 3: 88-95.
- wikipedia.org (2017)Receiver operating characteristic.
- Bhad W, NayakS, Doshi U (2011) A new approach of assessing sagittal dysplasia: the W angle. The European Journal of Orthodontics35: 66-70.
- Bassani D, SilvaC, Oppermann R (2006) Validity of the Community Periodontal Index of Treatment Needs’ (CPITN) for population periodontitis screening. Cadernos de Saúde Pública22: 277-283.
- Jain S, DebbarmaS, JainD(2016) Bias in Dental Research / Dentistry. Annals of International Medical and Dental Research2: 2395-2814.
- FlemingPS, Johal A(2010) Self-ligating brackets in orthodontics. A systematic review. The Angle Orthodontics80: 575-584.
- Jain S, Sharma N (2016) Guideline for systematic reviews. International Dental and Medical Journal of Advance Research2: 1-10.
- Jain S, Dubey S, Jain S (2016) Designing and validation of a questionnaire. International dental & medical journal of advanced research 2:1-3.
- Edwards P(2010)Questionnaires in clinical trials: guidelines for optimal design and administration. Trials11.
- Kazi A, Khalid W (2012)Questionnaire designing and validation. Journal of Pakistan Medical Associatio 62: 514-516.
|Instrument||Readings(in Kg)||Valid / reliable||Inference|
|Instrument A||50||50||50||Reliable and Valid||Precise and Accurate|
|Instrument B||40||40||40||Reliable but Not Valid||Imprecise & Inaccurate|
|Instrument C||40||45||50||Not reliable & Not Valid||Imprecise and Inaccurate|
Table 1: Validity and reliability of three instruments.
|Scale||DFS||Avoidance||Physiological arousal||Fear of specific
|Fear of specific Stimuli/ situation||0.95*||0.71*||0.62*|
Table 2: Convergent validity was calculated as follows.
|DFS total||32.62 (11.39)||42.04 (15.56)||< 0.001||9.426||0.70|
|Avoidance||10.45 (3.72)||13.56 (5.96)||< 0.001||3.104||0.64|
|Physiological arousal||7.58 (2.79)||9.36 (3.98)||< 0.001||1.776||0.54|
|Fear of specific
|14.58 (6.33)||19.13 (7.50)||< 0.001||4.545||0.66|
Table 3:Negative dental experience at childhood.
|Sound||0||No visible evidence of a distinct carious lesion is present.||0.6||Acceptable|
|Sealant||1||Pits and/or fissures are at least partially covered with a sealant material.||0.2||Not Acceptable|
|Restoration||2||A cavity is restored with an (in) direct restorative material.||0.6||Acceptable|
|Enamel||3||Distinct visual change in enamel only. A clear caries related discoloration is visible, with or without localized enamel breakdown.||0.6||Acceptable|
|Dentine||4||Internal caries-related discoloration in dentine. The discoloured dentine is visible through enamel which may or may not exhibit a visible localized breakdown of enamel.||0.4||Not Acceptable|
|5||Distinct cavitations into dentine. The pulp chamber is intact.||1||Acceptable|
|Pulp||6||Involvement of the pulp chamber. Distinct cavitations reaching the pulp chamber or only root fragments are present.||1||Acceptable|
|7||A pus containing swelling or pus releasing sinus tract related to a tooth with pulpal involvement.||0.5||Acceptable|
|Lost||8||The tooth has been removed because of dental caries.||0.9||Acceptable|
|Other||9||Does not correspond to any of the other descriptions.||-0.2||Not Acceptable|
Table 4: CAST: Caries Assessment Spectrum and Treatment codes with content validity ratio.
|Between 510-560||Class I|
|Less than 510||Class II|
|More than 560||Class III|
Table 5: ROC Curve Showing W angle at Different Groups.
|· Gold standard method/Valid method||· Sample size estimation
· Random sequence generation
· Allocation containment
· Intention to treat analysis
· Reporting of withdrawal
Table 6: Criteria for assessment of validity of any research methodology.
Citation: Jain S (2017) Accuracy in Dental Research. Dentistry Adv Res 2016: GDSC-118.