Dentistry: Advanced Research

Volume: 2017; Issue: 1
7 Aug 2017

Accuracy in Dental Research

Original Article

Sandhya Jain*

Department of Orthodontics and Dentofacial Orthopedics, Government College of dentistry, India

*Corresponding author: Sandhya Jain, Department of Orthodontics and Dentofacial Orthopedics, Government College of dentistry, Indore, Madhya Pradesh, India, Tel: +91 9425045455; E-mail: researchorthodontics@gmail.com; orthogdcindore@gmail.com

Received Date: 09 February, 2017; Accepted Date: 03 April, 2017; Published Date: 10 April, 2017

1462985612_pdfs

Abstract

Introduction

References

Tables

Suggested Citation

Abstract

 

Aim of the article is to help researchers carrying out studies in the field of dentistry with sound methodology which will give more accurate and precise result. Appropriate precaution has to be taken to prevent loss of time, money and other resources and most importantly poor quality of the study which will affect the concreteness of the result.

 

Conclusion: Validity and reliability are 2 important aspects of any measurement for accuracy. Validity refers to accurately measuring what it claims to measure while reliability is an assessment of reproducibility and consistency of a measurement or instrument. Validity and reliability are threatened by different forms of bias. There is a proper technique to achieve validity and reliability. Precautions should be taken to ensure sound methodology.The article highlights different aspects of validity.

 

Keywords:Validity; Accuracy; Research

 

Source of Funding

 

The study was self-funded by the authors and their institution or No external funding, apart from the support of the author’s institution, was available for this study.

 

Conflict of interest

 

Dr. /Prof. Sandhya Jain have no conflict of interest.

 

Clinical Relevance

 

Scientific Rationale for Study

 

To equip clinician and researchers with an understanding of basic steps or quality results. Relevant dental examples are also provided in the article for easy understanding.

 

Principal finding

 

This is a review article in which Reliability and validity which are two different aspect of accuracy are discussed in detail. This article helps in providing proper knowledge about methodology used to check the validity and accuracy of the result obtained from any study.

 

Practical Implication

 

Clinical study is done with sound methodology without any kind of bias will yield accurately precise results. Studies are only useful if they are based on methodological appropriateness.

Introduction

 

Accuracy is the closeness of any assessment to its actual value. Validity and reliability are 2 important aspects of any measurement for accuracy.Validity refers to accurately measuring what it claims to measure while reliability is an assessment of reproducibility and consistency of a measurement or instrument. For any instrument to be valid it should measure what it intends to measure.E.g.:- If an instrument is used to measure caries, it should measure caries and not the enamel hyperplasia.

 

If a machine weighs 40 Kg instead of 50 Kg weight, the validity of instrument is questionable(Table 1). But if instrument measures 40 Kg repeatedly it is a reliable instrument.

 

There are certain precautions we should take into considerations while performing study. If we do not take these precautions it will incorporate the bias and will adversely affect the quality of the result.

 

Types of Validity

 

External Validity

 

External validity is about generalization. If a study produces similar results among different kinds of populations, the result is said to be generalized. If the population behaves differently (lack of population validity) or environment influences behaviour (lack of ecological validity) the result cannot be generalized. Study done at high altitude may be different from that done at low altitude. Similarly, maturational time of adolescents may be different for different countries for various reasons.

 

Internal validity

 

Internal validity ensures good experimental design to produce accurate results. Appropriate construct and content of the questionnaire, randomization, blinding, matching ensure sound methodological concept for concrete results. Results need to be interpreted in an objective and critical way, before assessing their implications and drawing conclusions[1].

 

Types of Internal Validity

 

  • Construct

– Convergent

– Discriminant

  • Content validity
  • Criteria validity.
  • Face validity (Logical validity) and Apparent validity

 

Face validity (Logical validity) and apparent validity

 

Face validity should be judged by subjective assessment and relevance of the questionnaire to the participants, face validity is determined by a review of the items and not through the use of statistical analysis.

 

Face validity is a measure of how representative a research project is ‘at face value’ and either appears to be a good project.

 

For example: if we have to collect the sample of handicapping malocclusion and if there are cases with normal occlusion and mild malocclusion are also included in the study we can visualize and say that normal occlusion and mild malocclusion are not fit for the study.

 

Similarly, if we have to collect the sample of aggressive periodontitis and there comes a case of marginal gingivitis or periodontitis,we can easily differentiate between them for face validity.

 

Whereas in Apparent validity, question is generated in consultation with opinion of experts and subjects themselves. All questions should be logical.

 

ConstructValidity

 

It consists of demonstrating that the measure is related to other similar measures of the similar characteristics and not related to other characteristics. It defines how well a test or experiment measures up-to its claim and degree to which instrument reflects the concepts, how well the instrument is constructed[2].

 

Convergent Validity

 

Convergent validity was conducted by correlating scores between Dental Fear Scale (DFS), total score and global question (do you fear going to dental office). Convergent validity is a subtype of construct validity, it determines whether 2 similar constructs corresponds with each other or not while discriminant validity shows differentiation between 2 dissimilar groups that can be easily differentiated. Convergent validity shows high correlation whereas discriminant validity shows low correlation coefficient.

 

Convergent validity of Dental Fear Scale (DFS) was calculated by correlating scores between global question and Dental Fear Scale (DFS) (Table 2).

 

Discriminant Validity

 

This type of validity tests concepts that have no relationship to each other.

 

For example:

 

Dental fear scale shows discriminant validity between children with negative and no negative dental experience at childhood. (Table 3).

 

Content Validity

 

Most dimensions of concept are included to evaluate itsmaterial/advice is sought from experts, pilot study/factorial analysis. Content validity ratio refers to extent to which a measure represents all facets of a given construct. An element of subjectivity to determine content validity, in which a degree of agreement is requires about what a particular personality trait such as extraversion represents. The gain of a high content validity prevented by a disagreement about a personality trait.

 

For example:

 

In order to assess the content validity of Caries Assessment Spectrum And Treatment (CAST) Index in Indian population, twenty Subject Matter Experts (five each from Conservative, Paediatrics, Public health and Oral medicine) were asked to determine if each of codes (0-9) of cast index were essential or non essential and also justify the reason for it by writing their comments for each code[3](Table 4).

 

Formula for Content Validity Ratio (CVR),

 

CVR= (ne-N/2) / (N/2)

 

ne-no of dentist with post-graduate degree qualification indicating “essential”.

N – Total no. of dentist with post-graduate degree qualification.

 

(0.5 and above -Acceptable; 0.4 and below – Not acceptable;CVR score 1 is highly acceptable). On the basis of CVR score code 1, 4, 9 were found to be non essential for Indian population.

 

Criteria Validity

 

It is a best measure of validity.

 

It assess whether a test reflects certain set of abilities like concurrent validity and predictive validity.

 

Concurrent validity

 

It compares result of two things (new and old test) at same time. Comparing newly designed test with already existing gold standard at the same time for concurrent validity to check whether new test is valid or not.

 

Predictive validity

 

It is a measure of how well a test predicts. It involves testing, subjects and comparing them at some point in future. For example, if a student appearing in medical entrance excels we can predict that he will perform better in future also.

 

Validation through ROC (Receivers Operating Curve) / Sensitivity and Specificity

 

Receiver Operating Characteristic(ROC)

 

It is a graphical plot that represents performance of a binary classifier system as its discrimination threshold is varied. This curve plots the True Positive Rate (TPR) against the False Positive Rate (FPR). The true-positive rate is also known as sensitivity, recall or probability of detection whereas false-positive rate is also known as the probability of false alarm orfall-outand can be calculated as (1 -specificity)[5].

 

Receiver operating characteristic analysis provides tools which select possibly optimal models. From the cost context or the class distribution it also discards suboptimal ones independently (and prior to specifying). In a direct and natural way this analysis is related to cost/benefit analysis of diagnostic decision making[5].

 

Sensitivity

 

Sensitivity measures the proportion of positives that are correctly identified (e.g., the condition which correctly identifies the percentage of sick people who are having disease)[5].

 

Specificity

 

Specificity measures the proportion of negatives that are correctly identified as such (e.g., the measurement of healthy people who are correctly identified as not having the condition)[5].

 

Example:

 

To assess the sagittal relationship between maxilla and mandible with accuracy and reproducibility a new cephalometric measurement introduced, named the W angle[6].

 

Receiver Operating Characteristic Curve use to examine the sensitivity and specificity of W angle to discriminate between 3 different skeletal pattern group(Class I, Class II, Class III).

 

ROC showed that W angle less than 510has 96% sensitivity and 90% specificity for discriminating the class II group from class I group. A w angle greater than 560 has 95% sensitivity and 98% specificity for discriminating the class III group from class I group (Table 5).

 

Therefore ROC shows that;

 

Cut-off points

Between class I & class II is 510

Between class I & class III is 560

 

Two versions of Community periodontal Index and Treatment Need(CPITN) partial and full version were validated using ROC Curves in Brazilian population [7].

 

Discussion

 

Reliability does not depend on validity while validity depends on reliability also. It should weigh accurately to be a valid instrument. Methodology of any research should be full proof to give accurate result. Any kind of bias in the study will distort the findings. If selection bias is not prevented by allocation concealment /randomization, while allocating patients to different study groups the result will be altered. Similarly, if there is no blinding and the investigator / analyser know about the groups he is measuring there may be bias affecting the final result. Thus method and methodology both should be valid to produce accurate result. This will reduce systemic error.

 

Accuracy is lack of errors. For any result to be accurate it should be free from any kind of bias and at the same time instrument / method should be a gold standard method or a valid method. Similarly random error should be minimised to increase the precision. Precision is opposite to random error. Thus both validity and reliability are important aspect of accuracy. Systemic errors are caused by different forms of bias affecting the mean value. Random errors do not affect the mean value but affect reliability.

 

Validity and reliability are threatened by different forms of bias [8]. For example – Selection bias, recall bias and non-response bias etc.

 

For equal distribution of confounding factors there should be perfect matching between experimental and control group. Proper selection method (Randomization) should be followed to allocate patients to each group. The method of allocation should not be known to the researcher (Allocation concealment). Blinding of subject, investigator and analyser helps in controlling different forms of bias. E.g., Behaviour bias. Appropriate steps should be taken to avoid recall bias, non response bias etc. All these precautions ensure sound methodology(Table6).

 

There is a proper technique to achieve validity and reliability [11].

 

To perform any research study often we have to create valid and reliable questionnaire / test. There are certain guidelines to carefully construct and develop its contents so that most dimensions of concept are included by seeking advice from experts[11-13]. It should be well constructed to reflect the concept. Its results should correlate with gold standard method. High Positive correlation shows good convergent validity. Similarly discriminate validity shows low correlation between 2 different groups with theoretically different concepts.

 

A question developed in a different country may not be a valid measure in our country. Just because questionnaire has been piloted or used in previous studies does not mean it is valid or reliable. Questionnaire should be cross culturally adopted and validated.

 

Conclusion

 

Qualities of any research depends on various factors like validity of instrument, intra and inter observer reliability, presence of known and unknown confounding factors, presence of bias etc.

 

To enhance quality of the study, researchers should take precautions in designing studies to reduce bias, which can be achieved through perfect matching, blinding and adapting appropriate selection method.

 

Questionnaire should have proper content validity and should adapt according to the need of study.Sound and concrete results canonly be produced with appropriate method and methodology.

References

 

  1. Jain S, Jain S, Punyani PR, JainD (2015) Basics of interpreting results. International Dental & Medical Journal of Advanced Research 1: 1-4.
  2. Saunders B, Trapp R(1998)Basics and Clinical Biostatistics. (2nd edition), United states of America: Prentice-Hall International Inc, USA. Pg No:58-59.
  3. Phansopkar S, Hegde-Shetiya S, Devadiga A, Agrawal D, MahuliA, et al. (2015) Face and Content Validation of Caries Assessment Spectrum and Treatment Index among Few Subject Matter Experts in India.International Journal of Dental Health Concerns 1: 13-18.
  4. Jain S, Sharma N, Jain D(2015) Basic fundaments of designing a quality research. Journal of Advanced Medical and Dental Sciences Research 3: 88-95.
  5. wikipedia.org (2017)Receiver operating characteristic.
  6. Bhad W, NayakS, Doshi U (2011) A new approach of assessing sagittal dysplasia: the W angle. The European Journal of Orthodontics35: 66-70.
  7. Bassani D, SilvaC, Oppermann R (2006) Validity of the Community Periodontal Index of Treatment Needs’ (CPITN) for population periodontitis screening. Cadernos de Saúde Pública22: 277-283.
  8. Jain S, DebbarmaS, JainD(2016) Bias in Dental Research / Dentistry. Annals of International Medical and Dental Research2: 2395-2814.
  9. FlemingPS, Johal A(2010) Self-ligating brackets in orthodontics. A systematic review. The Angle Orthodontics80: 575-584.
  10. Jain S, Sharma N (2016) Guideline for systematic reviews. International Dental and Medical Journal of Advance Research2: 1-10.
  11. Jain S, Dubey S, Jain S (2016) Designing and validation of a questionnaire. International dental & medical journal of advanced research 2:1-3.
  12. Edwards P(2010)Questionnaires in clinical trials: guidelines for optimal design and administration. Trials11.
  13. Kazi A, Khalid W (2012)Questionnaire designing and validation. Journal of Pakistan Medical Associatio 62: 514-516.
Tables

 

Instrument Readings(in Kg) Valid / reliable Inference
1st 2nd 3rd
Instrument A 50 50 50 Reliable and Valid Precise and Accurate
Instrument B 40 40 40 Reliable but Not Valid Imprecise & Inaccurate
Instrument C 40 45 50 Not reliable & Not Valid Imprecise and Inaccurate

 

Table 1: Validity and reliability of three instruments.

 

Scale DFS Avoidance Physiological arousal Fear of specific

Stimuli /situation

DFS total        
Avoidance 0.84*      
Physiological arousal 0.78* 0.63*    
Fear of specific Stimuli/ situation 0.95* 0.71* 0.62*

 

Table 2: Convergent validity was calculated as follows.

 

  No (n=961)

Mean (SD)

Yes(n=294)

Mean (SD)

p-value Difference Effect size
DFS total 32.62 (11.39) 42.04 (15.56) < 0.001 9.426 0.70
Avoidance 10.45 (3.72) 13.56 (5.96) < 0.001 3.104 0.64
Physiological arousal 7.58 (2.79) 9.36 (3.98) < 0.001 1.776 0.54
Fear of specific

Stimuli/situation

14.58 (6.33) 19.13 (7.50) < 0.001 4.545 0.66

 

 

Table 3:Negative dental experience at childhood.

 

Characteristic Code Description CVR Interpretation
Sound 0 No visible evidence of a distinct carious lesion is present. 0.6 Acceptable
Sealant 1 Pits and/or fissures are at least partially covered with a sealant material. 0.2 Not Acceptable
Restoration 2 A cavity is restored with an (in) direct restorative material. 0.6 Acceptable
Enamel 3 Distinct visual change in enamel only. A clear caries related discoloration is visible, with or without localized enamel breakdown. 0.6 Acceptable
Dentine 4 Internal caries-related discoloration in dentine. The discoloured dentine is visible through enamel which may or may not exhibit a visible localized breakdown of enamel. 0.4 Not Acceptable
5 Distinct cavitations into dentine. The pulp chamber is intact. 1 Acceptable
Pulp 6 Involvement of the pulp chamber. Distinct cavitations reaching the pulp chamber or only root fragments are present. 1 Acceptable
Abscess/

Fistula

7 A pus containing swelling or pus releasing sinus tract related to a tooth with pulpal involvement. 0.5 Acceptable
Lost 8 The tooth has been removed because of dental caries. 0.9 Acceptable
Other 9 Does not correspond to any of the other descriptions. -0.2 Not Acceptable

 

Table 4: CAST: Caries Assessment Spectrum and Treatment codes with content validity ratio[4].

 

W angle Malocclusion
Between 510-560 Class I
Less than 510 Class II
More than 560 Class III

 

Table 5: ROC Curve Showing W angle at Different Groups.

 

Method Methodology [9,10]
·                     Gold standard method/Valid method ·                     Sample size estimation

·                     Random sequence generation

·                     Allocation containment

·                     Blinding

·                     Intention to treat analysis

·                     Reporting of withdrawal

 

Table 6: Criteria for assessment of validity of any research methodology.

 

Suggested Citation

 

Citation: Jain S (2017) Accuracy in Dental Research. Dentistry Adv Res 2016: GDSC-118.

Leave a Reply