3.  Introduction

Since its inception in 1963, the ‘Red List’, which is published by the International Union for Conservation of Nature (IUCN), has been an invaluable tool for species conservation. The IUCN Red List has been used to project the probability and time at which the Earth will reach the sixth mass extinction [1], to quantify the most pervasive threats to imperiled species across the globe [2], to forecast the impacts of climate change on future extinction rates [3,4] and to estimate extinction in biodiversity hotspots [5]. The vast impact of its use is apparent from the 4.2 million annual visits to the Red List website [6] and the over 2,000 journal articles that now reference the Red List as a source of information as of October 2016 [7]. In addition to its impact on the scientific literature, the IUCN Red List has played an increasingly prominent role in conservation planning by governmental agencies, NGOs, and scientific institutions [6]. For example, it has been used as a model for regional and national Red Lists [8] and as a way to inform bio-assessment inventories and monitoring programs [9]. Use of the Red List has also been encouraged in a wide variety of conservation activities, including guidance for captive breeding programs in zoos [10,11] setting specific species targets in conservation plans [12] and using recovering species on the Red List as models for effective conservation strategies [13].

For these reasons and more, the Red List has been called the “most comprehensive resource detailing the global conservation status of plants and animals” [14] and one of “the most widely used tools available to conservationists worldwide” [15]. As the impact and use of the Red List has grown, several evaluations have asked how reliable the scientific information is from which we make management decisions and policy recommendations. Originally, the Red List compiled information based on the opinions of experts who were working directly with at-risk species in the field [14]. In the early 1990s, the Red List underwent a significant revision to transition from the use of expert opinion to more quantitative listing criteria that required data about species population sizes, geographic ranges, and primary threats. Despite these efforts to move towards more quantitative information, criticism remained about the lack of transparency and data being used to justify species listings [6,16]. This continued criticism prompted further revisions in the early 2000s to hone the justification criteria and assessment process further by redefining key terms used in species listings, providing guidance to listing authors on how to handle uncertainty, and improving supporting document requirements [6]. Proponents of the Red List claim that these improvements have led to a transparent process in which species listings are now based on actual data that is reliable and verifiable [14,17]. However, it is worth noting that during the listing process, authors are still given substantial flexibility in what information they use to justify listing a species as threatened [18]. Qualitative information and expert opinions are not only allowed, but encouraged when other information is not available. Given this, we wanted to know the present extent to which authors are using quantitative data, qualitative information or expert opinion to justify the listing of threatened species. Here, we provide an assessment of the type of evidence that is currently being used by assessors to justify species listings in the Class Mammalia on the IUCN Red List. We focused our review on mammals because this class is one of the best studied groups of organisms, with nearly all known species assessed and mammals being disproportionately represented in the conservation literature [19]. Therefore, we assumed that mammals rank among the best documented, and most evidence-based groups of organisms tracked by the IUCN Red List. Using guidelines established by the IUCN for listing species as vulnerable, endangered, or critically endangered, we catalogued the type of literature used by assessors to calculate or measure the quantitative criteria defined for each justification criteria for a randomly selected 20 percent of all mammalian species listings (241 listings total). Our analyses show that it is likely that most assessors continue to rely on qualitative and expert opinion because most provide no references to support the quantitative thresholds defined for each justification criteria. Of those listings that are provide references, the use of inaccessible references and expert opinion is still common with only half of the cited assessments using scientific literature to calculate justification criteria. We conclude by suggesting (1) the Red List still has a way to go before listings are justified with data-driven evidence, and in the meantime (2) the Red List should work to make the evidence behind listings more transparent to end-users so that researchers in particular can discern what is not scientifically defensible information.

4.   Materials and Methods

On 1-June 2016, we randomly selected 241 (20%) of the 1,208 threatened Mammalian species (classified as either critically endangered, endangered, or vulnerable) that are listed on the IUCN Red List. Our final subsample included 45 critically endangered, 93 endangered, and 103 vulnerable species (Table 1). This subsample was representative of the list of threatened mammals evidenced by the fact that our selected subset was not statistically different from the relative distribution of critically endangered, endangered, and vulnerable species across all threatened mammal species listed on the IUCN Red List (chi-square contingency test, χ² = 6, df = 4, P = 0.20). Each IUCN Red List assessment for a given species contains seven sections that provide supporting information for the assessment. These include: (i) Taxonomy, (ii) Assessment Information, (iii) Geographic Range, (iv) Population, (v) Habitat and Ecology, (vi) Threats, and (vii) Conservation Actions [6]. For purposes of our analyses, we focused on (ii) Assessment Information, which is the primary location where listing authors are supposed to provide information describing why a species qualifies for threatened status. Within section (ii) Assessment Information, authors are required to use at least one of five criteria to justify a species threatened status: (A) evidence of a large proportional reduction in the population size of the species, (B) evidence of a small geographic range-either in the extent of occurrence or area of occupancy, (C) evidence of a small and declining population size, (D) evidence of an extremely small population size, (D2) evidence of an unusually restricted geographic range, or (E) a quantitative analysis, such as a population viability analysis, that indicates a high risk of extinction [18] (Table 1) summarizes the quantitative thresholds for each for the IUCN Red List justification criteria. In the few instances where a listing author provided no justification under criteria A-E (5 of 241 listings), or the instances where there were no citations provided in the (ii) Assessment Information section, we read through all remaining 7 sections of the assessment to locate any justification or citations provided by the author in the other sections that could be used to justify species listings as threatened. Thus, our approach was conservative in that it maximized the chance of finding any information used to justify listing of the species. For our analyses, we first identified the criteria and the species-specific value calculated by the author to justify listing the species on the IUCN Red List. For example, Aegialomys galapagoensis (Rodentia, Cricetidae) was listed based on criteria D (geographic range restricted to one location) because it only exists on Santa Fe island within the Galápagos archipelago and Cavia intermedia (Rodentia, Caviidae) was listed based on criteria D ((less than 50 mature individuals) because there are only 42 known individuals in their native range in Brazil. In these examples, Santa Fe island and 42 individuals are the species- specific value. We then compiled a list of any references that were used by the author for the species-specific value, either in section (ii) or in another section, and searched for each reference using Google Scholar, the ISI Web of Science, the JSTOR database, Google, or the journal/organization’s website. In those instances where a reference was not available on the searched sites, or could not be found due to an improper citation, it was classified as RI for ‘reference inaccessible’. All remaining references were collated, reviewed in detail, and then grouped into one of four categories:

·  Unpublished Data (UP). If the listing author cited ‘personal communication’ or ‘unpublished data’, the reference was designate as UP.

·   General information (GE). If the listing author cited a book or online web-based source giving basic information on a species taxonomy, ecology, and natural history without reference to how that information was attained, and without reference to any scientific study that could justify the information, it was listed as GE. An example would be a Peterson’s field guide showing a species range map.

· IUCN Red List (IUCN). In one listing, the author simply referenced an earlier published IUCN species assessment as justification.

· Scientific Literature (SL). If the reference(s) were to peer-reviewed journal articles or grey literature papers that had published results from a scientific study it was classified as SL.  We read through all references categorized as SL to access whether they did, in fact, contain the data purported to support the justification criteria.

Multiple Sources (MS). In a select few instances (5 of 241), listing authors cited multiple sources of information. In these instances, we read through all sources of information that were cited to access whether they did, in fact, contain the data purported to support the justification criteria.

5. Results

Of the 241 assessments of mammalian species that we reviewed from the IUCN Red List, 78% (188 of 241) provided no references or data to justify the listing criteria (Figure 1). For the remaining 22% of assessments (53 of 241), 2.5% (6) cited references that were inaccessible, 2.9% (7) cited unpublished data, 4.6% (11) cited a general reference that had no scientific data, and 0.4% (1) cited a former IUCN listing. These all represent cases where we could not identify any credible scientific information or data to justify the listing of the mammal species. The number of assessments in which the listing authors cited scientific studies to justify the listing – either using sources from peer-reviewed journal articles or studies published in the gray literature – was just 7.5% (18). Of these, all 18 references contained the appropriate data to justify the listing criteria. There were an additional 4.6% (10) of assessments in which authors cited multiple sources in the listing. Of these, five references contained the appropriate data to justify the listing criteria, two did not contain the data for which they were cited, and three references were not accessible to us (non-English languages, articles that could be located, but which we did not have a subscription, etc…). So, in sum, we found that 23 of 241 (9.5%) assessments referenced scientific studies that contained data to support the stated listing criteria.

6. Discussion

Despite the IUCN’s intention to generate a list of threatened species that is founded on scientifically rigorous information, we found that less than 10% (23) of the randomly selected species assessments we reviewed for threatened mammal species had verifiable sources of quantitative scientific data to justify their listing on the IUCN Red List. While our analyses focused solely on mammals, this group is one of the most well-studied groups of threatened and endangered species [19]. Thus, we suspect are findings probably apply to other groups of organisms as well. This appears to contradict how the IUCN publicizes itself because while the IUCN states that the list is both objective and scientifically-based [6] we found that most assessments did not provide a transparent explanation of why the species should be listed as threatened. Consequently, the findings of our study contrast with how the IUCN publicizes the scientific rigor of the Red List and with how many researchers use and cite the list. It is important to understand how authors are justifying the threatened status of species because researchers routinely use the Red List as scientific data. Researchers have used the IUCN Red List to estimate modern rates of extinction [1], identify the most pervasive threats to biodiversity [2], predict how environmental change will impact future extinction rates [3,4] and prioritize areas for conservation [5] However, there appears to be a gap between how people perceive the scientific rigor of the Red List and the actuality of what information is being used to justify species listings. This, in part, is caused by a lack of transparency regarding what information is being used to justify species listings. Although the IUCN encourages use of scientific data in the listing process, assessors are still allowed to support the listing of a species by “observing, estimating, inferring, or suspecting” [18] the population or geographic range reduction. This has been built into the listing guidelines to provide authors a way to list species which are truly in danger and need active conservation and protection, but which have not benefited from scientific study.

We are not suggesting that this type of information shouldn’t be used, rather that this practice has created a problem. Regardless of whether the listing is backed by actual data or only expert opinion, the entry on the IUCN Red List site only displays the quantitative listing criteria. For example, if a listing author uses expert opinion to decide a species is ‘Critically Endangered’ based on criteria A1, they are stating that they believe the population has declined by more than 90% over the last 10 years but do not have data to support that claim. However, the only information provided on the IUCN Red List, is that the species has suffered a 90% or greater reduction in population size with no mention that expert opinion was used due to lack of verifiable data. This example highlights how this practice results in a less then transparent listing process because for most end-users of the Red List it would appear as if all species on the IUCN Red List are supported by data. However, we only found 9.5% of entries citing scientific literature to justify listing criteria.

7.   Conclusion

To improve transparency, while still allowing for the greatest number of species to be listed, we suggest that the IUCN Red List should require authors to clearly state whether the justification criteria are based on actual data or some other form of inference. One way to easily separate the listings based on expert opinion versus scientific data would be to implement a quantitative ranking system that is decided by reviewers to represent the relative rigor of the data used to support listing a species as threatened. This ranking would allow for end-users to sort and assess species listings by the quality of data support, and could be used to prioritize species for which there is a lack of data. We further suggest that the IUCN consider adding more information about the listing authors themselves. We often found it difficult, if not impossible, to determine who the listing assessor(s) and reviewer(s) were since the only identifying information provided on assessments was the last name and first initial. Frequently, this was not enough information to identify the assessor through publicly available sources. Even when we could locate the author, often there was no information regarding the author’s background and expertise. This is particularly important if the IUCN Red List is going to continue to use expert opinions to justify species listings, because it gives end-users the information needed to judge the credibility of the assessors and the reviewers. To address this concern, the contact information for the main assessors and expertise biographies for all assessors and reviewers could be made publicly available on the IUCN webpage for each species listing. Our paper is just the most recent critique to point out the lack of data used to support species listings on the IUCN Red List [20]. The pervasive, and continued lack of quantitative data being used to justify species listings brings up a key question we must ask ourselves as a community of conservation biologists. Do we want to have an evidence-based tool to list species that are known to be under a certain level of threat and extinction risk, or do we want a species list that assigns risk more broadly based primarily on subjective criteria and expert opinions? If having quantitative listing criteria is still the goal, then the IUCN Red List needs to be more transparent about what information is being used to justify listings and/or adopt practices that help ensure listings are justified by scientifically credible information.

8.       Acknowledgments

We want to thank C. Rakowski and others for helpful feedback on earlier versions of this manuscript, and the IUCN for their helpful information

9.       Conflict of Interest

No authors have any conflicts of interest.

Figure 1: Classification of the references (Reference Inaccessible (RI), Unpublished Data (UP), IUCN Red List Assessment (IUCN), General Reference (GE), Scientific Literature (SL), and Multiple Sources (MS)) cited by assessors to justify listing criteria for a random sample of 241 (20%) of mammal species assessments on the IUCN Red List. When we were not able to access a cited reference but it was clear it was scientific literature we classified it as NA (Not Accessible) when determining whether the reference contained appropriate data.

Table 1: A summary of the quantitative thresholds for each of the IUCN Red List justification criteria for listing a species as either Critically Endangered, Endangered, or Vulnerable.

1. Barnosky AD, Matzke N, Tomiya S, Wogan GOU, Swartz B, et al. (2011) Has the Earth's sixth mass extinction already arrived? Nature 471: 51-57.
2. Wilcove DS, Rothstein D, Dubow J, Phillips A, Losos E (1998) Quantifying threats to imperiled species in the United States. Bioscience 48: 607-615.
3. Thomas CD, Cameron A, Green RE, Bakkenes M, Beaumont LJ, et al. (2004) Extinction risk from climate change. Nature 427: 145-148.
4. Thuiller W, Lavorel S, Araujo MB, Sykes MT, Prentice IC (2005) Climate change threats to plant diversity in Europe. Proceedings of the National Academy of Sciences of the United States of America 102: 8245-8250.
5. Brooks TM, Mittermeier RA, Mittermeier CG, da Fonseca GAB, Rylands AB, et al. (2002) Habitat loss and extinction in the hotspots of biodiversity. Conservation Biology 16: 909-923.
6. IUCN (2013b) The IUCN Red List of Threatened Species.
7. Web of Science (2014) ISI Web of Science. Thomas Reuters.
8. Brito D, Ambal RG, Brooks T, Silva ND, Foster M, et al. (2010) How similar are national red lists and the IUCN Red List? Biological Conservation 143: 1154-1158.
9. OSEWIC (2008) COSEWIC'S assessment process and criteria. COSEWIC, Gatineau.
10. Flynn KJ (2010) Do external resource ratios matter? Journal of Marine Systems 83: 170-180.
11. Hayward M, Conde DA, Colchero F, Gusset M, Pearce-Kelly P, et al. (2013) Zoos through the Lens of the IUCN Red List: A Global Metapopulation Approach to Support Conservation Breeding Programs. PLoS ONE 8: e80311.
12. Pfab MF, Victor JE, Armstrong AJ (2011) Application of the IUCN Red Listing system to setting species targets for conservation planning purposes. Biodiversity and Conservation 20: 1001-1012.
13. Hayward MW (2011) Using the IUCN Red List to determine effective conservation strategies. Biodiversity and Conservation 20: 2563-2573.
14. Rodrigues A, Pilgrim J, Lamoreux J, Hoffmann M, Brooks T (2006) The value of the IUCN Red List for conservation. Trends in Ecology & Evolution 21: 71-76.
15. Gardenfors U, Hilton-Taylor C, Mace GM, Rodriguez JP (2001) The application of IUCN Red List criteria at regional levels. Conservation Biology 15: 1206-1212.
16. Mrosovsky N (1997) IUCN's credibility critically endangered - Commentary. Nature 389: 436-436.
17. Butchart SHM, Bird JP (2010) Data Deficient birds on the IUCN Red List: What don't we know and why does it matter? Biological Conservation 143: 239-247.
18. IUCN (2013a) Documentation standards and consistency checks for IUCN Red List assessments and species accounts. Version 2. Adopted by the IUCN Red List Committee and IUCN SSC Steering Committee.
19. Clark JA, May RM (2002) Taxonomic bias in conservation research. Science 297: 191-192.
20. Butchart SHM (2008) Red List Indices to measure the sustainability of species use and impacts of invasive alien species. Bird Conservation International 18: S245-S262.