The (Sorry) State of Forensics in the US [and perhaps the world]

Reposted from The Wrongful Convictions Blog

by Phil Locke, Science and Technology Advisor, Ohio Innocence Project

In 2009, The National Academies of Science of the United States published its Congressionally commissioned report:  “Strengthening Forensic Science in the United States – A Path Forward.”  Chapter 5 of the report presents a review of a number of forensic disciplines and their shortcomings.  A qualitative summary (by this author) of Chapter 5 findings is presented in the following chart:

I have “graded” each of the forensic disciplines on the following attributes:

1)  Statistical reliability for “class inclusion” of a suspect

2)  Statistical reliability for “individual identification” of a suspect.

3)  Statistical reliability for “class and individual exclusion” of a suspect.

4)  Verified scientific validity with documented statistics.

5)  Clear non-ambiguous terminology related to statistical validity of results.

6)  Does not rely on competence, training, experience, or judgement of individual examiners.

Of great concern is all the red, pink, and yellow on the chart.  Here is a link to a downloadable (and legible) copy of the chart:

NAS Ch5 Summary_Rev_4 copy

The National Academies were given the Congressional “charge” in the Fall of 2005 to investigate and report on the state of forensics in the US.  By the Fall of 2006, a panel of 52 scientists, academics, and experts had been assembled, and started work.  Two and a half years later, and after exhaustive review of all findings, the report was published.  What it had to say about forensics in the US (and by extension, the world) was not very good.  In summary, what they found was that forensics (with the exception of DNA) lacks scientific rigor and statistical validation.

It can be said that every forensic discipline (with the exception of DNA) fails the test of “show me the data from which I can compute a probability of occurrence.” This is, of course, echoed in the Daubert doctrine’s “no known error rate”.

Has this lack of scientific rigor and statistical validity led to wrongful convictions?  ABSOLUTELY.

But …. more about the validity of forensics in future posts.


Filed under NAS Report, Resources

4 responses to “The (Sorry) State of Forensics in the US [and perhaps the world]

  1. Frederic Whitehurst

    Phil, but what has caused this failure of fact finding in our justice system? We can blame everyone from the Supreme Court to the individual attorneys but that does nothing more than label and move on. Why are we unable to fix this broken wing? The NAS report simply says at the federal level what so many of us have been saying for decades. HOW ARE WE GOING TO FIX THIS???
    Fred Whitehurst

  2. Phil Locke

    Fred – your question is, of course, the one that really matters. I’m a raging fan of “root cause analysis”, and I think this goes to the very core of our criminal justice system. I’m not sure there’s a clear answer, and any attempt to address it will be long and complicated. However, this will be good fodder for a future post. Thank you.
    Phil Locke

  3. koppl

    The NAS report suggests oversight. Some problems with oversight are discussed here:
    A somewhat different approach to improvement found here:

    Click to access d834fab5860d5cf4b3949fecf86d3328.pdf

  4. Phil Locke

    Prof. Koppl, thank you for bringing this material to my (our) attention. You clearly have insight into the issues, and I will study these.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s