NIJ Study of Near Misses and Wrongful Convictions

The National Institute of Justice recently released their study Predicting Erroneous Convictions which examines why some innocent people are wrongfully convicted while others are acquitted, using a case comparison method rather than a traditional “case study” method. Researchers at American University who compared a group of 260 cases that occurred between 1980 and 2012 where an innocent defendant was exonerated only after conviction with a control group of 200 cases where an innocent defendant was acquitted or had charges dismissed before trial.

Unlike prior studies which have focused on what causes an innocent defendant to be charged, this study’s results can be used to determine what is preventing the innocent defendant from being acquitted or having charges dismissed once in the system. This is a helpful distinction, as it allows a greater focus on criminal defense, forensic practices, and prosecutorial discretion.

A Mixed Method Approach

The study’s quantitative analysis yielded 10 primary factors that led to a wrongful conviction rather than the “near miss” dismissals and acquittals:

• A younger defendant
• A criminal history
• A weak prosecution case
• Prosecution withheld evidence
• Lying by a non-eyewitness
• Unintentional witness misidentification
• Misinterpreting forensic evidence at trial
• A weak defense
• Defendant offered a family witness
• A “punitive” state culture

A qualitative analysis of the data followed in which a panel of criminal justice experts analyzed 39 cases to determine how the ten factors were connected and whether “tunnel vision” (when law enforcement professionals focus more on building a case against one suspect at the expense of ignoring contradictory evidence) played a role.

The final report offers recommendations to prevent future wrongful convictions, including recommendations on defense practice, exculpatory evidence, eyewitness identification, false confessions, forensic error, police misconduct, weak prosecution evidence, systemic failures and tunnel vision. The report emphasizes that the interactions of these factors as much as the individual factors themselves are to blame for systemic breakdowns leading to erroneous convictions. This type of interaction requires a comprehensive approach to reform in order to prevent future errors.

Traditional Wrongful Conviction Factors

Factors that have traditionally been suggested as sources of erroneous convictions, such as false confessions,  witness misidentification, and racial bias, appeared at statistically similar rates in both sets of cases. Thus, these factors likely only increase the chance that an innocent suspect will be charged but not the likelihood that the charge will result in conviction rather than dismissal or acquittal.

Forensic Error as a Primary Factor in Wrongful Convictions

As one would expect, this study revealed that errors in forensic evidence were correlated with an increased likelihood of erroneous conviction. However, this study’s results suggest a new perspective on how forensic error impacts case outcomes. Errors in forensic testimony, rather than errors in the actual testing were the most common source of forensic error.  Forensic testimony errors include: not providing the jury with key forensic information, overstating the inculpatory nature of evidence, use of inaccurate statistics, and misstating the certainty of results and forensic techniques. The implication is that an emphasis on quality control at the interpretation and testimony stages should be added to previous policy recommendations that have focused on improving quality of forensic lab procedures.

While DNA is a widely respected as a forensic tool, this study found that the prohibitive cost of DNA testing often renders it a tool of last resort, with other less accurate forensic tools like fingerprinting, hair comparison analysis and serology testing being used first.

Recommendations for Improving Forensic Error

• Timeliness: Forensic investigation, especially DNA testing, should be conducted early in a case to rule suspects in or out, rather than later once an investigation gains steam and there is a danger of it being used more to confirm or check what police/prosecutors already believe.
• Training/Education: Prosecutors and defense attorneys need more education about forensic testing techniques and should get clarifications about results if they do not fully understand a report. Police officers in more jurisdictions need to be trained in crime scene investigation and should then be present when technicians are collecting evidence to help direct and expand the search.
• Supervision: Forensic labs should be more closely supervised and have procedures that are peer reviewed. There should be discussions of “near miss” cases, which could follow, for example, in the vein of the morbidity and mortality conferences that the medical community holds to discuss mistakes in patient care.

For more information, watch the forthcoming recording of the Wrongful Convictions seminar given by the report’s authors, found here.

Forensics@NIST 2012 videos available online

The National Institute of Standards and Technology (NIST) is a non-regulatory federal agency dedicated to advancing measurement science, standards, and technology. NIST manages grants for technological research, works with employers and private contractors, and maintains a laboratory for in-house research in a wide variety of fields, including forensic science.

Each year, NIST holds a conference to showcase its ongoing research in forensic science. The most recent conference, Forensics@NIST 2012, featured three days of presentations on developments in DNA analysis, firearms and tool mark analysis, fire research, fingerprints, biometrics, multimedia forensics, and other techniques. The agenda, which lists all of the presentations, can be found here. The event was broadcast live on the web, and NIST recently posted videos of all the presentations on its website. The videos are short, ranging from 10 to 20 minutes each, and are mostly accessible to a lay audience.

These videos are a good resource for attorneys interested in learning about the development of new forensic techniques. NIST researchers presented research and recommendations on a variety of new techniques, including rapid DNA typing, 3D topography for matching bullets, facial recognition software, and enhancing latent fingerprints. Some of these techniques may be used by forensic practitioners in the near future.

The videos may also be useful for attorneys seeking to understand the limitations of current forensic methods. NIST’s research is geared toward improving current forensic techniques, and as such serves as commentary on the limitations of those techniques. For example, NIST researchers presented on the development of a standard for bullet and casing comparisons in firearm analysis, in part out of concern that the lack of a standardized control prevented reliable comparisons across laboratories.

Finally, some of the videos provide useful background information for attorneys looking for an overview of the forensic techniques currently in use. For example, a presentation from the first day titled “Overview of Firearms Projects” includes a short primer on the methodologies used in bullet and firearm comparison.

ABA Resolution concerning forensic evidence

The ABA House of Delegates approved a Resolution in 2012 urging judges to consider several factors when determining the manner in which expert testimony is presented in criminal trials. The Resolution and its accompanying report urge attorneys and judges to seek “innovative solutions” to help jurors understand the significance and limitations of scientific evidence, such as altering trial structure to allow expert witnesses for both parties to testify consecutively and  avoiding declaring a witness to be an expert in front of the jury.

The ABA Resolution and report draw heavily from other ABA standards and from the 2009 NAS Report on the state of forensic science in the United States. More information on the landmark NAS Report can be found here.

The ABA report also critiques trial attorneys’ lack of substantive knowledge regarding scientific evidence and their ability to effectively challenge misleading forensic testimony. “Until an elevation in the knowledge base of trial attorneys is achieved,” the ABA report warns, “the adversarial system will continue to falter with respect to the proper presentation of forensic scientific evidence.”

The ABA Resolution lists several areas of concern for testimony by forensic experts. Highlights include:

Use of Clear and Consistent Terminology

The Resolution urges judges to consider “whether expert witnesses use clear and consistent terminology in presenting their opinions.” The report warns that terms such as “match,” “consistent with,” “similar in all respects tested,” and “cannot be excluded as the source of” have no accepted definition or standardized meaning in the scientific community.

Limitations of Forensic Techniques

The Resolution urges judges to consider whether experts present testimony in a way that accurately conveys any limitations in the forensic techniques they employ. The report points out that experts in disciplines such as microscopic hair analysis sometimes exaggerate the reliability of subjective techniques with misleading phrases like “zero error rate,” claiming that these methods are error-free when performed “correctly.” The report also criticizes the use of phrases with no accepted scientific meaning, such as “reasonable scientific certainty.”

Avoiding Claims of Uniqueness

The Resolution also advocates precluding experts from offering explicit or implied claims of uniqueness unless their findings are supported by empirical research. The report notes that fields such as firearms comparison and handwriting analysis often rely on subjective comparison by analysts with no empirical research to validate their techniques. Testimony by such experts gives jurors an impression that such “matches” represent absolute identification. In particular, the report recommends prohibiting experts from testifying that a match has been made “to the exclusion of all others” unless the experts’ methodology has been validated by empirical statistical research.

Although most judges are unlikely to exclude evidence solely on the basis of the ABA Resolution, attorneys may attempt to use the Resolution to limit the scope and impact of expert testimony in their cases. It isn’t clear how much weight individual judges will give the ABA Resolution, or whether they will interpret the Resolution as placing a higher burden on parties seeking to use expert testimony than already required under North Carolina law. Nevertheless, the Resolution provides strong support for attorneys trying to preclude experts from offering misleading testimony about the significance of their findings, and it calls on judges to monitor the presentation of forensic evidence more closely.

Stateline and NPR report on the state of forensic science labs

In a two-part report, Stateline examines the problems facing crime labs across the country.  Part one of the series addresses recent problems plaguing crime labs and the need for judicial education regarding the reliability of forensic evidence.  Part two introduces a new forensic lab in Washington, D.C. which operates independently of state law enforcement agencies and has incorporated many of the recommendations of the 2009 National Academy of Sciences report.

You can read the part one of the Stateline report here  and part two here.

The Greg Taylor case and lab accreditation were the focus of a Nov. 20, 2012, All Things Considered report on NPR. ASCLD/LAB accredited the NC State Crime Laboratory during the time when Greg Taylor’s case was worked and still accredits the lab. In the report, defense attorney Marvin Schechter questions ASCLD/LAB’s accreditation process and calls for forensic science labs to be made independent of law enforcement.  You can listen to the story here.

Fire Investigation Publications Available

The National Institute of Justice has made available the following reports that address various techniques used in fire investigations. These reports may be useful to attorneys handling cases were arson is alleged. Each report attempts to document best practices for investigating specific aspects of fires.

• Forensic Investigation Techniques for Inspecting Electrical Conductors Involved in Fire – 2012 DOJ publication by Richard J. Roby, Ph.D. and Jamie McAllister, Ph.D. that looks at the physical characteristics of energized and non-energized wires subjected to various types of fire exposures.
• Reducing Uncertainty of Quantifying the Burning Rate of Upholstered Furniture – 2012 DOJ publication by Marc L. Janssens that investigates how to estimate the burning rate of upholstered furniture and how to express the uncertainty of this prediction.
• Spontaneous Ignition in Fire Investigation – 2012 DOJ publication by James G. Quintiere, Justin T. Warden, Stephen M. Tamburello, and Thomas E. Minnich that addresses the principles of spontaneous ignition and its potential role as the cause and origin of a fire.
• Thermal Properties Database – 2012 DOJ publication by Arnaud Trouve’ and Thomas Minnich. This publication explains the Burning Item Database which describes the burning characteristics of common household and office items.

Impression & Pattern Evidence Symposium – Online today!

The Impression & Pattern Evidence Symposium is taking place today through Thursday, August 9, 2012. The live program, sponsored by the National Institute of Justice (NIJ) Office of Justice Programs and the Federal Bureau of Investigation (FBI) Laboratory Division, is being streamed online here.

The agenda is available here. Topics include the latest developments and challenges to fingerprint, firearm, document, tiremark, bloodstain pattern and other types of pattern and impression evidence as well as context bias, calculation of error rate, documenting through bench notes, outsourcing lab analysis, and the way to present degree of certainty.

The entire program is available for free online streaming, so sign up and watch the sessions that are of interest to you!

Notice and demand procedures

Jessica Smith’s recent post, Don’t Look a Gift Horse in the Mouth, on the School of Government’s North Carolina Criminal Law blog reminded readers about the use notice and demand statutes in cases involving lab reports or chain of custody statements. Notice and demand statutes, in a nutshell, allow prosecutors to obtain a waiver of a defendant’s confrontation rights by providing to the defendant notice of the state’s intention to introduce lab reports or chain of custody statements without live testimony. If the defendant does not object, such evidence may be introduced at trial without the testimony of the analyst or evidence custodian.

Prof. Smith’s post provides a useful table which summarizes the requirements of the notice and demand statutes. The table and additional information is also available in this School of Government Bulletin. Attorneys who wish to object should refer to the table and the corresponding statutes to determine the timing  of their objection/demand for live testimony. A sample Notice of Objection is available on the IDS forensics website (scroll down to “Other Motions and Orders”).

Update: See this 7/25 post by Jessica Smith on proper execution of notice and demand procedures and recent case law addressing notice and demand.

DNA for the Defense Bar

A fantastic new publication is available for criminal defense attorneys working on cases with DNA evidence. DNA for the Defense Bar was published by the National Institute of Justice in June 2012 and is available for free download here. Normally I don’t recommend printing out publications because I love trees, but this is one resource that you’ll want to print out and keep on hand when you’re working on cases with DNA evidence.

This manual was written by a group of experienced defense attorneys and DNA experts including Jack Ballantyne, Catherine Cothran, Jules Epstein, Christine Funk, Chris Plourd, Vanessa Potkin, Ron Reinstein, and Edward Ungvarsky.  Its approach to basic and advanced topics is easy to understand. The manual contains excellent examples and figures so that attorneys can visualize the concepts discussed and compare the figures in the manual to lab reports in their own cases.There are clear explanations of topics such as artifacts, interpretation of results, cold case hits and CODIS, DNA collection issues, laboratory issues, newer techniques such as mtDNA and Y-STR, and statistics. There is a glossary that attorneys can use to understand the concepts in this publication and the language used in lab reports.

In addition to explaining the science and techniques of DNA analysis, the manual offers advice on topics such as opening and closing statements, jury selection, and cross examination. This is a great reference for any attorney working on cases with DNA evidence.

Digital Forensics for Attorneys

The UNC School of Government has posted a new on-demand virtual CLE entitled, Digital Forensics for Attorneys. Digital forensics expert Larry Daniel teaches this one-hour course which attorneys can view for free or purchase for $50 if CLE credit is needed. Daniel’s program provides an overview of digital forensic concepts, case examples, and relevant terminology. Attorneys will learn the basic information needed to understand the process of computer and cell phone forensics; the primary areas of focus in digital forensics; and the proper methods for search and seizure of electronic evidence.

Daniel discusses document metadata and the capabilities of computer forensic recovery of email, internet history, documents, and pictures. He also provides answers to common questions, such as: What is a forensic copy of a hard drive? What kind of information can be recovered?  How do I know if the evidence was properly obtained and preserved? What is a computer forensics expert and what should an attorney expect from such an expert?

This program is a great way for attorneys to learn more about the type of analysis performed on digital devices and is an excellent refresher for anyone confronting digital forensic evidence in a case.

The (Sorry) State of Forensics in the US [and perhaps the world]

Reposted from The Wrongful Convictions Blog

by Phil Locke, Science and Technology Advisor, Ohio Innocence Project

In 2009, The National Academies of Science of the United States published its Congressionally commissioned report:  “Strengthening Forensic Science in the United States – A Path Forward.”  Chapter 5 of the report presents a review of a number of forensic disciplines and their shortcomings.  A qualitative summary (by this author) of Chapter 5 findings is presented in the following chart:

I have “graded” each of the forensic disciplines on the following attributes:

1)  Statistical reliability for “class inclusion” of a suspect

2)  Statistical reliability for “individual identification” of a suspect.

3)  Statistical reliability for “class and individual exclusion” of a suspect.

4)  Verified scientific validity with documented statistics.

5)  Clear non-ambiguous terminology related to statistical validity of results.

6)  Does not rely on competence, training, experience, or judgement of individual examiners.

Of great concern is all the red, pink, and yellow on the chart.  Here is a link to a downloadable (and legible) copy of the chart:

NAS Ch5 Summary_Rev_4 copy

The National Academies were given the Congressional “charge” in the Fall of 2005 to investigate and report on the state of forensics in the US.  By the Fall of 2006, a panel of 52 scientists, academics, and experts had been assembled, and started work.  Two and a half years later, and after exhaustive review of all findings, the report was published.  What it had to say about forensics in the US (and by extension, the world) was not very good.  In summary, what they found was that forensics (with the exception of DNA) lacks scientific rigor and statistical validation.

It can be said that every forensic discipline (with the exception of DNA) fails the test of “show me the data from which I can compute a probability of occurrence.” This is, of course, echoed in the Daubert doctrine’s “no known error rate”.

Has this lack of scientific rigor and statistical validity led to wrongful convictions?  ABSOLUTELY.

But …. more about the validity of forensics in future posts.