Author Archives: Ryan Niland

Forensics@NIST 2012 videos available online

The National Institute of Standards and Technology (NIST) is a non-regulatory federal agency dedicated to advancing measurement science, standards, and technology. NIST manages grants for technological research, works with employers and private contractors, and maintains a laboratory for in-house research in a wide variety of fields, including forensic science.

Each year, NIST holds a conference to showcase its ongoing research in forensic science. The most recent conference, Forensics@NIST 2012, featured three days of presentations on developments in DNA analysis, firearms and tool mark analysis, fire research, fingerprints, biometrics, multimedia forensics, and other techniques. The agenda, which lists all of the presentations, can be found here. The event was broadcast live on the web, and NIST recently posted videos of all the presentations on its website. The videos are short, ranging from 10 to 20 minutes each, and are mostly accessible to a lay audience.

These videos are a good resource for attorneys interested in learning about the development of new forensic techniques. NIST researchers presented research and recommendations on a variety of new techniques, including rapid DNA typing, 3D topography for matching bullets, facial recognition software, and enhancing latent fingerprints. Some of these techniques may be used by forensic practitioners in the near future.

The videos may also be useful for attorneys seeking to understand the limitations of current forensic methods. NIST’s research is geared toward improving current forensic techniques, and as such serves as commentary on the limitations of those techniques. For example, NIST researchers presented on the development of a standard for bullet and casing comparisons in firearm analysis, in part out of concern that the lack of a standardized control prevented reliable comparisons across laboratories.

Finally, some of the videos provide useful background information for attorneys looking for an overview of the forensic techniques currently in use. For example, a presentation from the first day titled “Overview of Firearms Projects” includes a short primer on the methodologies used in bullet and firearm comparison.

Leave a comment

Filed under Resources

Dogs – 1, Defendants – 1


Today, the U.S. Supreme Court decided the second of two dog sniff cases before it this term. In Florida v. Jardines, the Court held that taking a drug-sniffing dog on the front porch of a house is a search for purposes of the Fourth Amendment. The Court held that the front porch is within the constitutionally-protected curtilage area of the house and that a general invitation to approach a house’s front door does not include an invitation to bring a dog onto the porch to conduct a search for drugs. The majority did not decide whether the dog sniff also violated the suspect’s reasonable expectation of privacy.

In February 2013, the Court decided Florida v. Harris, another dog sniff case which sheds new light on appropriate ways of challenging the adequacy of probable caused based a drug dog’s alert. Jeff Welty’s Criminal Law Blog provides a full summary of the case, its holding, and the issues it raises.

In Harris, a Florida police officer searched a motorist’s truck after the officer’s dog alerted to the presence of narcotics in the vehicle. The officer’s search revealed materials used in the production of methamphetamine, but no controlled substances. Although the dog had completed several training programs, the police kept no records of its performance in the field. The defendant argued that a drug dog cannot provide probable cause unless it has been shown to be reliable in the field, as opposed to controlled environments. The Florida Supreme Court agreed and suppressed the evidence from the search.

The U.S. Supreme Court reversed, unanimously rejecting an inflexible rule requiring records of a dog’s performance in the field to establish probable cause. The Court held that the adequacy of a dog’s alert for the purposes of probable cause must be evaluated under the totality of the circumstances. The Court considered the dog’s completion of a “bona fide” training program to be sufficient to create a presumption of reliability. Because the defendant had no evidence to rebut that presumption, the Court agreed with the trial court that the officer had probable cause to search the truck.

The Court’s opinion in Harris may provide new opportunities for attorneys seeking to challenge probable cause in drug dog cases. The Court’s totality-of-the-circumstances approach, combined with its insistence that defendants be able to contest the state’s evidence, should open the door to discovery on all aspects of a drug dog’s training and performance. In addition to evidence of a dog’s performance in the field, attorneys should request discovery on the dog’s performance in training, as well as the general effectiveness of the training program the dog completed. Use of independent experts may also be necessary to evaluate such evidence and effectively contest the state’s evidence.

Leave a comment

Filed under Drug Dogs

ABA Resolution concerning forensic evidence

The ABA House of Delegates approved a Resolution in 2012 urging judges to consider several factors when determining the manner in which expert testimony is presented in criminal trials. The Resolution and its accompanying report urge attorneys and judges to seek “innovative solutions” to help jurors understand the significance and limitations of scientific evidence, such as altering trial structure to allow expert witnesses for both parties to testify consecutively and  avoiding declaring a witness to be an expert in front of the jury.

The ABA Resolution and report draw heavily from other ABA standards and from the 2009 NAS Report on the state of forensic science in the United States. More information on the landmark NAS Report can be found here.

The ABA report also critiques trial attorneys’ lack of substantive knowledge regarding scientific evidence and their ability to effectively challenge misleading forensic testimony. “Until an elevation in the knowledge base of trial attorneys is achieved,” the ABA report warns, “the adversarial system will continue to falter with respect to the proper presentation of forensic scientific evidence.”

The ABA Resolution lists several areas of concern for testimony by forensic experts. Highlights include:

Use of Clear and Consistent Terminology

The Resolution urges judges to consider “whether expert witnesses use clear and consistent terminology in presenting their opinions.” The report warns that terms such as “match,” “consistent with,” “similar in all respects tested,” and “cannot be excluded as the source of” have no accepted definition or standardized meaning in the scientific community.

Limitations of Forensic Techniques

The Resolution urges judges to consider whether experts present testimony in a way that accurately conveys any limitations in the forensic techniques they employ. The report points out that experts in disciplines such as microscopic hair analysis sometimes exaggerate the reliability of subjective techniques with misleading phrases like “zero error rate,” claiming that these methods are error-free when performed “correctly.” The report also criticizes the use of phrases with no accepted scientific meaning, such as “reasonable scientific certainty.”

Avoiding Claims of Uniqueness

The Resolution also advocates precluding experts from offering explicit or implied claims of uniqueness unless their findings are supported by empirical research. The report notes that fields such as firearms comparison and handwriting analysis often rely on subjective comparison by analysts with no empirical research to validate their techniques. Testimony by such experts gives jurors an impression that such “matches” represent absolute identification. In particular, the report recommends prohibiting experts from testifying that a match has been made “to the exclusion of all others” unless the experts’ methodology has been validated by empirical statistical research.

Although most judges are unlikely to exclude evidence solely on the basis of the ABA Resolution, attorneys may attempt to use the Resolution to limit the scope and impact of expert testimony in their cases. It isn’t clear how much weight individual judges will give the ABA Resolution, or whether they will interpret the Resolution as placing a higher burden on parties seeking to use expert testimony than already required under North Carolina law. Nevertheless, the Resolution provides strong support for attorneys trying to preclude experts from offering misleading testimony about the significance of their findings, and it calls on judges to monitor the presentation of forensic evidence more closely.

Leave a comment

Filed under Experts, NAS Report, Resources

Part 3: ISO and the NC Forensic Sciences Act of 2011

This is the third in a three-part series on lab accreditation, analyst certification, and ISO-compliant lab procedures by Ryan Niland.

In response to concerns about the failures of the North Carolina State Crime Laboratory (formerly known as the SBI lab), North Carolina passed the Forensic Sciences Act (FSA) in 2011. Among other things, the FSA requires that for a report to be admissible in court without the testimony of the analyst, it must be produced by a lab accredited by an accrediting body which is a signatory to the International Laboratory Accreditation Cooperation (ILAC) Mutual Recognition Arrangement for Testing.

ASCLD/LAB’s traditional accreditation program, sometimes referred to as its “Legacy Program,” does not use ILAC standards. Consequently, in September 2012, the North Carolina State Crime Laboratory adopted and posted new lab procedures on its website in order to seek additional lab accreditation according to ILAC standards. ILAC requires forensic laboratories to follow procedures set by the International Organization for Standardization (ISO).

ISO is an organization that develops international standards in many areas, including management, food safety, and nearly all mass-produced products. Forensic labs seeking ISO accreditation must follow the procedures in ISO 17025, which contains general requirements for all testing and calibration laboratories. ILAC has produced guidelines (ILAC G19) for laboratories applying ISO 17025 procedures in a forensic context. ISO 17025 and ILAC G19 set out general principles that labs must adhere to, but each lab that seeks ISO accreditation will have its own procedures interpreting these standards. The UN has put together an excellent guidebook on complying with ISO 17025 available here.

All state lab tests performed after September 17, 2012 should follow the new lab procedures. Analysts at the lab report that the greatest procedural changes occurred in the Forensic Biology/DNA and Drug Analysis/Toxicology sections, as well as in how documentation is kept. The lab procedures for tests administered prior to September 17, 2012 have been removed from the lab’s website. Attorneys should contact the lab to obtain copies of the older procedures, or they can contact the Forensic Resource Counsel for assistance.

The North Carolina State Crime Lab accepted public comment on the new lab procedures, and independent experts provided a number of critiques regarding the new procedures. The critiques included questions about the lab’s forensic biology testing procedures, DNA analysis interpretation guidelines, drug sampling protocols, and results statements used in fingerprint comparisons. The Forensic Resource Counsel has copies of many of these comments and can provide them to you and discuss with you how they may relate to the analysis performed in your case.

Part 1 of this series is available here. Part 2 is available here.

1 Comment

Filed under Crime Labs

Part 2: The Lab Accreditation Process

This is the second in a three-part series on lab accreditation, analyst certification, and ISO-compliant lab procedures by Ryan Niland.

Several major organizations offer accreditation services for forensic labs in the United States. The American Society of Crime Laboratory Directors/Laboratory Accreditation Board (ASCLD/LAB) is by far the largest accrediting body for forensic labs, followed by Forensic Quality Services (FQS), the American Association for Laboratory Accreditation (A2LA), and the American Board of Forensic Toxicology (ABFT).

Labs seeking accreditation generally start the process by performing an internal organizational audit. If the lab believes that its policies, procedures, equipment, and other characteristics conform to the required standards, the lab submits a report to the accrediting body. The accrediting body then sends a team of external examiners to confirm the internal audit’s findings. The examiners use a checklist to ensure that the lab’s policies and procedures meet the accrediting body’s requirements, and may also review analysts’ case files for consistency with the requirements. The lab is given an opportunity to respond and correct any deficiencies reported by the examiners before it receives accreditation. The SBI lab’s ASCLD/LAB Inspection Reports can be viewed online here. Accreditation typically lasts for two to five years before the lab must be re-inspected.

In addition to the accreditation offered by the major accrediting bodies, the FBI maintains separate Quality Assurance Standards (QAS) for all labs that participate in the CODIS national DNA database. (Click here to read the specific requirements for these laboratories.)  Labs that participate in the database are subject to annual audits, and the North Carolina State Crime Lab’s website contains links to their QAS audit reports (under the heading “DAB Audit Reports”). Appendix A to each report contains a short summary of all problems uncovered by the audit and the lab’s responses, if any. Appendix D to each report details the certifications of the lab’s DNA technicians.

Although courts have assumed previously that work performed in accredited labs is reliable, an increasing body of evidence shows that accreditation provides little assurance that forensic analyses are carried out with sound scientific methodology. Investigators have uncovered failures in at least 28 accredited laboratories since 2005, including mis-identification of fibers, DNA contamination, mix-ups of physical evidence, expired chemicals, and false calibration dates.

One of the most notable of these failures occurred in the NC State Bureau of Investigation’s (SBI) serology lab, where false and misleading lab reports tainted at least 230 cases over a 16-year period. The SBI lab was accredited by ASCLD/LAB five times during this period; none of its inspectors detected the widespread problems with the lab’s policies and procedures.

Similar failures have been reported in other accredited labs. ASCLD/LAB re-accredited the US Army Criminal Investigation Command in 2006 even after learning that the lab provided inadequate supervision to a DNA analyst who falsified reports, cut corners, and found DNA in samples where none existed. ASCLD/LAB accredited San Francisco’s crime lab in 2005, only to have the lab shut down when it was discovered that an analyst had been stealing drugs from the lab. External audits revealed heavy case loads for lab workers and that technicians were having trouble maintaining the chain of custody for evidence.

Several factors may help explain the failure of ASCLD/LAB and other accrediting agencies to detect major deficiencies in forensic labs. First, examiners review a very small number of cases—typically five per analyst—to confirm compliance, and the cases are selected by supervisors rather than chosen at random. Additionally, examination teams consist of forensic analysts from other labs, so there is some concern that these individuals may be reluctant to report problems for fear of reprisals against their own labs when they are audited. Although ASCLD/LAB accepts complaints, investigations into these issues are kept confidential. Finally, labs are given advance notice of when accreditation inspections will occur, rather than being subjected to surprise inspections.

Part 3 of this series will explain ISO accreditation for forensic labs. Part 1 of this series is available here.

1 Comment

Filed under Crime Labs

Part 1: What’s the difference between accreditation and certification?

This is the first in a three-part series on lab accreditation, analyst certification, and ISO-compliant lab procedures by Ryan Niland.

Understanding the distinctions between lab accreditation and analyst certification can be important when critically examining forensic analyses. Although both processes relate to quality assurance for forensic analysis, the term accreditation generally applies to laboratories as a whole, while certification applies to individual analysts and technicians.

Accreditation refers to a recognition process for laboratories that meet certain standards for management, operations, equipment, and security. Four major accrediting bodies offer accreditation programs for forensic labs in the United States (ASCLD/LAB, FQS, A2LA, and ABFT – we’ll explain more about these organizations in Part 2), and each body has its own requirements. Lab accreditation is generally a voluntary process, but nine states (including North Carolina) now require their labs to achieve some forms of accreditation. Although the accreditation process is intended to promote standardization and quality control, recent scandals at crime labs in North Carolina, San Francisco, and elsewhere have shown that lab accreditation does not always ensure sound science.

Certification refers to a credentialing process for individual analysts and technicians who meet certain standards for education, training, and experience. Innumerable organizations offer certification for forensic analysts in fields ranging from toxicology to forensic art.  Each certifying agency has its own requirements for education, training, and experience, and while some certification programs are rigorous, others can be little more than diploma mills. For example, one organization offers certification as a “forensic consultant” to anyone who watches a 90-minute video and passes a short, multiple-choice test. The Forensic Specialties Accreditation Board (FSAB) monitors the certification process for many of these organizations. A good starting point for determining whether a forensic certification is of merit is to determine whether the certification is offered by an FSAB-accredited organization. More information about the certifications held by NC State Crime Laboratory analysts is available here.

Although most lab accreditation programs do not require that analysts be certified, the FBI’s Quality Assurance Standards (QAS) do require some forms of certification for DNA analysts at labs that participate in the CODIS national DNA database.

Part 2 of this series will explain how a lab becomes accredited and some potential shortcomings in that process.

1 Comment

Filed under Crime Labs