Part 3: ISO and the NC Forensic Sciences Act of 2011

This is the third in a three-part series on lab accreditation, analyst certification, and ISO-compliant lab procedures by Ryan Niland.

In response to concerns about the failures of the North Carolina State Crime Laboratory (formerly known as the SBI lab), North Carolina passed the Forensic Sciences Act (FSA) in 2011. Among other things, the FSA requires that for a report to be admissible in court without the testimony of the analyst, it must be produced by a lab accredited by an accrediting body which is a signatory to the International Laboratory Accreditation Cooperation (ILAC) Mutual Recognition Arrangement for Testing.

ASCLD/LAB’s traditional accreditation program, sometimes referred to as its “Legacy Program,” does not use ILAC standards. Consequently, in September 2012, the North Carolina State Crime Laboratory adopted and posted new lab procedures on its website in order to seek additional lab accreditation according to ILAC standards. ILAC requires forensic laboratories to follow procedures set by the International Organization for Standardization (ISO).

ISO is an organization that develops international standards in many areas, including management, food safety, and nearly all mass-produced products. Forensic labs seeking ISO accreditation must follow the procedures in ISO 17025, which contains general requirements for all testing and calibration laboratories. ILAC has produced guidelines (ILAC G19) for laboratories applying ISO 17025 procedures in a forensic context. ISO 17025 and ILAC G19 set out general principles that labs must adhere to, but each lab that seeks ISO accreditation will have its own procedures interpreting these standards. The UN has put together an excellent guidebook on complying with ISO 17025 available here.

All state lab tests performed after September 17, 2012 should follow the new lab procedures. Analysts at the lab report that the greatest procedural changes occurred in the Forensic Biology/DNA and Drug Analysis/Toxicology sections, as well as in how documentation is kept. The lab procedures for tests administered prior to September 17, 2012 have been removed from the lab’s website. Attorneys should contact the lab to obtain copies of the older procedures, or they can contact the Forensic Resource Counsel for assistance.

The North Carolina State Crime Lab accepted public comment on the new lab procedures, and independent experts provided a number of critiques regarding the new procedures. The critiques included questions about the lab’s forensic biology testing procedures, DNA analysis interpretation guidelines, drug sampling protocols, and results statements used in fingerprint comparisons. The Forensic Resource Counsel has copies of many of these comments and can provide them to you and discuss with you how they may relate to the analysis performed in your case.

Part 1 of this series is available here. Part 2 is available here.

Part 2: The Lab Accreditation Process

This is the second in a three-part series on lab accreditation, analyst certification, and ISO-compliant lab procedures by Ryan Niland.

Several major organizations offer accreditation services for forensic labs in the United States. The American Society of Crime Laboratory Directors/Laboratory Accreditation Board (ASCLD/LAB) is by far the largest accrediting body for forensic labs, followed by Forensic Quality Services (FQS), the American Association for Laboratory Accreditation (A2LA), and the American Board of Forensic Toxicology (ABFT).

Labs seeking accreditation generally start the process by performing an internal organizational audit. If the lab believes that its policies, procedures, equipment, and other characteristics conform to the required standards, the lab submits a report to the accrediting body. The accrediting body then sends a team of external examiners to confirm the internal audit’s findings. The examiners use a checklist to ensure that the lab’s policies and procedures meet the accrediting body’s requirements, and may also review analysts’ case files for consistency with the requirements. The lab is given an opportunity to respond and correct any deficiencies reported by the examiners before it receives accreditation. The SBI lab’s ASCLD/LAB Inspection Reports can be viewed online here. Accreditation typically lasts for two to five years before the lab must be re-inspected.

In addition to the accreditation offered by the major accrediting bodies, the FBI maintains separate Quality Assurance Standards (QAS) for all labs that participate in the CODIS national DNA database. (Click here to read the specific requirements for these laboratories.)  Labs that participate in the database are subject to annual audits, and the North Carolina State Crime Lab’s website contains links to their QAS audit reports (under the heading “DAB Audit Reports”). Appendix A to each report contains a short summary of all problems uncovered by the audit and the lab’s responses, if any. Appendix D to each report details the certifications of the lab’s DNA technicians.

Although courts have assumed previously that work performed in accredited labs is reliable, an increasing body of evidence shows that accreditation provides little assurance that forensic analyses are carried out with sound scientific methodology. Investigators have uncovered failures in at least 28 accredited laboratories since 2005, including mis-identification of fibers, DNA contamination, mix-ups of physical evidence, expired chemicals, and false calibration dates.

One of the most notable of these failures occurred in the NC State Bureau of Investigation’s (SBI) serology lab, where false and misleading lab reports tainted at least 230 cases over a 16-year period. The SBI lab was accredited by ASCLD/LAB five times during this period; none of its inspectors detected the widespread problems with the lab’s policies and procedures.

Similar failures have been reported in other accredited labs. ASCLD/LAB re-accredited the US Army Criminal Investigation Command in 2006 even after learning that the lab provided inadequate supervision to a DNA analyst who falsified reports, cut corners, and found DNA in samples where none existed. ASCLD/LAB accredited San Francisco’s crime lab in 2005, only to have the lab shut down when it was discovered that an analyst had been stealing drugs from the lab. External audits revealed heavy case loads for lab workers and that technicians were having trouble maintaining the chain of custody for evidence.

Several factors may help explain the failure of ASCLD/LAB and other accrediting agencies to detect major deficiencies in forensic labs. First, examiners review a very small number of cases—typically five per analyst—to confirm compliance, and the cases are selected by supervisors rather than chosen at random. Additionally, examination teams consist of forensic analysts from other labs, so there is some concern that these individuals may be reluctant to report problems for fear of reprisals against their own labs when they are audited. Although ASCLD/LAB accepts complaints, investigations into these issues are kept confidential. Finally, labs are given advance notice of when accreditation inspections will occur, rather than being subjected to surprise inspections.

Part 3 of this series will explain ISO accreditation for forensic labs. Part 1 of this series is available here.

Part 1: What’s the difference between accreditation and certification?

This is the first in a three-part series on lab accreditation, analyst certification, and ISO-compliant lab procedures by Ryan Niland.

Understanding the distinctions between lab accreditation and analyst certification can be important when critically examining forensic analyses. Although both processes relate to quality assurance for forensic analysis, the term accreditation generally applies to laboratories as a whole, while certification applies to individual analysts and technicians.

Accreditation refers to a recognition process for laboratories that meet certain standards for management, operations, equipment, and security. Four major accrediting bodies offer accreditation programs for forensic labs in the United States (ASCLD/LAB, FQS, A2LA, and ABFT – we’ll explain more about these organizations in Part 2), and each body has its own requirements. Lab accreditation is generally a voluntary process, but nine states (including North Carolina) now require their labs to achieve some forms of accreditation. Although the accreditation process is intended to promote standardization and quality control, recent scandals at crime labs in North Carolina, San Francisco, and elsewhere have shown that lab accreditation does not always ensure sound science.

Certification refers to a credentialing process for individual analysts and technicians who meet certain standards for education, training, and experience. Innumerable organizations offer certification for forensic analysts in fields ranging from toxicology to forensic art.  Each certifying agency has its own requirements for education, training, and experience, and while some certification programs are rigorous, others can be little more than diploma mills. For example, one organization offers certification as a “forensic consultant” to anyone who watches a 90-minute video and passes a short, multiple-choice test. The Forensic Specialties Accreditation Board (FSAB) monitors the certification process for many of these organizations. A good starting point for determining whether a forensic certification is of merit is to determine whether the certification is offered by an FSAB-accredited organization. More information about the certifications held by NC State Crime Laboratory analysts is available here.

Although most lab accreditation programs do not require that analysts be certified, the FBI’s Quality Assurance Standards (QAS) do require some forms of certification for DNA analysts at labs that participate in the CODIS national DNA database.

Part 2 of this series will explain how a lab becomes accredited and some potential shortcomings in that process.