FBI's Forensic Test Full of Holes

Convictions are in question now that FBI forensic evidence has been discredited

November 18, 2007

By John Solomon, Washington Post Staff Writer

Hundreds of defendants sitting in prisons nationwide have been convicted with the help of an FBI forensic tool that was discarded more than two years ago. But the FBI lab has yet to take steps to alert the affected defendants or courts, even as the window for appealing convictions is closing, a joint investigation by The Washington Post and "60 Minutes" has found.

The science, known as comparative bullet-lead analysis, was first used after President John F. Kennedy's assassination in 1963. The technique used chemistry to link crime-scene bullets to ones possessed by suspects on the theory that each batch of lead had a unique elemental makeup.

In 2004, however, the nation's most prestigious scientific body concluded that variations in the manufacturing process rendered the FBI's testimony about the science "unreliable and potentially misleading." Specifically, the National Academy of Sciences said that decades of FBI statements to jurors linking a particular bullet to those found in a suspect's gun or cartridge box were so overstated that such testimony should be considered "misleading under federal rules of evidence."

A year later, the bureau abandoned the analysis.

But the FBI lab has never gone back to determine how many times its scientists misled jurors. Internal memos show that the bureau's managers were aware by 2004 that testimony had been overstated in a large number of trials. In a smaller number of cases, the experts had made false matches based on a faulty statistical analysis of the elements contained in different lead samples, documents show.

"We cannot afford to be misleading to a jury," the lab director wrote to FBI Director Robert S. Mueller III in late summer 2005 in a memo outlining why the bureau was abandoning the science. "We plan to discourage prosecutors from using our previous results in future prosecutions."

Despite those private concerns, the bureau told defense lawyers in a general letter dated Sept. 1, 2005, that although it was ending the technique, it "still firmly supports the scientific foundation of bullet lead analysis." And in at least two cases, the bureau has tried to help state prosecutors defend past convictions by using court filings that experts say are still misleading. The government has fought releasing the list of the estimated 2,500 cases over three decades in which it performed the analysis.

For the majority of affected prisoners, the typical two-to-four-year window to appeal their convictions based on new scientific evidence is closing.

Dwight E. Adams, the now-retired FBI lab director who ended the technique, said the government has an obligation to release all the case files, to independently review the expert testimony and to alert courts to any errors that could have affected a conviction.

"It troubles me that anyone would be in prison for any reason that wasn't justified. And that's why these reviews should be done in order to determine whether or not our testimony led to the conviction of a wrongly accused individual," Adams said in an interview. "I don't believe there's anything that we should be hiding."

The Post and "60 Minutes" identified at least 250 cases nationwide in which bullet-lead analysis was introduced, including more than a dozen in which courts have either reversed convictions or now face questions about whether innocent people were sent to prison. The cases include a North Carolina drug dealer who has developed significant new evidence to bolster his claim of innocence and a Maryland man who was recently granted a new murder trial.

Documents show that the FBI's concerns about the science dated to 1991 and came to light only because a former FBI lab scientist began challenging it.

In response to the information uncovered by The Post and "60 Minutes," the FBI late last week said it would initiate corrective actions including a nationwide review of all bullet-lead testimonies and notification to prosecutors so that the courts and defendants can be alerted. The FBI lab also plans to create a system to monitor the accuracy of its scientific testimony.

The Post-"60 Minutes" investigation "has brought some serious concerns to our attention," said John Miller, assistant director of public affairs. "The FBI is committed to addressing these concerns. It's the right thing to do."

The past inaction on bullet-lead contrasts with the last time the FBI's science was called into question, in the mid-1990s, when 13 lab employees were accused of shoddy work and of giving overstated testimony involving several disciplines, including explosives as well as hair and fiber analysis. Back then, the Justice Department reviewed hundreds of cases in which FBI experts testified, and it notified prisoners about problems that affected their convictions. The government did so because prosecutors have a legal obligation to turn over evidence that could help defendants prove their innocence.

Current FBI managers said that they originally believed that the public release of the 2004 National Academy of Sciences report and the subsequent ending of the analysis generated enough publicity to give defense attorneys and their clients plenty of opportunities to appeal. The bureau also pointed out that it sent form letters to police agencies and umbrella groups for local prosecutors and criminal defense lawyers.

Even the harshest critics concede that the FBI correctly measured the chemical elements of lead bullets. But the science academy found that the lab used faulty statistical calculations to declare that bullets matched even when the measurements differed slightly. FBI witnesses also overstated the significance of the matches.

The FBI's umbrella letters, however, glossed over those problems and did little to alert prosecutors or defense lawyers that erroneous testimony could have helped convict defendants, one of the recipients said.

"Frankly, the letters that they sent them, you know, were minimizing the significance of the error in the first place," said defense lawyer Barry Scheck, whose nonprofit Innocence Project has helped free more than 200 wrongly convicted people. The letters said that "our science wasn't really inaccurate. Our interpretation was wrong. But the interpretation is everything."

The FBI said last week that the 2005 letters "should have been clearer." Scheck has now been asked to assist the FBI's review.

Since 2005, the nonpartisan Forensic Justice Project, run by former FBI lab whistle-blower Frederic Whitehurst, has tried to force the bureau to release a list of bullet-lead cases under the Freedom of Information Act. The Post joined the request, citing the public value of the information. But the government has stalled, among other things seeking $70,000 to search for the documents.

"By stonewalling and delaying the release, Justice has ensured that wrongfully convicted citizens are deprived of their right to appeal or seek post-conviction relief because the statute of limitations in many states has expired," said David Colapinto, the lawyer for the group.

As part of its review, the FBI will release all bullet-lead case files involving convictions.

The Scope of the Cases

Most of the estimated 2,500 instances in which the FBI performed bullet-lead exams involved homicide cases that were prosecuted at the state and local levels, where FBI examiners often were summoned as expert witnesses for the prosecution.

To compile an independent list, The Post and "60 Minutes" conducted a nationwide review, interviewing dozens of defense lawyers, prosecutors and scientific experts. The effort also included a sweep of electronic court filings conducted by four summer associates at the New York law firm Skadden Arps Slate Meagher & Flom.

In many of the cases that raise the most compelling questions, the inmates might have a hard time winning the public's sympathy. Some had criminal backgrounds and most were convicted with at least some additional circumstantial evidence linking them to gruesome crime scenes. But the common thread is that removing the flawed bullet-lead evidence has created reasonable doubt about guilt in the minds of legal experts, the courts and at least one juror.

In North Carolina, Lee Wayne Hunt, 48, remains in prison after being convicted 21 years ago of a double murder. Hunt was an admitted marijuana dealer, but has steadfastly denied involvement in the killings. The FBI testified that its bullet-lead analysis linked fragments from the victims to a box of bullets connected to Hunt's co-defendant. That was the sole forensic evidence against Hunt. State prosecutors recently conceded that the analysis should not be considered "scientifically supported and relied upon."

In addition, the attorney for Hunt's co-defendant, who committed suicide in prison, has since declared that his client carried out the murders alone.

Despite both developments, Hunt has been denied a new trial.

"What they're relying on here is technicalities to keep an innocent man in prison," said Richard Rosen, Hunt's attorney.

Another North Carolina case highlights the impact that FBI bullet-lead testimony had on local jurors. James Donald King faces execution after being convicted of killing his two wives. He admitted to killing his first wife, spent time in prison, was released on parole, remarried and then was convicted of murdering his second wife.

The court is considering whether to grant a new trial.

"If the state had not introduced evidence linking a bullet in Mr. King's car to the bullet fragments in the victim, there would have been reasonable doubt in my mind as to Mr. King's guilt," juror Michelle Lynn Adamson said in an affidavit supporting his appeal.

Other defendants have had mixed results:

• In Maryland, the Court of Appeals last year reversed the murder conviction of Gemar Clemons and ordered a new trial, concluding that the FBI's bullet-lead conclusions "are not generally accepted within the scientific community and thus are not admissible."
• In New Jersey, courts have reversed and reinstated convictions in cases involving bullet lead. The conviction of one defendant, Michael Behn, was reversed, but he recently was re-convicted on other evidence.
• Shane Ragland's conviction in the 1994 killing of a University of Kentucky football player was reversed after Kathleen Lundy, an FBI bullet-lead examiner, pleaded guilty to giving false testimony in his case about bullet-lead manufacturing. A few weeks ago, Ragland pleaded guilty to a lesser charge and is now free.

Ernest Roger Peele, a retired FBI agent who testified about bullet matching in 130 cases, stands by his testimony but said that sometimes the nuances of science get "lost in the adversarial nature of the courtroom." He said he would no longer tell jurors that bullets can be linked to specific boxes because of the science academy's findings.

Peele, who said he was frustrated that he was never contacted by the academy, added that his bullet matches were meant to be "a part of a puzzle" and never the only forensic evidence. "Is it possible there are innocent people in jail? Yes. Is it possible that bullet lead was part of that process? Yes."

The Origins of the Science

The FBI's bullet-lead analysis was created more than four decades ago to link suspects to crimes in cases in which bullets had fragmented to the point where traditional firearms tracing -- based on gun-barrel groove markings -- would not work.

So FBI scientists used chemistry to try to find matches. Their assumption was that bullets made from the same batch of lead would have the same chemical composition. U.S. bullet-makers recycle lead from car batteries and melt it down in huge amounts, and it was believed that each batch would produce bullets sharing the same trace elements.

The FBI first used the technique after Kennedy's assassination, hoping to determine whether various bullet fragments came from the same gun. In July 1964, then-FBI Director J. Edgar Hoover wrote to the commission investigating the assassination that the bureau's findings were "not considered sufficient" to make any matches.

By the early 1980s, the bureau was the only practitioner of the science and routinely used it to help state and local police link crime-scene bullets to those in a gun or a box owned by a suspect. There are few federal murder statutes, but the FBI routinely helps local law enforcement by providing forensic expertise in homicide cases.

In the mid-1990s, Lundy used the science to help prove that Clinton White House lawyer Vincent W. Foster committed suicide, internal FBI documents show.

In the early days, bullet fragments were subjected to neutron beams that would allow scientists to measure the presence and amounts of at least three chemical elements: antimony, arsenic and copper. If two bullets had similar measurements of those three elements -- the FBI allowed for a small margin of error -- they were declared a match.

In 1996, the bureau switched to a new method called "inductively coupled plasma optical emission spectroscopy," in which scientists identified and measured seven trace elements in the bullets, adding the elements bismuth, cadmium, tin and silver. The goal was to increase the precision of the tests. But at the same time that it was measuring more elements, the FBI doubled the margin of error for declaring matches.

"Not enough suspects were being caught in the new net using seven elements, so they chose to use a bigger net," said Clifford Spiegelman, a statistician at Texas A&M University who reviewed the FBI's statistical methods for the science academy.

The bureau conducted a study in 1991 that called bullet-lead analysis a "useful forensic tool" that produced "accurate" and "reproducible" matches.

The study, however, raised two concerns.

First, it found that bullets packaged 15 months apart -- a span that assumed separate batches of lead -- had the exact composition, potentially undercutting the theory that each batch was unique.

Second, it found that bullets in a single box often had several different lead compositions. That finding, it cautioned, should have "significant impact on interpretation of results in forensic cases."

Peele, the retired bullet-lead examiner, was the primary author of that study. He said he still felt comfortable having told jurors in the past that bullets from the same box could be expected to match, as long as his remarks were carefully qualified.

In the Hunt case, he testified that his match of the crime-scene bullets to those in the suspects' box was "typical of everything we examined coming from the same box or the next closest possibility would be the same type, same manufacturer, packaged on or about the same day."

Peele said that he always tried to tell jurors that some bullets in the same box might not match. Still, he said it was reasonable for jurors to conclude that matching bullets could have come from the same box. "I don't think it's misleading as long as it's fully explained," he said.

Some of Peele's colleagues went further. FBI examiner John Riley told a Florida jury: "It is my opinion that all of those bullets came from the same box of ammunition." A New Jersey prosecutor suggested that the bullets matched by the FBI were as unique as a "snowflake or fingerprint."

Today, the FBI regards all such testimony as inaccurate. "The science does not and has never supported the testimony that one bullet can be identified as coming from a particular box of bullets," said Adams, the retired FBI lab director.

A Challenge From Within

The FBI's about-face was prompted by a challenge from within its ranks.

William Tobin, an FBI lab metallurgist for a quarter-century, won accolades working on cases such as the crash of TWA Flight 800, in which he helped prove that the plane was downed by an accidental fuel-tank explosion, not terrorism. Shortly before he retired, Tobin was approached by a woman who believed that the bullet-lead science used against her brother, a New Jersey murder defendant, was flawed. Still employed by the bureau, Tobin was not permitted to help.

But when he retired in 1998, he decided to look further. Bullet matching had always been done by the lab's chemists, and as a metallurgist, Tobin wondered about their assumptions. Soon he joined with Erik Randich, a metallurgist at Lawrence Livermore National Laboratory.

By 2001, the two had finished a study that challenged the key assumptions that the FBI had been making about bullet lead. They found that bullets made from the same batch did not always match, because subtle chemical changes occurred throughout the manufacturing process. Tobin bought bullets at several stores in Alaska and found that a large number of bullets with the same composition and manufacturing date were often sold in the same community, suggesting that it was wrong to assume that a bullet match could be narrowed to one suspect.

"It hadn't been based at all on science but, rather, had been based on subjective belief," Tobin said in an interview. "Courts, and even practitioners, had been seduced by the sophistication of the analytical instrumentation for over three decades."

Soon, Tobin began appearing as a witness for defendants challenging FBI bullet-lead matches. Courts began to take notice, too, and the FBI suddenly faced a barrage of questions about a science that had gone unchallenged for three decades.

Adams asked the National Academy of Sciences in 2002 to examine the FBI's work, temporarily halting new bullet-lead matches. Two years later, the academy's findings stunned the bureau.

The panel concluded that although the FBI had been taking accurate bullet-lead measurements in its lab, the statistical methods and its expert testimonies were flawed.

The science "does not . . . have the unique specificity of techniques such as DNA," and "available data does not support any statement that a crime bullet came from a particular box of ammunition," the panel concluded. All the FBI could say going forward was that bullets made from the same batch "are more likely" to match in chemical makeup than those made from different batches. Adams soon declared that such testimony was so general that it had no value to jurors, and he ended the technique.

The FBI Response

The FBI went on the offensive to portray its decision in the best light.

In a news release dated Sept. 1, 2005, the bureau declared that it "still firmly supports the scientific foundation of bullet lead analysis" but that it was ending the technique because of the questions about its "relative probative value," the "costs of maintaining the equipment" and the "resources necessary to do the examinations."

The bureau also sent form letters to the more than 300 police agencies it had assisted with the science and to the umbrella groups representing local prosecutors and local criminal defense lawyers so they could "take whatever steps they deem appropriate."

The letters cited the academy's report but did not call attention to the magnitude of the FBI's internal concerns.

For instance, the letters stated that the impact of the academy's findings "on previously issued examination reports remains unaddressed." In fact, the FBI had conducted its own review to determine how often bad statistics led to mistaken matches.

In March 2005, the chief of the FBI chemistry unit that oversaw the analysis wrote in an e-mail that he applied one of the new statistical methods recommended by the National Academy of Sciences to 436 cases dating to 1996 and found that at least seven would "have a different result today." Marc A. LeBeau estimated that at least 1.4 percent of prior matches would change.

If the FBI employed other statistical methods the number of non-matches would be "a lot more," LeBeau wrote. In fact, when the bureau tested one method recommended by the academy on a sample of 100 bullets, the results changed in the "large majority of the cases," he wrote.

Despite the concerns, the FBI provided affidavits in at least two cases seeking to help prosecutors sustain convictions that were based on bullet-lead matches.

In one such affidavit introduced in Maryland, the FBI cited the academy's report but did not mention it faulted the bureau's statistical methods.

That omission concerns the chairman of the academy panel.

The affidavit "does not discuss the statistical bullet-matching technique, which is key and probably the most significant scientific flaw found by the committee," said Kenneth MacFadden, a private chemistry expert.

MacFadden and Spiegelman said they also believed the affidavit was misleading, because it estimates that the maximum number of .22-caliber bullets in a batch of lead was 1.3 million. The academy said the number could be as high as 35 million.

In a May 12, 2005, e-mail, the deputy lab director told LeBeau, "I don't believe that we can testify about how many bullets may have come from the same melt and our estimate may be totally misleading."

FBI officials said Friday they will stop using the affidavit.

"They said the FBI agents who went after Al Capone were the untouchables, and I say the FBI experts who gave this bullet-lead testimony were the unbelievables," Spiegelman said. ____________________________________________________

"60 Minutes" correspondent Steve Kroft and producers Ira Rosen and Sumi Aggarwal, Washington Post research editor Alice Crites and staff researcher Madonna Lebling, and freelance researcher Jilly Badanes contributed to this report.