Why forensic bite mark analysis lacks teeth

Steven Mark Chaney was convicted in 1987 of the murder of a drug dealer who had been stabbed to death. The evidence against Chaney at his original trial included the testimony of a dental expert who stated that it was virtually certain that Chaney had bit the victim on the arm at some point during the killing – that the bite marks found on the victim’s body were a match to Chaney’s. But where did such certainty arise from – and is it justified?

Perhaps not, said the Texas Forensic Science Commission (TFSC), which recommended in February that bite mark evidence should not be used in court. Only months before this recommendation, Chaney was freed from prison “after experts and prosecutors agreed that the bite mark analysis used to secure his conviction was nothing but junk science”, according to the Houston Press.

A subcommittee panel is now trying to determine how many previous convictions in Texas using bite mark evidence need to be reviewed. Nationwide, however, Chaney’s case is by no means unique. Peter Neufeld of the Innocence Project – which works to overturn wrongful convictions – wrote in March 2015 that: “Twenty-four men who were indicted or convicted based largely on bite mark evidence have been exonerated by DNA testing.”

Evidence… or the lack thereof
A 1984 study by Rawson, et al., is often cited as evidence of the unique individuality of bite marks.¹ However, many have criticized this study as flawed, and have cautioned that its results should never be used in a legal case (see box below). Indeed, other research suggests that bite mark analysis is not nearly as reliable in identifying individuals as proponents believe. A 1975 study showed that examiners made incorrect identifications 24% of the time under ideal laboratory settings, and 91% of the time when the bite marks were photographed 24-hours after being made.

In addition, during a 1999 workshop of the American Board of Forensic Odontology, examiners attempted to match four bite marks to seven dental models. The test resulted in a false positive rate of 63%. Meanwhile, a 2001 study of bites made in pig skin produced a false positive rate of 17%.

Compounding the unreliability of such assessments, court records show that both fact witnesses and expert witnesses have misused probabilities when interpreting bite mark comparisons in legal cases.² In Chaney’s case, the court was told there was a “one in a million” chance the bite mark was made by someone other than the defendant – despite there being no scientific basis for such a probability. Meanwhile, in the case of Robert Lee Stinson – who served 23 years in prison for rape and murder based in large part on bite mark evidence – the jury heard how the bite marks “had to have been made by teeth identical” to Stinson’s and that there was “no margin of error” in this conclusion.

Such definitive statements should never have been made, judging by a 2009 report by the National Academy of Sciences.³ The report – ‘Strengthening forensic science in the US: A path forward’ – discussed the use of forensic science by law enforcement and devoted four pages to bite mark comparisons. According to the report, bite mark comparisons are subjective, as there is no agreement among experts regarding the uniqueness, quality or quantity of the individual bite mark characteristics needed for a proper comparison.

In addition, the report noted that there was no central repository of bite marks and patterns – the kind of resource that would be needed to reliably estimate the probability of any particular bite mark characteristic within the broader population. Its verdict was that “the scientific basis is insufficient to conclude that bite mark comparisons can result in a conclusive match.”

The prosecution rests
The American Association for the Advancement of Sciences is working on a bite mark reanalysis report that is expected to be released within the next 12-18 months. However, the evidence we do have leads us to conclude that bite mark analysis is a flawed forensic technique, its deficiencies exacerbated when coupled with dubious statistical statements offered during testimony. A complaint drafted by The Innocence Project and sent to the Texas Forensic Science Commission (and posted online) reflects this view. To summarise the complaint:

1. Bite mark analysis has not been validated or proven reliable;
2. Dentition has never been scientifically demonstrated to be unique;
3. Human skin is not capable of accurately recording dentition features;
4. Bite mark analysis is not generally accepted in the scientific community (see, e.g., the National Academy of Sciences report);
5. Forensic dentists cannot reliable associate a dentition with a bite mark; and
6. Bite marks are prone to serious error and have led to many miscarriages of justice.

Whether the balance of argument and the weight of evidence is enough to convince courts to dismiss bite mark analysis from ongoing and future cases remains to be seen, however.

• H. James (Jim) Norton is Professor Emeritus of Biostatistics, Carolinas HealthCare System, Charlotte, N.C. He has been an expert witness or consultant in eight legal cases including six that involved forensic evidence. George Divine is a senior research biostatistician at Henry Ford Hospital, Detroit, Michigan.
• In the upcoming April edition of Significance, Norton and Divine – along with William Anderson – review the statistical shortcomings of microscopic hair analysis. Subscribe now, or join the Royal Statistical Society or the American Statistical Association to receive your copy.

 

Rawson et al. 
In 'Statistical evidence for the individuality of the human dentition', Rawson et al. concluded that: “The unique nature of the human dentition is confirmed.”¹ But was it? 

Rawson collected wax dental impressions from a convenience sample of 1,200 subjects. The indentations were filled with zinc powder and radiographic overlays were enlarged and hand-traced onto gridded computer paper. A subsample of 397 subjects was used in the final analysis. For each of 12 teeth, the center point for each tooth was plotted, along with a determination of the corresponding x co-ordinate, the y co-ordinate, and the angle of rotation.

These three data points per 12 teeth were entered into a database for all 397 subjects. If two subjects had an x-value within 1 mm of each other, they were considered equivalent (i.e. not distinct from each other). The same 1 mm equivalence was used for the y co-ordinate, while two angles within 5^o of each other were considered equivalent. Then the total number of unique positions was determined for each of 12 teeth, by multiplying the distinct number of x co-ordinates by the number of distinct y co-ordinates and by the number of distinct angles of rotation.

The number of reported total positions ranged from 104 for tooth #27 to 240 for tooth #10. Rawson et al. then computed the product of the number of positions for the six maxillary (upper jaw) teeth as 2.2 x 10¹³, 6.08 x 10¹² for the six mandibular (lower jaw) teeth, and 1.36 x 10²⁶ for both. Since these numbers exceed four billion – the estimated number of people on Earth at that time – the paper concluded that human dentition is unique.

There are a number of flaws in Rawson’s paper, however.⁴ Hand tracings are known to be problematic in regards to accuracy. The sample was not random, and it is not clear how the 397 subjects in the final analysis were chosen. Additionally, the final number of unique possible dentitions was calculated using the product rule, but this calculation is only valid if the data are independent. Yet it would seem obvious that the position of a tooth would be influenced by the positions of the teeth surrounding it. Therefore the product rule does not apply in determining the total number of unique positions for human dentitions.

But the largest error may not be in the calculations but rather in applying these results to bite marks on human skin. Even if a person’s dentition is unique, bite marks on such a malleable material as skin are unlikely to be unique.

 

References

1. Rawson RD, Ommen RK, Kinard G, Johnson J, Yfantis A. (1984) Statistical evidence for the individuality of the human dentition. Journal of Forensic Sciences, (29:1)245-253.
2. Garrett BL, Neufeld PJ. (2009) Invalid forensic science testimony and wrongful convictions. Virginia Law Review, 11(2) 1-97.
3. National Research Council, National Academy of Sciences. (2009) Strengthening forensic science in the United States: A path forward.
4. Pretty IA. (2006) The barriers to achieving an evidence base for bitemark analysis. Forensic Science International, (159S) S110-S120.