Science and Law
11th January 2015

Statistical Significance Testing in Mass Tort Litigation

Statistical significance testing is a fundamental principle relevant to mass tort cases involving complex scientific information (e.g., pharmaceutical and medical device cases as well as toxic torts and consumer fraud cases). Simply stated, if a scientist wants to show that one factor causes changes in another factor (e.g., a drug treatment, a chemical exposure, a source of radiation), conventionally he must rely upon statistical significance testing to demonstrate that difference.

Statistical Significance Testing in Mass Tort Litigation

Statistical significance testing is a fundamental principle relevant to mass tort cases involving complex scientific information (e.g., pharmaceutical and medical device cases as well as toxic torts and consumer fraud cases). Simply stated, if a scientist wants to show that one factor causes changes in another factor (e.g., a drug treatment, a chemical exposure, a source of radiation), conventionally he must rely upon statistical significance testing to demonstrate that difference. Demonstrating that a factor does not cause changes in another factor is a more nuanced and complex issue, but statistical significance testing also becomes a critical factor.

As we have discussed before on this blog, statistical significance testing is necessary because, before concluding that the findings of a study are reliable, we must rule out the role of chance (in addition to addressing confounding and bias). In other words, if we want to rely upon a study showing that an exposure is linked to changes in disease rates, it must be shown that the changes demonstrated in that study are not due to random chance.

Yet the need for significance testing in science is so fundamental that it is difficult to find specific quotes that provide the rules for when and how statistical significance testing must relied upon. For example, when can we rely upon a relaxed p value of 0.1 as opposed to the more conventional 0.05? 

There have been robust debates in the scientific literature – as well as in the courtroom and in the blogosphere – as to when and how one must rely upon statistical significance testing to prove (or disprove) a scientific finding. As an example of this, I urge everyone to take a look at this post by Nathan Schachtman on Deborah Mayo’s blog.

I describe below why this post should be of interest to you and some take-home messages that you may find of value in your practices. 

A Brief History of the Post

First, a brief history of the post and why it should be of interest to lawyers trying cases involving complex scientific issues:

  1. Naomi Oreskes (scientific historian) publishes an OpEd in the NY Times concluding that statistical significance testing is unfairly used by climate science deniers. 
  2. Nathan Schachtman publishes a blog post taking Oreskes to task on her criticism from a statistical perspective 
  3. Mayo cross-posts Schachtman’s post to her blog. 
  4. Mayo receives an abundance of comments from academics, statisticians, lawyers, and other professionals – a virtual confluence of experts in this realm. 

Why You Should Read the Post

It is worth going through the NY Times OpEd, Schathman’s scholarly and provocative post, and all the comments. I attempt to simplify the discussion and extract some of the more interesting and relevant examples below:

  • There are wonderful lessons about the use and mis-use of statistical significance testing in litigation and other controversial issues. For example, Schachtman takes Oreskes to task for her use of EPA’s meta analysis on passive smoking as an example of how to properly apply statistical significance testing. The regulatory record on passive smoking is a rich source for attorneys looking to highlight the problems with conducting meta analyses without a sound scientific basis. 
  • There are valuable links to discussions and authoritative sources that you may find useful in cases involving statistical significance testing. For example, in the comment section, Mayo links to a previous post where she assesses the use of confidence intervals vs. the use of statistical significance testing. While this is a very technical discussion and won’t make for light, bed-time reading, it could prove valuable to discuss with your experts when confronting this important issue in a case. 
  • Nathan links to a prior post where he discussed egregious testimony that misused statistical significance testing. There are a number of valuable examples here that you may find useful for your cases. 
  • Most provocatively, Sander Greenland joins the comments section and provides some excellent tips to defense lawyers who might cross-examine him in the future. He states: 

    “Please Nathan, the next time you or your colleagues cross-examine me, how about asking up front whether I actually think the treatment at issue is harmful based on the evidence as I know it. I routinely forewarn those who ask me to testify that they may not like the answers I give. Interestingly, this has only seemed to turn away defense lawyers, not plaintiff lawyers.”

Essentially, Dr. Greenland (an accomplished and scholarly epidemiologist) is simply saying that he is willing to testify for the plaintiffs on statistical methodological principles, even when he believes there is no reliable evidence supporting their cases. He counsels his adversary to just ask him whether he believes there is sufficient evidence to support the view that the agent in question causes the injury.

 

 

If you can muster the focus, I urge you to read the post and scan the comment stream. To be sure there are a host of different issues embedded here (e.g., use of confidence intervals vs. statistical significance testing, observational vs. randomized trials, confusion of “statistical significance” with certainty, Bayesian vs. frequentist approaches to hypothesis testing) – all of them critical to evaluating the types of data that are relevant to mass tort litigation involving human health endpoints. And clearly none of them are treated comprehensively. Yet, it is a treasure-trove of information that you and your experts may find informative with respect to your cases.

Did you like this post? We are here to help! Schedule a consult with one of our experts.
Just fill out the form bellow and we will contact you with more information.

  • This field is for validation purposes and should be left unchanged.

Recent Related Articles

Dr. David Schwartz’s IADC Presentation: The Use of Genetic Testing in the Courtroom [Download Slides]
15 July 2020
Dr. David Schwartz’s IADC Presentation: The Use of Genetic Testing in the Courtroom [Download Slides]

On Wednesday July 8, 2020, Dr. David Schwartz of Innovative Science Solutions presented at the IADC 2020 Virtual Annual Meeting on a panel titled The Use of Genetic Testing in the Courtroom. A complimentary copy of the panel presentation is now available for download. Read more

The Use of Genetic Testing in the Courtroom: Dr. David Schwartz will be Presenting at the IADC 2020 Virtual Annual Meeting
02 July 2020
The Use of Genetic Testing in the Courtroom: Dr. David Schwartz will be Presenting at the IADC 2020 Virtual Annual Meeting

Dr. David Schwartz of Innovative Science Solutions will be presenting at the IADC 2020 Virtual Annual Meeting on a panel titled The Use of Genetic Testing in the Courtroom. Read more

WEBINAR ANNOUNCEMENT: What’s in Those Genes?
12 June 2020
WEBINAR ANNOUNCEMENT: What’s in Those Genes?

Genetic Evidence Concerning Causation for Mesothelioma 16 June 2020 at 2pm Eastern Daylight Time. Read more

Talc and Asbestos Defendants Should Monitor and Utilize Published Studies Linking Mesothelioma to Genomic Causes
28 April 2020
Talc and Asbestos Defendants Should Monitor and Utilize Published Studies Linking Mesothelioma to Genomic Causes

Plaintiff experts having been asserting for decades that all mesotheliomas must be linked to some asbestos exposure. Indeed, this has led to the erroneous (but widespread) view that mesothelioma is a signature disease, only caused by asbestos exposure. Read more

Let’s work together

The journey to scientific and commercial success is often complex and always critical, if you are looking for an expert partner to help steer you to confident solutions, contact us today

Contact us