Science and Law
26th March 2014

Don’t Believe Everything You Read – Part 1

Scientific misinformation is rampant in the media and public policy arena. News stories citing misleading statistics, weak studies and uninformed sources can unjustly influence litigation. Two organizations founded by author and college professor S. Robert Lichter aim to rectify scientific misinformation and provide access to valid studies.

Don’t Believe Everything You Read – Part 1

STATS Combats Scientific Misinformation

Scientific misinformation is rampant in the media and public policy arena. News stories citing misleading statistics, weak studies and uninformed sources can unjustly influence litigation. Two organizations founded by author and college professor S. Robert Lichter aim to rectify scientific misinformation and provide access to valid studies. STATS and CMPA are valuable resources for attorneys building cases that depend on accurate scientific information.

STATS has garnered praise from the media and the scientific community. Scientific American hailed the organization’s methodology as “respected and widely adopted.” STATS director of research Rebecca Goldin, senior fellow Jon Entine, stats.org editor-at-large Trevor Butterworth, and contributing editor Geoffrey Kabat are at the top of their fields.

In an interview with ISS, Dr. Lichter discussed how STATS and CMPA can provide valuable insight to attorneys. Part 1 of the interview, which pertains to STATS, appears below. Check back next week for Part 2, which pertains to CMPA.

ISS: You founded STATS in 1994 as a resource to correct scientific misinformation in the media and in public policy. In your view, what are the implications of misinformation in the media resulting from bad science?

RL: If public policy isn’t informed by accurate, reliable evidence, how can we possibly hope that policy—and by extension politics—can deliver meaningful solutions to any of the problems we face? How can the public have any assurance that government can work and that experts actually know what they’re talking about? How can individuals make informed, reasoned decisions about their behavior, if the information they are receiving is exaggerated, spurious, or simply wrong?

These are questions that concern policymakers across the board, but they are particularly relevant to science-related legislation and regulation, and the public support on which they ultimately rest. In theory, you should be more confident if your decisions are based on scientific evidence. In practice, all too often bad science drives out good science in the marketplace of ideas.

ISS: What would you say is the most egregious example of the misuse of science in the last 10 years?

RL: You can look at this in two ways. The first is a general misunderstanding by journalists of what science is and how it can produce reliable knowledge through experimentation and replication—the scientific method, in other words. It is not clear that many journalists grasp this in practice, given the way new findings—often portrayed as miraculous or malevolent—get reported without context, without reference to the totality of experimental findings on a particular hypothesis. This also leads to the endless reporting of weak evidence—observational studies and statistical associations—as if such findings have the same weight as randomized control trials. The result is the familiar and oversimplified story line of “new study says X causes/reduces the risk of Y.”

Take the case of something as familiar and widely used as coffee. As Stanford University’s John Ioannidis pointed out in the British Medical Journal last November, there are over 34,000 papers in PubMed with the keywords coffee or caffeine. You can find a study associating either with almost any outcome. But journalists are particularly bad at contextualizing new research against the weight of the evidence, what scientists have concluded an entire body of research actually says is true or is most likely to be true. Instead, the more novel the finding, the more compelling the story. This is toxic to the public’s understanding of science.

Second, among individual controversies, the most extraordinary one is bisphenol A (BPA). It is difficult to think of an ongoing controversy where the key evidence, the randomized control trials and massive exposure studies by agencies such as the EPA, FDA and CDC, has been consistently ignored in favor of small sample size studies and cross sectional analyses. It is even more remarkable that both the FDA and EPA have repeatedly failed to replicate the original small studies that kicked this controversy off, and yet the media narrative is that the small studies are the ones we should pay attention to.

ISS: ISS serves the legal community on cases involving complex scientific issues. Our clients are lawyers that serve as outside and inside counsel to pharmaceutical, dietary supplement, cosmetic, chemical, and consumer product manufacturers. In what way would you say STATS is relevant to our clients?

RL: Analytical journalism is very popular right now, but we’ve been taking deep dives into the statistical and scientific basis for major and minor controversies since 1994. In the last several years we’ve moved in the direction of adding original reporting to our analyses, whereas the media have been trying to add numbers to their reporting. We believe that, fundamentally, numbers tell the story. We simply go further than any other publication with detailed in-depth reporting, often using sources who are ignored by the media because they don’t court publicity or communicate in sound bites. It takes time and effort, but when we nail the data, we drive other media coverage.

Our work is often paired with studies by the Center for Media and Public Affairs, which produces content analyses of media coverage and surveys of scientists. So when we characterize the nature of media coverage, or the opinion of the scientific community, our conclusions are based on comprehensive and systematic evidence, rather than anecdotal impressions or gut instinct. Our approach is analogous to Billy Beane’s “moneyball” approach in baseball or Nate Silver’s 538 blog in politics – we trust analytics and hard data over personal judgment.

Did you like this post? We are here to help! Schedule a consult with one of our experts.

Just fill out the form bellow and we will contact you with more information.

  • This field is for validation purposes and should be left unchanged.

Recent Related Articles

Dr. David Schwartz’s IADC Presentation: The Use of Genetic Testing in the Courtroom [Download Slides]
15 July 2020
Dr. David Schwartz’s IADC Presentation: The Use of Genetic Testing in the Courtroom [Download Slides]

On Wednesday July 8, 2020, Dr. David Schwartz of Innovative Science Solutions presented at the IADC 2020 Virtual Annual Meeting on a panel titled The Use of Genetic Testing in the Courtroom. A complimentary copy of the panel presentation is now available for download. Read more

The Use of Genetic Testing in the Courtroom: Dr. David Schwartz will be Presenting at the IADC 2020 Virtual Annual Meeting
02 July 2020
The Use of Genetic Testing in the Courtroom: Dr. David Schwartz will be Presenting at the IADC 2020 Virtual Annual Meeting

Dr. David Schwartz of Innovative Science Solutions will be presenting at the IADC 2020 Virtual Annual Meeting on a panel titled The Use of Genetic Testing in the Courtroom. Read more

WEBINAR ANNOUNCEMENT: What’s in Those Genes?
12 June 2020
WEBINAR ANNOUNCEMENT: What’s in Those Genes?

Genetic Evidence Concerning Causation for Mesothelioma 16 June 2020 at 2pm Eastern Daylight Time. Read more

Talc and Asbestos Defendants Should Monitor and Utilize Published Studies Linking Mesothelioma to Genomic Causes
28 April 2020
Talc and Asbestos Defendants Should Monitor and Utilize Published Studies Linking Mesothelioma to Genomic Causes

Plaintiff experts having been asserting for decades that all mesotheliomas must be linked to some asbestos exposure. Indeed, this has led to the erroneous (but widespread) view that mesothelioma is a signature disease, only caused by asbestos exposure. Read more

Let’s work together

The journey to scientific and commercial success is often complex and always critical, if you are looking for an expert partner to help steer you to confident solutions, contact us today

Contact us