The content of these pages is developed and maintained by, and is the sole responsibility of, the individual senator's office and may not reflect the views of the Nebraska Legislature. Questions and comments about the content should be directed to the senator's office at firstname.lastname@example.org
In 2015, a study appeared in the academic journal International Archives of Medicine titled “Chocolate with high Cocoa content as a weight loss accelerator”. Within weeks news of the study was reported in over 20 countries in print, on morning TV shows, and in online stories. The study’s results showed that the participants on a low carbohydrate diet who ate 1.5 ounces of dark chocolate daily lost more weight compared to those eating a low carb diet alone, were too good to be true.
The headlines were sensational. The coverage was global. The study was a hoax.
Filmmaker Peter Onneken and journalist John Bohannon developed the scenario to demonstrate how a scientifically unsound study, literally “junk science”, could be published by a journal and reported on by the media with no critical evaluation of the merits or validity of the data. Although the study was actually conducted, the sample size was very small, only 15 people, and the analysis of the results lacked standards and rigor expected of scientific inquiry. Not a single journalist examined the methodology or results of the study before reporting it to the world. In a very graphic and indisputable way, Onneken and Bohannon made their point.
On September 14 a report was filed with the Nebraska Legislature on the progress of a $250,000 pilot study of cannabidiol oil paid for by Nebraska taxpayers at the University of Nebraska Medical Center. LB 390, passed in 2015, authorized the study and required a progress report to the Legislature each year by September 15. The report filed is less than a page and a half in length and contains the information required by law.
The legislative report is not, by any stretch of the term, a scientific study. While the number of participants are listed (23), and some of the side effects are enumerated, there is no analysis of the significance or magnitude of the observations. The report states “the majority have demonstrating [sic] benefit”, but does not state what those benefits are, if “majority” means merely 12 of the 23, if the 3 patients who dropped out of the study due to negative effects are included, or whether the drug had different impacts on the 11 minors in the study compared to the adults. The small sample size, large number of variables that are mentioned, and the diversity of the study subjects are all red flags for anyone critically evaluating the outcomes of the study.
That did not, however, stop media from overstating conclusions based on the brief status report of the project. A September 15 Lincoln Journal Star article repeated the text of the report, without ever examining whether the results were statistically valid. In response to an email I sent to the report’s author, the UNMC Associate Vice Chancellor for Clinical Research, on September 18, he replied “the principal investigator is working with a statistician now on the data” and that complete data was not yet available even for review. Given the very small number of study participants, it is impossible to distinguish any observed effect from random chance without careful and rigorous statistical analysis.
The lack of statistical analysis or even a complete data set didn’t seem to matter to the Lincoln Journal Star editorial board, who proclaimed in a September 21 editorial headline “UNMC study reiterates need for a medical cannabis law”, even though the “study” didn’t even examine medical cannabis. The cannabidiol oil in the study is actually a commercial product that received FDA approval in June of this year. Just a few days later, the Omaha World Herald News Bureau also cited the “study” in an article about proposed medical marijuana legislation. Senator Anna Wishart, who is again proposing the policy, sent the Lincoln Journal Star article to all senators via email on September 18 as “evidence” in support of her medical marijuana policy.
When media articles and legislative reports are used as the basis for making policy, they should, at minimum, be based on sound and rigorous analysis. Making conclusions based on a small, incomplete data set that has not yet undergone any peer review is reckless and irresponsible. The failure of the report’s author and the journalists to inform the public that the data set has not yet been subject to analysis has the potential to mislead and misinform. A single email was all it took for me to obtain the facts.
While the chocolate-weight loss scenario showed how absurd sensationalized science reporting can be, the reporting on the cannabidiol study is not a joke. When taxpayer money is used to fund a study, the assumption is the results will be presented in a valid, unbiased manner. I have recommended to the report’s author he amend his filing with the Legislature to reflect the incomplete and unanalyzed status of the data. Verifying the veracity of information should be the first job of a journalist, especially when their articles are used by lawmakers and editorial boards to promote public policy. This instance indicates lawmakers cannot assume accuracy in newspaper articles without verifying the details themselves. The facts matter. Public trust and human lives are at stake.