OHDSI Symposium Panel-Building Trust: Evidence and its Communication
The 2020 OHDSI Symposium (www.ohdsi.org/ohdsi-2020-global-symposium-schedule/)is October 19 for the main day, and at 1pm ET is a panel on trust in science that should be fun (www.ohdsi.org/symposium-panel-building-trust/).As always, the Symposium is free; you just need to register.
The panelists areas follows:
• Amy Abernethy, MD, PhD, the Principal Deputy Commissioner of the U.S. FDA
• Patti Brennan, RN, PhD, the Director of the National Library of Medicine, NIH
• Magdalena Skipper, PhD, Editor in Chief, Nature
• Deborah Nelson, JD, Associate Professor of Investigative Journalism, University of Maryland
• Roni Caryn Rabin, MS, science reporter for the New York Times
Here is the 1pm panel content:
Building Trust: Evidence and its Communication
COVID-19 was a new infectioninflicting enormous suffering with no known treatment and little known aboutits transmission. The scientific community roared into action, quicklygenerating evidence from observational data and beginning numerous trials. Theeditorial and peer-review process was overwhelmed, and the urgent need forevidence required rapid assessment of the studies' rigor and importance. Highlypublicized retractions and corrections contributed to a general feeling that alarge proportion of the published work was substandard and misleading. TheOHDSI community strives to avoid substandard work by providing a publicprotocol before beginning research, reporting results only frommulti-institution studies, verifying results with a full set of diagnostics,executing data quality checks, reporting data provenance, etc. Such rigor takestime, however, and given the current paradigm of celebrating being"first," there remains little opportunity to provide reliableevidence after the journals have accepted early questionable results. Suchundue focus on speed gives the unfortunate impression that once sufficientunreliable evidence has been published, reliable answers are no longer needed.
Point and counterpoint 1
· The scientific community took onan unprecedented challenge and produced observational evidence at ahigher-than-ever rate, and while some studies were faulty, on the whole, itproduced useful information that benefited the fight against COVID-19.
· Based on the poor, misleadingevidence that has been published in the past and for COVID-19, the scientificprocess must undergo major revision.
Point and counterpoint 2
· An enormous number of studies,including observational research and small clinical trials, were placed onarchives with no review and reported on in the press; this must be bettercontrolled.
· The archives provided a legitimatemeans of supplying early results, and the press well understood the limitationsof the archives and generally reported the caveats.
Point and counterpoint 3
· With the emergence of large-scaletechniques to generate evidence and the inability of reviewers to adequatelyjudge studies with thousands of results, the publishing community must engagemore tightly with the evidence generation community, creating new models topromote evidence generation and assess the validity of the evidence.
· The publishing community mustremain distant from the evidence generation community to maintain scientificindependence and avoid conflicts.
(The plenary session is, "Large-scalenetwork phenotype development, evaluation and characterization," andshould also be interesting. It's an international symposium, so we are runningit for 18 hours with some duplication to cover the many time zones, and theplenary is at 1am and 8am ET.)