Science is not to be trusted

It is to be examined, discussed, questioned, and tested

Science is probably not what you think it is, and that’s okay.

As we can still love our parents when we stop seeing them as infallible God-like “adults” and learn about their full humanity, we see science as a messy process and still love it as beautiful and capable of revolutionizing our society.

Most people learn about science in school as a compendium of facts about the universe. Heat causes liquids to boil and turn into gases. Electric currents can move along a copper wire. DNA encodes the information that makes living organisms what they are.

While many of those facts are true (or more accurately, there is no evidence yet giving reason to doubt them), viewing science as an encyclopedia is misleading, it prevents the public from interfacing with the front-lines of science, and thereby inhibits the ability of science to inform the public in a time of crisis like the COVID-19 pandemic.

Science is anything but constant and monolithic. During the COVID-19 pandemic, the public saw inside the sausage factory of modern science and puked. Did masks work, or did they not? Were school closures effective at saving lives or not? Did vaccines provide long-lasting protection against infection? Did SARS-CoV-2 emerge in a lab? What was sold to the public as The Science one day became misinformation the next, and vice versa. Many members of the public are understandably disoriented at best, revolted at worst. “Distrust” in science skyrocketed among conservatives and “trust” in science rose among liberals. By presenting science as a monolithic belief system, a compendium of facts to be trusted and not questioned, we created a science-policy interface that policed emerging science as misinformation and misled the public about the nature of science itself, all but guaranteeing a partisan response by prohibiting public participation and engagement in the scientific process.

The truth is easy to say: scientists working on the front-lines of science hold different views. We disagree. We read some papers and say “cool! I want to take this idea to the next level”. We read other papers, say “this is garbage!!!” and consider whether it’s worth the time and effort to publish the reasons for our distaste. In the process of thousands of people in any one niche field of science reading papers, agreeing with some and disagreeing with others, replicating some results and disproving others, the collective body of knowledge slowly whittles down to a set of reproducible experiments and theories that have yet to be disproven. The long arc of science bends towards truth, but only if we preserve the integrity of the process by which we disagree and discuss evidence.

Throughout COVID-19, there was considerable effort encouraging people to “follow the science”. The mantra, “follow the science”, was often weaponized in public discourse to suggest “The Science” implied one side’s policies were “right” and the other side’s policies were “wrong”. In reality, throughout the pandemic scientists read the literature, had different assessments of every one paper, and engaged in science by planning & publishing their next work. Whoever came up with “Follow the Science” greatly underserved the public, and we further misrepresent science by asking whether or not folk “trust” science. Science is not a belief system, so it’s not something to be trusted. Science is a social process which anyone can join, it is a conversation with evidence to be examined, discussed, questioned, and tested. Science is not limited to Ivory Towers and people with PhDs. Anyone, no matter how anonymous or weird they are (in our idiosyncratic views of “weird”), can examine a paper, question some results, discuss them, and change our perspectives. Or at least, that’s how it should be.

A personal example of open, public participation in science occurred in some of my own work during the COVID-19 pandemic. In April 2020, Justin Silverman, Nathaniel Hupert, and I suspected confirmed US COVID cases were undercounting the true extent of the pandemic. We counted the number of patients visiting medical providers with influenza-like illness (ILI) in March during previous years and compared it to the number of patients with ILI in March of 2020. We found a significantly higher number of patients with flu-like symptoms in March 2020 than previous years. We combined the number of ILI patients per provider with the number of providers in each state to estimate of the number of ILI patients in each state. We estimated over 20 million people could have been infected across the US in March 2020. More infections, with the same number of deaths, meant a lower probability of dying given infection - this potential good news could dramatically change how we forecasted upcoming COVID-19 surges across the US. People would still die, but maybe medical systems and society would not collapse in states like South Dakota or Florida, where managers opted out of containment policies.

We shared our pre-print on Twitter, it got picked up by data visualization experts at The Economist, and overnight our notifications exploded. Tens of thousands of people read our abstract, and it would’ve been easier to drink out of a firehose than to make sense of that mayhem at that scale. Back in April 2020, saying that COVID might not be as bad as previous estimates (e.g. >1% infection fatality rates) was seen by many scientists as equivalent to saying “COVID is a hoax” but, to me, as a statistician, it was important to share estimates and not bias them based on who said what is a hoax. Many scientists clamored rather unconstructively, saying our paper was garbage, not for any real reason but, rather, because they thought it was “dangerous” or upsetting to public health policy (specifically, the public health policies they preferred - that’s not quite a scientific judgement). We looked out for critiques, and found only critics, until suddenly a person named Seth Stevens-Davidowitz chimed in with a comment deep inside a thicket of threads. Seth’s comment was a good comment.

Seth was not anyone we knew, nor did he present himself as an epidemiologist, nor were we aware of any fancy pedigree. However, Seth pointed out that our approach for scaling up ILI patients per-provider to a state-level, when applied to the whole country, implied many more patients visited the hospital across the US in a year than other reliable measurements suggest. Our results implied too many patients, and we needed to reconcile this. Technically we didn’t “need” to reconcile this - maybe we could’ve squeaked past peer reviewers, since Seth’s comment didn’t go viral, but we believed Seth was right and we were wrong so we felt an ethical obligation to correct our work in light of Seth’s good point. We didn’t ignore Seth nor tell Seth he was unqualified, we didn’t block Seth on Twitter and assert that we were the experts. In fact, Seth didn’t even need to be Seth Stevens-Davidowitz for us to hear the soundness of his point - if an account named RoboCat1984 made the same point, we would’ve heard it all the same, because it was a good point. As scientists, my colleagues and I were eager to keep open minds.

We ultimately agreed with Seth. We realized that the providers giving data to the CDC tended to be large medical providers, so we adjusted our method to scale up ILI visits to the state level in a way that implied our total patients in the US equaled the total patients in the US estimated by other, more reliable methods. Our final paper estimated over 8 million people were infected - still a lot more than the 100,000 cases at the time. Some scientists hated us still. Some said our “flip-flopping” showed how bad we were at science, or that we were dishonest & trying to support Donald Trump. For me, that was just another day in science. We were trying our best, and staying humble, incorporating feedback from smart rando’s on Twitter who made good points.

I stayed involved with COVID-19 forecasting all the way through BA.5, with many other stories I could tell you, but there’s a more important story to focus on today. After forecasting medical demand, I returned to my pre-COVID roots of pathogen spillover to study the origins of SARS-CoV-2. Feeling rather accomplished in the battles over COVID outbreak forecasting like King Richard returning from the Crusades, I expected calm reading by the fire in my castle. I read the literature claiming a lab-origin of SARS-CoV-2 is “impossible” or “implausible” or “improbable”, that the furin cleavage site insertion is “illogical”, that evidence for a zoonotic origin was “dispositive”, and more. Despite initially believing a zoonotic origin, I thought all of that work was garbage.

For example, take Worobey et al.’s analysis of early case data claiming to have found “dispositive” evidence that SARS-CoV-2 originated in the wet market. The paper was entirely within my skillset, and I immediately felt its conclusions were unsound. I believe, as many others have detailed, that the spatial locations of early case data could not determine the origin of an outbreak because (1) the spatial biases in how we collect early cases are impossible to correct for absent transparent background surveillance systems we don’t have in Wuhan (2) the data Worobey at al. used excluded earlier cases with no ties to the wet market, (3) the spatial smoothing of environmental testing misrepresented the relevant granularity, such as surfaces under animal traders being as likely to test positive as surfaces under vegetable traders, (4) Gao et al. tested animals in the wet market and not one animal tested positive, (5) we can’t blindly trust the Chinese government to provide accurate, unbiased data given that, under a lab origin, unbiased data would reveal their fault in the pandemic, and more reasons. Despite Twitter threads highlighting these objections, published papers & many pre-prints, the authors have not addressed any of these reasons, nor have they tempered their language & overconfidence. Instead, Worobey himself continues broadcasting his work without acknowledging limitations or representing the objections of many scientists like myself.

Seth would surely be ignored by this crew, no matter how good his point was.

I read this group’s other preprint - Pekar et al. - and that paper, too, fell within my wheelhouse. That paper, too, has such severe methodological limitations that I have zero confidence in its conclusions. You simply cannot conclude the origins of a virus based on the structure of the virus’ evolutionary tree, certainly not with the models they used to model how viral evolutionary trees grow in early outbreaks, and there’s even strong evidence suggesting the empirical premise - their tree itself - was wrong. I wrote the authors private emails raising my concerns, and they never wrote back. So, I tweeted about it and eventually colleagues and I wrote a paper detailing our reasoning. We shared the paper on Twitter, and the authors attacked us by saying we weren’t “The Experts.” Many proceeded to block me and there was hilarious shit-talking aplenty. With my King Richard armor from years in the COVID warzone, these tweets bounced off me like bullets off of superman.

Just another day in science.

In my scientific due diligence on the origins question, I read careful assessments of the other theory about a lab origin. Lab-origin assessments came from mostly anonymous accounts who feared being called racist conspiracy-theorists by high-follower, blue-check-marked accounts policing this issue on Twitter (including some working with fact-checkers to call lab-origin claims “misinformation”!). Among the anonymous accounts was also a handful of brave, exceedingly brilliant non-anonymous people with obscure institutional affiliations and who, it seems, have yet to be found by the world. Diamonds of scientific human capital in the rough. Some lab origin possibilities were unfounded or loony, yet it’s my job as a scientist to find the signal in the noise and make it known.

So, I studied the evidence suggesting SARS-CoV-2 emerged from a lab and the many scenarios being considered for a research-related origin.

I saw a significant lack of zoonotic evidence that is easy to obtain, evidence that would reject a lab origin, evidence that we even looked for yet couldn’t find. Technically, we still don’t *know* there are no aliens on the moon, or even here on Earth, but we’ve looked for them with methods that should be able to find them if they’re there, and we haven’t found them so they’re probably neither here nor on the moon. Same with the missing zoonotic evidence. In addition to the missing zoonotic evidence, I found the evidence suggesting a lab origin to be very compelling. Most compelling was the constellation of evidence surrounding the DEFUSE grant proposing to insert a human-optimized furin cleavage site in a SARS-CoV infectious clone in Wuhan. Scientists believing SARS-CoV-2 arose from a lab pointed out that, exactly as DEFUSE spelled out in 2018, SARS-CoV-2 emerged in Wuhan with a human-optimized furin cleavage site.

What are the odds of that?

Quite low, it turns out. If we had the DEFUSE grant in-hand in January 2020, when the first SARS-CoV-2 genome was released from Wuhan, we could immediately see the FCS and its human-optimized codon. The odds of such a human-optimized FCS in a SARS-CoV in Wuhan alone (i.e. excluding the infectious clone part) is around 1 in 30 million or so.

However, the puzzle wasn’t complete. Additional evidence could change that number.

Was there any evidence SARS-COV-2 was an infectious clone? In seeking answers on this issue, I stumbled across tweets by Valentin Bruttel and Tony VanDongen, two internet rando’s who I’d never heard of before, yet these two random people were evidently quite intelligent and making truly brilliant points. Valentin’s Avatar looked like it could be the front of a heavy metal album, and Tony’s anonymous-looking avatar of his eye and part of a mask would strike fear in the hearts of lesser men. However, Valentin and Tony were being kind and saying smart things, so I listened.

They noticed that infectious clones are commonly assembled with a known method called “type II directional assembly”, and they visually observed that SARS-CoV-2 appears to have the fingerprint of that exact method. I got in touch with Valentin + Tony and we collaborated to turn this evidence into a paper, with them being awesome bioengineers and me helping quantify the odds of seeing such strong evidence of infectious cloning in a wild coronavirus. We wrote our analysis up in a paper, I wrote a pop-science article explaining what we found, and we tried to use careful language saying SARS-CoV-2’s restriction map is “consistent with” an infectious clone. Language matters a lot in science - we did not say SARS-CoV-2 “is” an infectious clone nor that it “disproves” a natural origin, yet it does suggest a theory that SARS-CoV-2 has a synthetic origin, a theory we encourage people to test, and we believe SARS-CoV-2 is a reverse genetics system, or basically an IKEA virus (whether natural or not).

The Economist picked up the story, and the whole world erupted into battle once again. The Economist article and The Telegraph beautifully document the intensity of scientific discourse on this topic. The language was colorful, to put it delicately. To the best of our ability, we responded kindly to the rather hostile discourse by clarifying who we are and what our intentions are. We listened through the rancor, as I’d done previously to find Seth’s insight on the ILI paper, and we felt this global melee of discourse uncovered some valid points for future research. We acknowledge the scientists who brought up those good points, yet we also felt those points don’t undermine our results inasmuch as they provide additional hypotheses for alternative explanations and future research. Science goes on! After drinking the firehose of rowdy rhetoric and finding some needles of insight in the haystack of hate, we debriefed on this global engagement in a statement that we believe our synthetic origin theory of SARS-CoV-2 still stands.

Another day in science.

As someone who studied & forecasted spillover pre-COVID, my scientific journey has led me to believe SARS-CoV-2 most likely originated in a lab, and the most important piece of evidence contextualizing the rest of evidence suggesting a lab origin is the DEFUSE grant. If you were forecasting the genomic and geographic features of a SARS CoV pandemic using pre-COVID methods, I estimate roughly 1 in 56 billion chance of a SARS-CoV emerging in Wuhan with such a human-optimized furin cleavage site & type II restriction map with such strong resemblance of an infectious clone. If you were forecasting the genomic and geographic features of a lab leak from someone conducting the work in the DEFUSE grant, the virus would emerge in Wuhan and look exactly like SARS-CoV-2 in all of these ways in which SARS-CoV-2 is anomalous among natural coronaviruses. The weight of this evidence is overwhelming. I’ve been around the block and seen a lot of arguments in my days in science, I’ve seen a lot of unresolved issues, yet I have never seen such strong evidence so cavalierly dismissed as zoonotic origin proponents are doing when they say “all the evidence” suggests a zoonotic origin and “no evidence” exists for a lab origin.

Science should not be trusted in general, but we need to be especially diligent in acknowledging science as suspect when the science of the matter concerns the possibility that scientists, health science funders, and managers overseeing science in labs in Wuhan, played a role in killing 18 million people. Such an investigation is ripe with conflicts of interest and reputational risks, as prior to a science-caused accident there will be many coteries of scientists who played some role in encouraging, conducting, funding, and/or overseeing the research that caused harm.

Yet, despite the massive body of evidence making a spillover-scientist like me believe SARS-CoV-2 did not spillover, the zoonotic origin proponents continue to use their media access to broadcast their papers without giving some time or fair consideration to objections to their papers. Rather than engage with the public, they block any scientist, let alone member of the public, who disagrees with them. They claim they, alone, are The Experts, and when someone raises an objection they simply talk louder to more media outlets & more followers. They greatly misrepresent the evidence of the matter in outlets as widely read as Washington Post and LA Times, corrupting the interface between science and society, misrepresenting both science as a collective process with multitudinous views, and repeatedly misrepresenting in a reliably biased way the facts of the matter during ongoing congressional investigations. The authors repeatedly claim to be summarizing “all the evidence”, yet nowhere do they discuss the severe, mathematically provable limitations of their work, the objections of other scientists they’ve blocked, or the many pieces of evidence suggesting a lab origin.

Nowhere in “all the evidence” do they mention DEFUSE or the many features of SARS-CoV-2 shockingly consistent with DEFUSE.

Yet, they want the public to trust them, to follow their science.

To me, these scientists’ promulgation of their flawed work and their willful (or oblivious? which is worse?) biased exclusion or misrepresentation of the evidence of a lab origin is one of the worst research ethics violations in human history that I am aware of, second only to creating the virus itself. There is the crime, and there is the coverup putting media-grabbing scientists who misrepresent the facts of the matter in league with the researchers who conducted the work on CoVs in Wuhan and refuse to share their lab notebooks or databases. These scientists are asserting themselves as authorities while brushing aside credible objections to their work irrespective of who raises them. In the middle of congressional investigations on SARS-CoV-2 origins, these scientists are writing op-eds that mislead the public and managers on the probable research-related cause of over 18 million deaths worldwide, using their expertise to obfuscate an historic truth and obstruct the investigations we need to make our world safe from dangerous research.

My scientific journey studying SARS-CoV-2 origins has led me believe that a small coterie of scientists are, in fact, responsible for creating SARS-CoV-2 in a lab. They, their funders, and many scientists connected with them & the funders, and many scientists who championed doing this risky research are all reliably abusing their status as experts to misrepresent facts of the matter. The researchers studying CoVs in Wuhan are refusing to share their research. Peter Dasak refused to share his DEFUSE grant or admit conflicts of interest of working on CoVs with labs in Wuhan when writing letters to The Lancet calling lab-origin theories “conspiracy theories”. Funders at NIH, NIAID, and the Wellcome Trust prompted, edited, and pushed a paper claiming baselessly with overconfident language that lab-origin theories are “improbable” or “implausible”.

As recently as yesterday, and during our desperately needed congressional investigations on SARS-CoV-2 origins, this clique of scientists is still running media campaigns claiming “all of the evidence” suggests a natural origin without ever mentioning DEFUSE. The relationship between science and society is a delicate one, and it’s one we’re still figuring out, yet clearly something is wrong with this picture. It is beyond unprofessional and unethical for scientists to run mass-media campaigns that misrepresent the evidence of the matter during congressional investigations into the possibility that the scientists they are connected with created a virus that killed three times more people than the Holocaust. Assertions that they are experts to be followed misrepresents science and its consultations (not leadership) of society, and their efforts to obstruct investigations into their own syndicate should be seen as comparable to oil companies muddying the science about climate change, tobacco companies muddying the science about lung cancer. Scientists who staked their reputations to risky research that likely led to millions of deaths are today muddying science itself.

Science should not be trusted. I say this as a scientist. Science has always been a rebellious act, a foray into battle with narratives that be. Richard Feynman described science as “belief in the ignorance of experts”. Science is not about the answers, per se, it’s about questioning the answers and trying to disprove the theory du jour, it’s about the longer-arc of the social process by which we share evidence and evaluate competing ideas. In times of crisis, science is not to be followed - it is to be examined, discussed, questioned and, for managers, incorporated alongside myriad other factors such as anthropological variation in folks’ beliefs, capacities, and wills to act.

While we learn about science in school as an encyclopedia of facts, the reality is that science is an epistemological warzone with ground rules, and we’re continually updating those ground rules as we go. The ground rules need to be revisited in light of the probable lab origin of SARS-CoV-2 and the actions of many scientists misrepresenting the evidence of the matter during WHO and congressional investigations of a potentially science-related catastrophe. There is a high likelihood that scientists amidst us, who fought beside us in this epistemological warzone, in a frenzied rush to get funding and fame, created a virus that leaked from a lab in Wuhan and resulted in over 18 million people dead, over 60 million extra people facing acute hunger, over 100 million kids thrown into multidimensional poverty, and an endemic curse of outbreak cycles that will infect our kids, our grandkids, and every generation as long as contemporary science can foresee.

The gravity of the situation should make all of our hearts sink. It should lead us to have a moment of silence every day. Instead, we see scientists claiming “all the evidence” suggests a natural origin in mass media outlets. Indeed, all the evidence can say anything you want it to once you omit all the evidence suggesting otherwise. I worry these conflicts of interest, biased representations of evidence, and gross imbalances of media power can corrupt the social process of science.

We are living through an unprecedented crisis. Throughout history, science has battled over paradigms and slowly the long arc of science has bent towards Truth, but none of those paradigm shifts pertained to science itself, least of all to the possibility that preeminent scientists with unprecedented mass-media reach played a role in an unprecedented atrocity. Compared to what science is capable of, SARS-CoV-2 was a petite Pandora’s jewelry box in an Amazon warehouse of larger possibilities, and some scientists are abusing their authority and expert-status to obstruct investigations that could inspire policies that stop scientists from opening other, bigger boxes in the Pandora’s Warehouse of modern biotechnology.

Please, do not “trust” science and do not blindly trust scientists, least of all those who exhibit a pattern of misrepresenting the entire facts of the matter on SARS-CoV-2 origins (the truth, the *whole* truth). Love science & scientists, even those with whom we disagree in glorious epistemological combat, but do not trust us.

Keep an open mind that even scientists like me can and will make mistakes. As someone members of the public view as “a scientist” I pledge to listen for good ideas no matter where they come from and do my best to update my thinking in light of new evidence. I will correct my mistakes and acknowledge whoever helped me see the light. Engage, question, discuss, and test science. Please, don’t stop there. For the love of future generations, please manage science, because we have failed to manage our own. Only by democratizing the skeptical essence of science and welcoming everyone to this epistemological battlefield with ground rules can we learn the mistakes of COVID-19 and collectively bend the long arc of science towards Truth.

Please, let’s improve the interface between science & society for the benefit of both.