I recently quipped that Peter Hotez’s lament of “Anti-Science” is an attempt to restore his image, since he’s a “fallen general of an epistemological banana republic”.
This quip was an intuitive shot from the hip, as with most of my writing, but those words stuck in my head a while after writing because they capture some very complex feelings of being a scientist in modern epidemiology & virology. Science is, in a way, a societal doll within a Russian doll of societies, and the particular scientific society I grew into as a scientist has what I feel are some serious problems that make it feel, at times, like a failed state. Don’t get me wrong, science is awesome and it’s a smidge hyperbolic for me to call it a “failed state”, but the concept of a failed state captures some institutional frustrations and inefficiencies and, with it, some possible solutions.
There are many metrics of a failed state, but the general idea is that in a failed state the government has lost its ability to govern. For nation-states, “the ability to govern” boils down to an ability to control the populace, tax people, control borders, and otherwise perform some necessary functions of a state. There is some disagreement on what functions are ‘necessary’ and whether such functionalist thinking is objective or rather a projection of one’s own values for how a state ought to be. Yet, the more abstract concept of a failed state is that, insofar as states have necessary functions, states that fail to satisfy those desirable functions are failing.
Institutions are established and complex patterns of human behavior; “pattern” implies repetition and reliability. A State is an institution that defines systems of government which can tax people, build militaries to defend people, create & enforce laws, and persist as a pattern of human collective behavior (humans voting, showing up to bureaucratic offices and doing their jobs, etc.).
Science is another institution, another established and complex pattern of human behavior within states which ideally follows a particular process (the scientific method) to learn new things. The scientific method, however, is an idealized and microscopic process within far messier, macroscopic institutions that determine how we fund and disseminate the results of this process. Science is funded in large part by states, so while some may question the ethnocentricity of declaring other countries as “failed states”, it seems more within the scope of our nation to assess whether scientific institutions are satisfying the functions we desire, functions which underlie our political will to fund scientific institutions. If our scientific institutions are not living up to the promise and following an impartial, unconflicted, and safe application of the scientific method to advance our societal goals, it’s within the scope of our system of government to refine our government’s relationship with and even management of scientific institutions.
Some of the scientific institutions I grew up in feel exceedingly dysfunctional, and we can upgrade science if it’s not functioning the way we want it to.
The Function of Science for the State
Why do we fund science?
I am a scientist and most of the money I’ve received in my scientific career has come from taxpayers through a series of federal agencies that aim to support research and development. Why do we support research and development? From the perspective of the state, research and development can lead to the innovation of new goods and services that increase the competitiveness of our economy, the health of our population, the dominance of our military, and the global standing of our nation. From the perspective of people, science provides all the aforementioned benefits along with a less tangible benefit of “learning cool things” that inspire us.
Learning the age of the universe or proving a neat mathematical theorem about knots, for example, improves the global standing of our nation’s scientists while also possibly inspiring people. Such “basic” science may not have immediate applications, but all applications come from basic science. There’s endless debate about how much to fund basic science that lacks immediate applications versus applied science that leads to immediate technological developments, but the overarching function of both, from the perspective of a State, is to boost scientific, economic, and military competitiveness.
Having defined the functions we wish science to satisfy to justify funding scientific institutions, we’re in a position of asking whether or not scientific institutions are satisfying those functions. If there are inefficiencies or concerns about our investment in science, we the people of a representative government and our representatives in government are in a position to change the institutions of science to improve the system’s function and increase the return on our investments in science.
By and large, I would argue that many scientific disciplines have produced significant returns on investment. In the past century, we have produced antibiotics that cure bacterial infections, vaccines that prevent and even eliminate diseases, novel treatments that dramatically improve the health of our population and increase our life expectancies by nearly two decades. Imagine if you were dying in 1923 at the age of 57 and someone told you that you could live another 20 years. How much would you pay for those 20 years? For me, personally, prolonging life by 20 years and reducing the burden of morbidity for 77 years of life would be worth substantial investment from my own pocketbook. The past 100 years of health science has tangible value measured in both market capitalization of tech companies as well as years in our lives.
Beyond health sciences, science has completely reshaped our society. We drive cars with such high fuel efficiency, and live in global networks of travel and trade, that many people in the US can afford to effortlessly cross the entire continent in a climate-controlled box of metal moving faster than the fastest land animal can sprint. If you don’t want to drive, you can fly a plane, a tube of metal rocketing through the sky at 500 miles per hour as we casually breathe normally and peer out the window moving faster than the fastest bird in the world at elevations higher than the tallest mountain in the world. While you’re on the plane, you can send invisible beams of light to communicate with WiFi, you can send messages to satellites orbiting our planet in space, and those satellites can post articles on Substack that the entire world can access. We are able to split atoms to produce energy from nuclear fission, we have achieved net gain on nuclear fusion, we have towering wind farms that convert wind itself into energy stored on a battery. From a biological perspective, humans have unlocked novel forms of metabolism that can power our technology and even lead to the conception of artificially intelligent models like ChatGPT trained on an astronomically vast amount of text posted on the internet, models that may very well have passed a Touring test 100 years ago.
Science is incredible from the eyes of an evolutionary biologist. With just 100 years of science, we now have dogs in space, we have satellites connecting the world on the internet and we have hominids sipping coffee in airplanes at 37,000 feet above sea level. Through science, a species on the planet has innovated complex behaviors in just 100 years that allow us to outrun a cheetah, outfly a peregrine falcon, out dive a sperm whale, outlive our ancestors, and outrun pathogen evolution with treatments and vaccines. The technology we’ve been able to produce from our scientific systems over the past 100 years is incredible. Our society, thanks in large part to science, has evolved so fast in the past 100 years it looks like a singularity of evolutionary accomplishments when plotted on the long scale of 3.6 billion years of life before us.
The Structure of the Scientific State: The Sausage Factory
I love science. Yet, like any scientist, I seek to improve things, including our scientific systems. We like to simplify science as “the scientific method”, and indeed the scientific method is the distinguishing behavior in scientific systems. However, our implementation of the scientific method at scale has led to modern scientific institutions that are messy and prone to inefficiencies and biases at every step in the life cycle of an idea.
We can’t gloss over the fact that the public saw inside the scientific sausage factory during COVID and rightfully vomited at some of our inefficiencies. We’re wise to zoom out and see the big picture, to examine the failures of scientific societies as well as the remarkable successes and strive to increase the successes while decreasing the failures. By understanding the structure of the scientific society and how it leads to dysfunction, we may be able to improve it.
Let’s just jump right into a contentious yet nuanced topic: vaccines. We’re wise to evaluate costs and benefits of vaccines and, at the same time, appreciate the modern mRNA vaccine technology is truly remarkable. If you asked our ancestors 100 years ago to make it, they wouldn’t be able to. They wouldn’t even know what we’re talking about. Through investment in basic science, we learned the central dogma of molecular biology, how that double-stranded DNA is the object of inheritance, how it is transcribed to mRNA and how mRNA is translated into proteins like the Spike protein of SARS-CoV-2 that can be recognized by our antibodies. To build the mRNA vaccines, scientists utilized our knowledge of the conversion of DNA to mRNA to proteins to create mRNA that makes proteins that generate immunity to disease.
We should evaluate the side effects thoroughly (something a vocal minority of scientists seems to discourage for fear of what we might learn or how ‘the public’ might interpret it). The technology for producing the vaccines may need to be improved to reduce impurities like DNA in what’s intended to be just mRNA. However, while we critically examine this new technology it’s also fair to acknowledge that it is our investments in science that produced this modern technology inaccessible to the world just a few decades ago.
As we applaud our scientific systems for the success of this technology, however, we also have to examine ways our scientific systems worked against this technology. After all, mRNA vaccines were not produced without frictions from our scientific society, and those frictions didn’t disappear overnight with a bunch of tweets chastising scientists from 20 years ago for being so oblivious and cruel in their treatment of Katalin Kariko, one of the scientists who played a pivotal role in the invention of mRNA vaccines. Dr. Kariko was denied funding, unable to publish, and consequently denied tenure and effectively voted off the scientific island. If we applaud her accomplishments, we must similarly critically evaluate why our scientific systems worked against her. As someone who played team sports my whole life, I learned the value of debriefing after every game and examining mistakes even after games that we won. Scientists should do the same.
That sausage factory we all peered into during COVID is the sausage factory I’ve inhabited my entire adult life, so it’s a mess I know quite well. The frustrations Dr. Kariko experiences are still present, in fact they may have gotten worse over the past 20 years. Like everyone else, I would like to fix the scientific sausage factory to improve the ability of our scientific institutions to fulfill their desired functions. To fix anything, we have to first acknowledge that it’s broken. We have to be critical of our institutions and the processes by which the inventor of mRNA vaccines was notoriously denied federal funding for her idea. She was denied funding so reliably that the university denied her tenure and she was forced to pursue her idea outside the scientific societies propped up by state funding. Thankfully, Dr. Kariko had the conviction to pursue her idea as an entrepreneur in the private sector.
The private sector is a remarkably innovative place. Were it not for such a competitive private sector science scene, it’s likely we wouldn’t have our COVID vaccines. If you have a good idea with a large potential market, and you can secure the idea through a patent, then you can find investors to support your idea and build a team to bring products to market. The private sector is remarkable at generating scientific products, but it isn’t alone. While most patents come from private sector, most private sector researchers like Dr. Kariko received formal training in academic labs, and hence our federal funding of science determines the funding and career advancement of academics that, in turn, can determine the pool of candidates spinning off to generate products in the private sector. The private sector also has its own limitations. It can be much more lucrative for a company to research and develop treatments than cures, because if you cure someone then you lose a customer. As someone who gets sick, I would prefer a cure over a treatment any day. For cures, we need good systems of funding science that’s not supported by the private sector.
So our nation funds science through grants that support academic as well as private sector work but there’s not just one nation that funds science. There’s an entire globe of countries that each funds science in their own way. In addition to nations, there are giant pots of money invested in assets and using the foam on top of their capital gains to fund science. Which ideas do those pots of money fund? They fund the interests and passions of whoever founded and/or runs the pot of money. We call these self-interested pots of money “foundations”, like the Wellcome Trust, Bill & Melinda Gates Foundation, and so on.
How does funding happen?
Grants
States and foundations post grant calls and scientists propose their research to these calls - let’s call this process grant inception. Grant inception allows science funders to specify the set of things they’re hoping to fund in this grant cycle. A grant call has the benefit of clarifying what managers are interested in, but of course it has the cost of potentially imposing too much top-down structure on what’s being funded. Sometimes funding agencies and foundations solicit feedback from scientists to get ideas on what they should post on the next grant call. This has the benefit of letting scientists drive grant inception, but it will inevitably represent only the scientists who are in the room providing feedback. Rooms of scientists seem like a good idea until you realize a committee would be so averse to every edge of Michelangelo’s David they would whittle it down to a sphere. Another problem with having many scientists in room deciding which grants to fund is that not all scientists are in the room. Someone like Dr. Kariko who was voted of the academic island, or even someone like me who has criticized NIAID for funding risky research and refusing to be transparent about this research, isn’t likely to be in the room, even if we want to be. Hence, there are many ways that scientific paradigms (e.g. a zoonotic origin of SARS-CoV-2) and research cartels (e.g. the Zoo Crew, with support from NIAID) can become entrenched by having disproportionate influence on grant inception.
Once grant calls are posted, scientists spend a lot of time (IMO way too much time) submitting grants. Once a massive stack of massive grants is submitted, a process of grant selection whittles down the many proposed ideas into a few that are funded. The process of whittling down grants often includes a round of bureaucratic rejections. For example, I had one grant rejected because I forgot to put the Project Management Plan inside the Supplemental Information folder. The Project Management Plan was in another folder, so my idea for revolutionary new diagnostic methods was rejected. Was that the best procedure to select revolutionary ideas?
After weeding out people who aren’t good at satisfying bureaucratic constraints (e.g. creative people and/or people with ADHD who have great ideas but difficulty following bureaucratic instructions), grant selection then typically involves recruiting a committee of people (often scientists) appointed by the state or foundation. Another room of scientists, with the same potential biases in their composition. The scientists review and score grants almost completely subjectively based on the perceived impact, feasibility, and other armchair assessments of the idea. If you get a reviewer who hates your guts, possibly from a previous ego-fueled scientific battle or because of your political views or possibly because they want to do your project, there’s nothing other than unenforceable ethics pinky-promises stopping the reviewer from giving your grant a shit score and providing made-up reasons for that shit score, thereby sinking your research proposal. This process of grant selection is the process that denied the inventors of mRNA vaccines their funding. A combination of grant inception that didn’t cover her work and grant selection that rejected it left Katalin Kariko unable to fund her research and develop very useful, Nobel-prize-winning technology.
Grants, and the people with the power of grants, can also influence scientists in their process of thinking about ideas. The public now knows that in February 2020, Kristian Andersen and Robert Garry had an NIAID grant worth $9 million next to a stack of other grants with similar high scores. Which grant will be funded? That was a decision left to Dr. Anthony Fauci. In an separate matter, Andersen wrote Anthony Fauci, the head of NIAID, that he believed SARS-CoV-2 may have leaked from a lab, that its genome was “inconsistent with expectations from evolutionary theory”. Fauci was aware that his agency had given grants supporting research on SARS CoVs at the Wuhan Institute of Virology, so Fauci was aware that a lab accident could involve research he supported and lead to serious investigations, especially if the virus caused a pandemic that led to significant loss of life.
With those elephants in the room, Fauci, Francis Collins (head of NIH), and Jeremy Farrar (head of Wellcome Trust) jumped on a call with Andersen et al. to discuss the matter. Already, having Fauci, Collins, and Farrar in a room is an intimidating presence - if one were to upset the people on this call, or if one were to look like a fool, these kings of health science funding had the power to reject the $9 million grant in that pile and put you on a bureaucratic shit-list that limits your ability to get funding from these kings of health science funding and their large pots of money. The lords of health science funding support a feudal system of science governance that gives the lords considerable influence over not just which ideas are funded, but which ideas scientists feel are acceptable to propose.
Because Andersen was in the middle of grant selection at the agency Dr. Fauci oversaw, Dr. Fauci had considerable power over Andersen et al. Thus, it was particularly disconcerting that this “discussion” of a possible lab origin did not involve an impartial review of bioattribution experts, the head of the CDC, forensic biologists from the FBI, and beyond. No. Dr. Farrar, a foreign national acting on behalf of our US government officials, instead invited other scientists with massive reputational risk in the matter, like Ron Fouchier who conducted gain of function research on the bird flu and Marion Koopmans who was close colleagues with Daszak, the PI of the NIAID grant sending subawards to the Wuhan Institute of Virology. These conflicted scientists reportedly berated Andersen et al. in front of the kings of health science funding, and after that call we see Andersen et al. in their subpoenaed Slack communications complain of the political consequences of publicly accusing anyone of creating SARS-CoV-2, even though they believed that was the truth. An idea was off limits because the people who funded risky virological work in Wuhan organized a biased set of scientists in the room to pressure Andersen et al. with the predictable result of forcing them to feel the idea of a lab origin of SARS-CoV-2 could affect their ability to get funding. The de-inception of an idea through informal social control is but one of the unwritten powers available to science funders.
Dr. Farrar then used his considerable influence to write editors of a major journal (Nature Medicine) to encourage they publish Andersen et al’s new paper with the new idea that the virus couldn’t possibly have originated in a lab funded by Farrar, Fauci, and Collins. Farrar helped them change the language a bit without being listed as a co-author or even having his contributions listed in the Acknowledgements. In science, we call that unethical practice “ghostwriting” - if there’s a paper claiming a drug is effective, we want to know whether or not the maker of the drug helped you write that paper. Similarly, if your paper relates to a possible research-related origin of a virus, we want to know if people who funded that research helped you write the paper.
In short, the public got to see how massive foundations can wield considerable influence in science, how science funders wield immense power in our scientific system, giving them the ability to potentially mold science by grant inception and grant selection, or even the elephant in the room that upsetting these funders could affect one’s ability to get future funding. Scientists exist in a somewhat feudal system, we must kiss the rings and answer to the lords who fund us, and those lords could very well choose to fund only the scientists who play by their rules and support the interests of the lords. This feudal structure isn’t limited to the US. Want to know how to get data from the Chinese government, as Eddie Holmes has over the course of his career? You must kiss the rings of the Chinese Communist Party and never say anything that upsets them, like the idea that SARS-CoV-2 originated in a lab, or you will be denied access to data and funding that can help you make a great paper.
A feudal system of powerful lords controlling ideas and research directions from the top-down was not what we had in mind when we wanted to scale up “the scientific method”.
Publications
After someone luckier (or better-connected to peers on grant review committees) than Dr. Kariko receives funding, perhaps someone as fortunate as Kristian Andersen whose $9 million grant was later funded by Dr. Fauci after Andersen wrote the idea Fauci pressures on him and not the idea Andersen conceived, scientists use the money to hire staff, students, and post-docs, they conduct research, and they publish it. The scientific method happens, in theory through getting hands-on-deck and conducting research. The scientific method is more or less going according to plan in modern science, and people who do bad, irreproducible work are usually identified by the irreproducibility of their work. While the scientific method is functioning well in my opinion, the publication of research is another aspect of our scientific sausage factory that differs materially from what the taxpayer & State may have in mind when they fund science.
Publications are short and ideally reproducible summaries of our work that can be costless posts on pre-print servers (for many fields like physics, pre-prints are as good as peer-reviewed papers) or costly publications in peer-reviewed journals. Publishing an open-access article on Nature, for example, costs $11,700. In return for scientists paying Nature over 1/3 Americans’ median income for a single article, scientists’ work is disseminated to the broad reader base of Nature, increasing the reach of a scientist’s work and giving scientists a line on their CV that increases their chance of getting tenure, getting future grants, and so on.
Journals do not accept all the papers that hit their editors’ desks, nor are those publication fees necessary to cover costs - scientists write papers for the journals, scientists review papers for the journals, scientists pay the journals for this exploitative process to kiss the rings of the journals, and Elsevier, the parent company of Nature, posts profit margins of 40%. Only about 8% of articles submitted to Nature are accepted, so it’s worth examining how papers get accepted in the same way we examine which scientists get welcomed into the room with Fauci, Collins, and Farrar.
How does Nature choose which papers to accept?
For starters, you have to get your foot in the editor’s door. You can get to know editors by reviewing papers or networking at conferences, but that only works if you were invited to review the papers or present at the conference, so they’re meeting editors presents an endogeneity problem and can feel like an old boy’s club. If you have a paper written, you then write cover letters attempting to give editors high-level views of the impact of your science but with the caveat that the editor is not a subject matter expert in your field of study so they may not be able to see the scientific big-picture. It certainly helps get your foot in the door if, for example, you have a well-connected juggernaut like the head of the Wellcome Trust pushing through the door, but not everyone gets that luxury and that sort of network is an old boy’s club that requires a cultural goodness of fit, thereby excluding people who are not the same culture as Farrar or Holmes (both Oxford alumni, and who gets to be Oxford alumni has its own well-known biases). Editors reject a lot of papers merely because the busy editor, in a cursory review of the cover letter, cannot recognize the names of the authors nor see the “impact” (an irreproducible and thereby unscientific metric), and so many scientific dreams are dashed merely because of the ignorance and scientific blindness of editors or the lack of connections from one’s unfortunate schooling or cultural differences from the juggernauts of the field. Welcome to epidemiology and virology!
The fault of rejection biases doesn’t fall only on the editors. Our journals that determine the dissemination of science are institutions that are themselves self-interested, so Nature as a business chooses papers to maximize its profits and maximizing profits requires being cognizant of revenue streams, carefully curating its image as a prestigious journal while navigating potential frictions and avoiding thorny political issues that could upset its revenues. Non-profit journals like Science are a distinction without a difference - they still care about revenues, they answer to their board, and they too have editors that are not oracles of all of science so they too will reject many valuable papers because of a mix of institutional and editorial disinterest.
The potential conflicts and inefficiencies of state-funded science publications filtered by journals are worth considering. The parent company of Nature - Elsevier - received almost $10 billion in revenues in 2019. The revenues of Elsevier are derived from a mix of publication costs from scientists all over the world, sponsored content, and subscriptions. The parent company of Elsevier - RELX group - is technically a British-based company but it has subsidiaries all over the globe, including in countries like China whose authoritarian government’s interests do not align with the interests of the citizens of the United States. This tangled web of interests ultimately underlies corporate actions of RELX that can determine the editors, which in turn can affect which science is disseminated and whose interests are served in the publication process.
Consider, for example, science concerning a possible lab origin of SARS-CoV-2. SARS-CoV-2 emerged in Wuhan, so the labs in question under a lab origin theory are first and foremost Chinese state-sponsored labs like the Wuhan Institute of Virology. Any scientific paper that finds evidence of a lab origin of SARS-CoV-2 and is submitted to Nature will land at the editor’s desk, and if the editor were to choose to publish the article it could upset the Chinese government and thereby interfere with the revenues of RELX and Elsevier by cancelled subscriptions or denied permits to pursue subsidiary companies in China. Consequently, research suggesting a lab origin of SARS-CoV-2 will have a high risk of pushback (if not an outright blockade) from up high.
Nature Medicine published the now notorious Proximal Origin paper by Kristian Andersen, Eddie Holmes et al. (with the help of uncredited co-author and Holmes’ old boy Jeremy Farrary). In Proximal Origin, Andersen et al. presented obviously untrue arguments and dubious logic to promulgate the false claims that a lab origin of SARS-CoV-2 was “implausible”. The paper was promulgated further by the editor of Nature Medicine claiming the article “debunked” the “conspiracy theory” that SARS-CoV-2 emerged from a lab, and a massive surge of media attention followed, including Dr. Fauci using his position as head of NIAID addressing the American people to point everyone to this paper without acknowledging his role in its conception. The false claim of a debunked lab origin is very convenient for health science investigators who funded the labs and for the Chinese government, as both would be subject to significant scrutiny if the truth of a more-likely lab origin were published instead. Now that subpoenaed Slack communications are revealing the authors knew their claims were untrue at time of writing, with Kristian Andersen famously saying a lab origin was “so friggin likely”, the journal is trying to walk-back the Proximal Origin paper by saying it was just an Opinion, not an Article, a distinction without a difference given the paper is cited as a publication all the same and even sent around by Fauci to US government agencies like the State department claiming that it is authoritative science, not merely an opinion.
As a matter of curiosity, I wrote the editors of Nature Medicine and asked if they would entertain publishing an alternative opinion. After all, I studied in and published on pathogen spillover pre-COVID as part of a DARPA PREEMPT team, and I (alongside many others) have helped uncover some of the evidence of a lab origin. It seems reasonable that a thriving scientific system would welcome such discourse as disproving old ideas is the essence of science. The editors responded that they would not publish an alternative opinion. The editors claimed “not enough new data has emerged”, despite the fact that almost all the evidence has emerged since April 2020: the 2018 DEFUSE grant proposing to insert a furin cleavage site in a bat sarbecovirus reverse genetics system in Wuhan, the restriction map of SARS-CoV-2 appearing a lot like a reverse genetics system, evidence of deleted early outbreak sequences, reports of BSL-2 conditions in the Wuhan Institute of Virology, reports of three WIV researchers falling ill with COVID-like symptoms, and so much more. Furthermore, sometimes science advances by improved analyses of old data. Why did Nature Medicine not publish my article?
Welcome to the commonly felt injustice at the heart of our scientific publication system. Why was my paper/idea among the 92% that are rejected?
My n=1 sampling of editorial decision-making at Nature Medicine is not conclusive regarding the motives and processes behind which articles (or Opinions) get published. The stated claim of “not enough new data” is somewhere between subjective and false, so the real reason for the rejection was unstated. It’s possible they didn’t accept the paper because publishing the paper could risk upsetting the Chinese government and disrupting their revenues. It’s also possible that I didn’t get my foot in the door because I didn’t have the head of the largest health science research foundation, Jeremy Farrar at the Wellcome Trust, in my old boy’s network to prop the door open for me like a good chap. It’s possible I’m not sufficiently famous to maintain their claims of VIP-prestige, or too controversial or disliked because I am averse to kissing rings, I hold uncommon political beliefs, and I advocate for policies that were disliked by the heads of health science funding (stranger things have happened).
Such is the nature of editorial decisions that determine scientific publication and dissemination. We can’t disentangle the reasons for a paper being rejected, and they could range from an editor’s ignorance to a journal’s geopolitical conflicts of interests. Yet, this is the process that massively biases what scientists and the world that funds science reads as the latest ideas coming out of science.
The Proximal Origin saga is but one unfortunate window into our sausage factory that revealed how scientific publications come to be, how scientific narratives can be drafted and disseminated with conflicts of interest, ghostwriting, and even risks of influence from adversaries of the United States, and how thousands of dollars to publish each article thusly are paid by the American taxpayer. The editorial rejection of my offer to publish a second opinion years later, after mountains of new evidence emerged, is also a classic example of the highly debatable reasoning behind most rejections in that pile of 92% of papers that don’t get their day in the court of Nature.
That’s how scientific sausage is made.
Grants determine who and what is funded enough to do work, and then publications in journals whose interests may not align with ours determines what science the world reads. The publication process above goes on to determine the success or failure of scientists, the advancement or stalling of their careers, the acceptance or rejection of their future grants. At every stage of this process, we encounter humans in a social system and a scientist’s social goodness-of-fit is often more important for an idea’s passage through every institutional gate than the merit of the idea itself.
Scientists who fit into this system, by whatever mix of true excellence and social or cultural-political conformity, become tenured, get invited to conferences, sit on grant committees, and consult our government in times of emergency. Scientists who don’t fit in, scientists who are blocked at any one of the many unfair bottlenecks of scientific production, are forced to pursue careers elsewhere. That’s how we’ve determined today’s tenured scientists, those are the experts you read in the news. Those scientists in turn are brought into the room to pitch their ideas for grant inception, which determines what work is funded, which pays for publications at outlets whose editors and shareholders may impose their own filters.
That is the life cycle of an idea, and a scientist, in our modern scientific system. Ideas are only as valuable as the cultural-political goodness of fit of the scientist proposing them, and ideas can face additional friction if they upset feudal lords of scientific funding or data access.
The COVID-19 pandemic showed the public this sausage factory of modern biological sciences. I’ve lived in this sausage factory for what feels like my whole life, and it has only gotten worse over time. The public saw unreasonable papers rise to prominence and fame thanks to the pre-existing prominence and fame of the authors (a famous instance of this was Nowak et al back in 2014). The public saw theories peddled by scientists with conflicts of interest (e.g. Eddie Holmes’ having spent most of his career kissing rings and getting connected with the science funding systems in China) only to be back-pedaled by independent scientists whose research was called “confected nonsense” by none other than the peddlers. Do you think Eddie and his chap Farrar will fund my future research? The epistemological warzone of science played out in public on the unlevel playing fields of scientific funding and dissemination systems, it resulted in hype cycles of irreproducible work divorced from the humility or cautious nuance of true scientific advancement, and it resulted in policies that underserved people and resulted in inequitable adverse public health consequences.
I wish I could say that biological sciences during COVID was not typical of these fields, but it is frustratingly familiar. The same frustrations we’ve observed during COVID are why John Ioannidis wrote in 2005 that most research publications are false and why Henry Bauer wrote in 2004 that knowledge monopolies and research cartels (like Andersen et al.) are barriers to meritocratic science. Scientists have cried for help for decades now, we have called for reform of granting & publication systems, and in the COVID-19 pandemic the public has had a chance to see why.
Cleaning the Sausage Factory
I critique our scientific systems because I love science. In the same way someone points out holes in a ship because they love the ship, I point out these weaknesses and failures of our scientific societies because I love our ship of state and I want science to be better able to steer us towards prosperity.
How can we fix this mess?
Thankfully, there’s a somewhat less-problematic field of science that is studying how we can fix science, it’s the science of science called “metascience”. Meta-scientists study everything from why most scientific research findings are false, how your scientific social network determines your productivity and success, how peer-review is biased, and more. I can’t cover all of meta-science and I won’t present myself as some kind of expert on this topic, but I will say that the question of how scientific societies fail to fulfill their desired functions is something we can study, understand, and fix. Metascientists are the doctors who may help us fix the systemic problems in our modern scientific systems, problems that I and other scientists have lived through for most of our adult lives.
These problems have solutions, if only we’re brave enough to imagine and implement them. For example, in the day and age of the internet we can ask: why do we have journals? People like to argue that peer-review improves the quality of the scientific literature, but the combination of journal-institutional, editorial, and scientific-paradigmatic biases lead to a publication trail in biology that is largely irreproducible. Irreproducible science is bad for the economy because it results in wasted R&D expenditures as people try to productize an overhyped idea only to waste their time and resources on something that was only published because the editor liked it and it confirmed their pre-existing beliefs. Journals are also highly inequitable - it’s absurd to be charging $11,700 for the public to access an article that they funded. The high costs of publication means that broke people can’t publish. No matter how revolutionary your ideas are or how much they improve our economic or military competitiveness, if you’re poor then you can’t publish in Nature. That’s morally outrageous and it’s also not fair competition - such publication systems favor incumbents who have piles of grant money, which, recall, they obtained from their long history of fitting in with this biased scientific system and pleasing the feudal lords of science funding.
There are alternatives. Pre-print servers allow us to post our work free of charge and they allow the public to access it. The public learned the value of pre-prints during the COVID-19 pandemic as these publications (yes, they are publications) allowed us to rapidly and frictionlessly share our ideas, accelerating science and helping scientific systems function at a higher level. However, tenured biologists who determine what matters for tenure in biology are ironically unable to evolve to reform their institutions for the 21st century - preprints don’t count, in the eyes of the old guard, and so we need to focus our critical eyes on their absurd norms and dubious metrics for the “impact” of a scientist. During the COVID-19 pandemic, many people demanded all pre-prints have the warning label of “pre-print”, delegitimizing the idea (even if it was right) until it passed institutional gatekeepers like the editors at Nature Medicine or paradigmatically entrenched researchers like Andersen and Holmes lurking anonymously as peer-reviewers.
In other words, the scientists who fit in and rose to tenure became the world’s experts, and they used their stature to reinforce the institutions in which they fit in well, institutions that have excluded others from science careers, others who are brilliant but unpublished because their grants didn’t get funded, the editors didn’t recognize their name, an anonymous peer reviewer would be proven wrong if the paper were published, and other bottlenecks that strangle scientific progress with the defensive hands of yesterday’s scientists.
If I could plug just one hole in the ship of science, I would reform our scientific dissemination system. In fact, this is why when I was semi-retired thanks to capital gains during COVID, I felt it was worth my time to co-found the company Selva with the goal of making “a social medium for science”. TLDR, that company failed because we didn’t have a clear and immediate way to generate revenues and thus had difficulty attracting investors. Plus, we weren’t the best team for the job. Live, try, fail, and learn. While our particular effort didn’t succeed, it’s still valuable to reimagine scientific dissemination systems that can bypass the unbearable costs and geopolitical conflicts of interest in Elsevier and the entrenched paradigms of Science.
Dr. Michael Eisen is a professor at Berkeley, the founder of the Public Library of Science (PLOS), and currently editor-in-chief at eLife. Dr. Eisen has long advocated for reform of our publication system, and right now he’s promoting a new model for publications at eLife in which articles are accepted prior to seeing their criticisms, peer-reviews are published alongside articles at low-cost, and scientists can read the back-and-forth to make up their own minds. This small change to publication would be a major reformation of scientific publishing as many scientists have been locked in anonymous battle with peer-reviewers, felt passionately that the reviewers were wrong, and yet had their science rejected from wider dissemination. It’s possible eLife’s model can attract paradigm-shifting science and disseminate revolutionary findings that otherwise would have been suppressed in anonymous paradigm-blocking “peer” review. Supporting eLife would require scientists, especially established scientists, to send their best work to this journal and count publications in eLife as a legitimate scientific product. That’s a very small ask; beware the motives of scientists who can’t accommodate even this minor change in the publication process.
It’s also possible that we can co-opt other systems with large scientific audiences to develop a better scientific dissemination system. Twitter/X, for example, has many scientists and is a common platform for sharing our work. X could easily allow posts to be flagged as “science” (at the discretion of the user, possibly requiring a preprint link), and it could design algorithms for more balanced sharing, ranking, and searching of scientific findings. We don’t want science discourse determined by whoever has the most followers (whether Twitter followers or Nature readers). We could easily devise algorithms that “suppress-the-winner” by increasing the friction for further reach as an article’s or user’s reach increases, thereby providing fertile informational space for new scientists and ideas to grow and be heard.
Interesting sidebar, this “kill-the-winner” effect is a well-documented ecological force called the Janzen-Connell effect or frequency-dependent effects. Ecologists believe these effects may explain why there are so many species of trees in tropical rainforests, and evolutionary biologists note that frequency-dependent selection can maintain diversity and prevent any populations from being ‘fixated’ on any one genotype (a process called ‘fixation’). These algorithms, and so much more, were part of my vision with Selva, hence the name “Selva” which is “jungle” in Spanish. These ideas could still be implemented on one of many platforms and doing so may break the oligopoly of journals that currently severely limit the function of our scientific system.
From a legislative perspective, Congress has the ability to completely revolutionize science with the power of the purse. Consider for the purpose of imagination a relatively minor earmark prohibiting the use of federal science funds to support advertisement or dissemination of science, including but not limited to the publication of science papers. Congress could also designate a federally-run platform to which researchers who wish to publish federally-funded work must submit their research, thereby providing us an alternative route for sharing our work. A federally-run platform for scientific dissemination need not be a sprawling bureaucracy but rather it could be a simple social platform with a feed algorithm to find and present work. Under such legislation, scientists previously reliant on federal grants to pay Nature $11,700 would no longer be able to draw from taxpayers’ wallets to pay Nature, Elsevier, and RELX, and they would have an alternative platform whose feed algorithms and engagement mechanisms (e.g. a button saying “replicated!”) we could moderate to increase the competitiveness of our science.
Yes, it’s possible eLife or something like Selva or Congressional actions could disrupt a journal system that has been with us for a while. Just because something has been with us for a while doesn’t mean it serves us well. The structural relic of our peer-review system has an unfair stranglehold on scientific publication that combines with its egregious editorial & peer-review practices and the feudal structure of science funding to create massive inefficiencies in the function of science as we intend it.
While we’re thinking big, we can also reform science funding. For example, one could implement grant funding procedures that “weed out the crazy, fund the rest at random” so people can’t funnel work to their friends and we’d be 100% reassured that Dr. Fauci didn’t fund Andersen in exchange for a paper obfuscating the probable lab origin of a virus in labs Dr. Fauci funded. I have like a dozen other ideas as well, including some that could turn a massive profit, so hit me up if you want to reform science.
A common theme of my own thinking about how to improve science is that I see immense value in ensuring taxpayers support a more efficient scientific system, and the current inefficiencies in our scientific system have a common theme of a few institutions having too much power or possibly having interests that don’t align with those of the US taxpayer. Do taxpayers want to support Elsevier and its entrenched interests, or do they want a more equitable and competitive scientific dissemination system? Do taxpayers want to support the research interests of whichever old, connected, and culturally-politically conformist scientists make it to grant review panels, or do they want more equitable funding of scientific research?
State-Building of Scientific Societies
We think of “failed states” and “state-building” in the context of nation-states, but the general concepts of institutional rehabilitation and reform are desperately needed in science. Scientists have been crying for help for decades, the inefficiencies of science are well known, and we can upgrade science if we turn our state-building attention to the institutions and societies of modern science.
I suspect that, for a single year’s allocation of a small fraction of the annual science budget, we could modernize our scientific systems. An upgrade can pay dividends, increasing the return on our annual investment in science and helping people 100 years from now write glowing praise about everything science and those who fund it have done for their lives.
On the topic of vaccines, of which the scientific method has been corrupted for many decades, what do you think about the idea that many vaccines actually got credit for the accomplishments of widespread sanitation? I've known that vaccine injuries have been highly politicized, but I didn't realize until I read through the process for getting vaccines approved, that there's never been a true placebo controlled study for most if not all of them. With regard to politics in science, Ancel Keys and his political slant was responsible for uncounted millions of illnesses and deaths, thanks to his false premises on heart disease, which gave birth to this juggernaut Big Food/Big Pharma industry and the resulting obesity and chronic illness epidemic. It was bad science writ large, and he was one of those powerful influences that kept better scientists from publishing their findings, lest they have their funding withdrawn.
Ah -- leave the academic and private scientific morass the way it is and just find your (scientific) way through the mud very carefully: in one way or the other it's always been this way.