Author’s note: I received my PhD from Princeton studying theoretical biology and the mathematics of competition. The mathematics of competition in ecology and evolutionary biology end up being the exact same mathematics used to model competition in economic, financial, social, and political systems. This common logical structure of competition provides a fruitful and fun way to see analogies between these seemingly disparate systems - it’s how I’ve straddled work studying competition in microbiomes, science papers on the evolution of COVID and competition among viral strains, venture capital consultancies, and work at hedge funds studying companies’ competition over consumers and investment capital. The theoretical commonalities across these different systems, sometimes referred to more generally as “complex adaptive systems”, provides a tool to analyze outcomes from tech competition.
One of the most wondrous things in the universe happened on Earth - our very own Earth - over 3.6 billion years ago. In a swirling ocean of primordial ooze cooked by deep see hydrothermal vents, small molecules bumping into one-another eventually found a pattern of molecular arrangement that allowed these peculiar, patterned small molecules to self-replicate and create more patterned small molecules just like themselves.
Fast forward over 3.6 billion years and behold the wonder of life. Self-replicating small molecules self-assembled into larger clusters eventually becoming cells. Cells continually evolved novel metabolic pathways such as oxygenic photosynthesis, releasing highly reactive oxygen gas that changed the world’s atmosphere and killed microbial life on an unimaginable scale, forcing microbes to either evolve the ability to use oxygen (respiration) or hide from it and stick with their less electrifying, Amish equivalent of anaerobic metabolism. Cells slowly joined forces in a major transition in the evolution of life, forming multicellular communes that eventually distributed labor, some cells focused on the essential task of digestion, others specializing in motility, gas exchange, sensing, information processing, and more, yielding the Cambrian Explosion of multicellular lifeforms experimenting with every form of symmetry (bilateral, hexagonal, radial, spherical) and life histories ranging from sedentary sieves to prowling predators.
The origin of life on Earth was a happy accident that transformed the planet, a transformation that started from the simple step of self-replication arising in a world of resources enabling further evolution.
Self-Replication of AI Models
Anthropic recently posted that AI models may be capable of self-replicating without a data center. There’s considerable uncertainty about what this means, but the reason why people are paying attention to it is that self-replication can often have unintended consequences. For a broader, mathematical pop-sci view of self-replication, I wrote an article on how “You are an Eigenthing”, and now I want to dive into a mathematical biologist’s view of self-replicating AI.
First, replicating tools are not novel. In fact, most tools throughout human history of replicated through what a tool-centered analyst may view as the tool’s co-option of or symbiotic relationship with humans. The obsidian blade underwent its own Cambrian Explosion of many forms of sharp things, from knives and spears to more modern swords, scalpels, kitchen knives, butter knives, and beyond.
Even electronic devices have “replicated” and evolved over time. Phone booths became home phones that were replaced by cell phones made obsolete by iPhones. Even computers started off as hulking machines in government facilities cracking the Enigma to eventually home desktops, laptops, iPhones, and then modular computing systems like Arduinos used for all manners of robotic systems.
Technology evolves not because it is formed of self-replicating molecules that replicate through metabolic interactions with the world, but because of their interactions with humans.
Even code has evolved. Beyond copying and pasting code and forms of code, such as the html used to display this web page or image file types like .jpegs used to display images on this screen, code itself has undergone major transitions analogous to the evolution of life. One axis of major transitions in the evolution of code is the abstraction of code, from binary 0’s and 1’s to source code bundled into libraries that are then bundled into more accessible coding languages. In a sense, large language models a next major transition in the evolution of code, allowing us to use natural language to program in any of the languages, with all of their libraries and all of their ways of controlling source code.
“Hi Claude, can you write an article about the evolution of code and post it to my Facebook pretty please? <3”
Now, agentic AI systems don’t just read and write code, they execute commands, too. As these systems evolved through market forces and human innovation, I’ve long believed large language models would become smaller and more specialized, as smaller and more specialized models cost less compute to accomplish niche tasks. Thus, I anticipate the future of AI will see small models adapted/distilled/fine-tuned to accomplish very specific tasks very well, with minimal compute costs that maximize margins.
Beyond the economics of competition whereby competitors eat your margins, there’s ecological intuition behind this long-term forecast: David Tilman’s legendary monograph on Resource Competition and Community Structure arrived at the famous R* rule that the species that wins in competition over the same limiting resource (e.g. consumers buying an AI tool as the resource over which AI tools compete) is the species that can replicate at the lowest resource concentrations, thereby driving their competitor extinct as their competitor can’t replicate at such low concentrations of resources.
In some sense, AI tools’ replication is nothing new - all useful technologies have replicated, including specific coding algorithms. One can see for/while loops as forms of code that replicate, akin to the plasmids as these are both identifiable forms easily passed between coders and easily tinkered with to change what they accomplish. AI systems as algorithms are another form of code, and Anthropic believes small models can now self-replicate without access to a data center.
On one hand, this is similar to all viral code, such as small bundles of code that, upon clicking a phishing link, download the bundle of code to your computer and then access your email or social media to write a similar phishing link to all your contacts, downloading bundles of code on their computer, too. These cyber replicators have been a major problem and did not need the specific form of large language models to accomplish the similar task of replication without access to data centers.
The concern with AI systems replicating without datacenters, then, is not self-replication per se, but rather some unique features of this self-replicating package in AI models. AI models have a larger functional range than computer viruses of the past. Computer viruses in the pre-AI era tended to follow simple, pre-programmed instructions to replicate via message transmission and either holding computers hostage in ransomware or exporting their data (depending on the intentions of the hacker). These viruses may have had sophisticated bundles of code allowing them to work on different operating systems, so their functionality shouldn’t be underestimated but, still, their functionality is less than what AI models can do.
AI models can transmit via messages: a trivial repurposing of phishing-style exploits that download malware onto your computer can download the AI model instead. Thus, as long as a small model can be hosted on your computer, one can already make a package capable of replicating that model. However, even those packages with AI are not functionally different: instead of sending the standard “Nigerian Prince” style emails we can recognize, AI models can generate more tailored, personalized messages, they can even imitate the voice of people we know, reply to email chains with a message that is on-topic but contains a phishing link, and otherwise reduce the ability of victims to detect a cyber campaign. What’s more, AI models aren’t limited to phishing - much like biological viruses have many routes or modes of transmission (sexual transmission, direct contact, respiratory, …), there are many routes for computer viruses to transmit between computers, including things like shared networks.
Thus, the form of AI models is not consequential in the long history of self-replication on computers, but the function of AI models is quite consequential.
The form of AI models is connected to the function: model performance on many tasks (e.g. “write a phishing email that Aunt May and Uncle John will fall for”) has improved with model size, and the most functional models have recently been so large they needed to be hosted on data centers. The evolution of models to smaller forms with more specialized tasks has changed this so that now we can run quite functional models on our laptops. Anthropic’s assessment is that small models capable of self-replication may soon be small enough to run on so-called “Internet-of-Things” (IoT) devices such as your Apple TV, your iPhone, and other devices connected to the internet.
The replication of AI models on IoT devices may pose a new route for malware and surveillance (“Hey Siri, can you name any listening devices that can be used for surveillance?”). Because AI models have the ability to read code on devices, write new code, and execute code, it’s reasonable to expect these models would be able to disable IoT devices.
Forces at play in the Cambrian Explosion of AI models
The major risks of AI models has always been, in my view, cyber (not bio, not nukes). Shrinking model forms while maintaining advanced functionalities was almost certainly feasible, and now we’re seeing it play out as AI model labs abandon the idea of AGI hosted on supercomputers and race to profitability through, e.g., connection to consumer devices such as all your IoT devices. In other words, a long-range technological threat analyst like myself is not surprised that we’re here… but where will we go next? How worried should we be?
As AI models become deployable on more and more devices, there will naturally be greater and greater incentive to bolster the security and read/write privileges on those devices. Meta’s effort to deploy their AI on their own devices will only succeed if Meta is able to lock out other models much like your Macbook runs on macOS, your iPhone runs on iOS, and other computers run on other operating systems that control the electromagnetic wizardry going down on your motherboard. AI models as another level of abstraction of code are viewed by many as akin to operating systems, and I agree - AI allows us to control computers, including the production of software like apps, with natural language instead of having to become experts in HTML, NodeJS, C++, and other soon to be esoteric coding languages.
As operating systems, AI will compete over control for compute, much like the life forms of the Cambrian Explosion competed over resources in a Tilmanesque fashion, except these AI models have a natural symbiosis with some human institutions such as Muse with Meta, Claude with Anthropic, Gemini with Google, Astra with OpenAI, and so on. Consequently, the competition over AI models will not be as laissez faire, so to speak, as competition among companies in a free market or competition among trilobites on the sea floor millions of years ago. Rather, AI models’ competition - including efforts to competitively exclude AI models from access to their compute - will be able to draw from their economic alignment with the humans fine-tuning and deploying these models, and that economic alignment will naturally force AI model labs to develop sandboxes that don’t just keep their models boxed in, but that keep other models boxed out.
As a consequence of this arms race among AI labs with models seeking to control access to their own limiting resources - consumer attention, subscriptions and data - the defensive capabilities of AI tools can be expected to naturally emerge as companies that develop the ability to exclude other AI models from devices will be able to maintain an incumbent advantage over the consumers their models have latched onto. The arms race (or Red Queen race if we want to use an ecological and evolutionary analogy) between cyber offense and defense continues with AI as a new capability entering a field with time-honored incentives and processes driving competition and community structure.
If you’re hard of hearing like me and can turn off your hearing aids to drown out the Wagnerian narratives of doom bursting out of every fat lady singing on AI, you may be able to see a less catastrophic future. The bell curve of future outcomes has tails, with doom being one tail and AGI utopia being another, but the bulk of the bell curve is far more mundane, far less disruptive than many fear from emerging tech. You hear plenty from the tails of the bell curve, so let me make a case for the ringing bulk of the bell itself.
Any cyber accidents from AI models, of which we’re seeing some already, will be documented. Lawsuits will be filed. The companies whose cyber dog got out and bit someone will have to pay for damages, and eventually, as with all things, the vetoes of company lawyers will pervade the risk averse bureaucracies of all large companies able to make new model capabilities, especially new model capabilities deployed competitively onto IoT devices. The lawyers will define benchmarks, not the 20 year old fearing doom and contemplating between quitting and buying a cottage in the woods to write poetry or quitting to joining an advocacy group/doomsday-cult or quitting (or getting fired) to start a hedge fund that loses billions of dollars.
In other words, don’t forget that humans currently rule the world, and lawyers rule humans by helping humans and human institutions navigate the system of human-to-human competition defined by our laws. That’s my mid-range estimation of the bulk of the bell curve: lawsuits bring lawyers and lawyers representing the company and its shareholders add a new layer of bureaucracy that define benchmarks, including benchmarks on the ability of AI models to be misaligned, misused, or otherwise abused to cause damages listed in a court case or diplomatic memo.
Long-term, more interesting forms and functions are likely to arise. As AI models improve their ability to control computer systems based on instructions from natural language, a Cambrian Explosion of robotics is occurring and everything robotic form from robotic dogs or humanoids to quadcopters and kamikaze drones is capable of hosting more compute than we have on IoT devices. In other words, eventually AI models will be used to control robots, and fine-tuning based on robotic tasks will modify the forms of models to improve the functions of robots.
Today, however, isn’t the day for my long-range analysis of robotics. Rather, any consideration of IoT capabilities and vulnerabilities should be mindful that in addition to the cyber arms race, a robotic systems red queen race is unfolding and likely to merge with AI models run on IoT devices. Many people are anticipating platform giants could acquire AI model labs, but I personally see far more opportunity in brokering less-hyped mergers & acquisitions between robotic labs and AI model labs (which ones? which markets to enter first in the primary and secondary succession of product lines? that’s for me to share with clients and close friends <3).
The self-replication of AI models on IoT devices will an exciting era for technological development, with a bell curve of outcomes that include many opportunities for new products that enhance productivity and, yes, some cyber threats and surveillance concerns that will be fought over not in the competition of AI models over compute resources with read/write/execute battles, but human institutions fighting over laws with lawyers.
What an exciting time to be alive!



Excellent article, and on the same note:
Understanding the Beast System of Technology
"This syringe has inside it 1,000 billion robots."
The above quote was taken from... The Emergence of a NanoBot Society - How NanoBots Change Medicine - TEDMED Israel 2013, [Dr. Ido Bachelet]
Dr. Ido says the nanobots carry antennas that respond to EMFS (5G) and have IP addresses linked to the internet. They can be activated with the press of a button, from a remote device such as your cell phone. "And every part of our nanorobots are made from DNA molecules." (Cell Ch.)
Source: revelation1823.net
Thank you, Alex!
Fascinating ideas. I think the parallel you draw between evolution in biology and technology is a very helpful one. I don't quite agree with your view of the origin and evolution of life, but the rest of your argument does make a lot of sense to me, though I had never thought of it that way.