This working note can be experienced in a more interactive fashion here, please give it a go.
The lights are blinding. The billions now going into AI drug discovery can be repaid only by more drugs through Phase 3, and the field is betting that an AI model trained on cell-based experimental data can tell which drugs will work in people. Previous attempts often hit the wall where drugs die. Every time the wall has moved, someone had measured people. The money that moves it next will be the money that observes patients sooner and more often.
Seven in ten drugs that start Phase 2 never pass it, and four in ten that reach Phase 3 fail there.1 Both are the same event: the first time a new drug meets a large, diverse population of sick people, over months, on the outcome it will be sold on. This note calls it the numberwall, or the wall for short: the point where complexity prevents forecasting. Before it, what a drug does follows chemistry and physiology that repeat from person to person, and models predict it well. At the wall, everything that differs between people starts to matter at once: their genes, their histories, the numbers their bodies defend, the other illnesses they carry.
The money now pointed at that wall is enormous. Isomorphic Labs raised $2.1 billion in May.2 Xaira launched with more than $1 billion, and Lila Sciences has raised $550 million to build autonomous labs.3 Biohub’s virtual-cell program carries about $1.3 billion in new commitments.4 Some of these programs describe themselves as basic science, and they may be very good science. Let’s set aside the pursuit of the answer to life, the universe, and everything, though I’m rooting for them to find it. Only new medicines can repay sums of this size, and new medicines come only from getting through the wall more often. So I judge the bet by Phase 3 and by how they intend to apply AI.
The bet is that a model trained on cells, far from patients, can tell which drugs will work in people. On the record so far it cannot. The wall behaves like a forecasting horizon, and weather, where AI has had one of its cleanest wins, shows what a much better model buys against a horizon: about half a day. My quarrel is with that one bet. Chemistry engines aimed at targets that human genetics has already validated are a different case: there the compass came from people. Andreas Bender, Jack Scannell, David Shaywitz and colleagues reached a similar place in Nature Reviews Drug Discovery this year, finding the evidence of AI’s clinical impact “disappointingly limited” and putting predictive validity, whether a measurement tells you anything useful about the outcome you care about, ahead of data volume.5 Along the way we ran our own tests, and pre-registered each one: we wrote down its method and pass mark, and froze the file, before looking at the data. The Methods section at the end reports every one, including those that did not help.