Something interesting is happening at the early end of the AI market. Spend an hour reading the newest cohorts of funded startups: the closest thing we have to a leading indicator of where conviction money believes the next decade lives, and the theme is hard to miss. The wave has gone physical. Drones. Robots. Machine tools. Wearable sensors. Instrument streams. Simulated hardware, simulated physics, simulated users. The purely textual AI company is quietly disappearing from the front of the funnel.
I want to be fair to this turn before I criticise it, because the premise underneath it is correct, and it's the most important thing the industry has admitted in years.
The Premise Is Right: Text Is Exhausted
The crawlable internet has been consumed. This isn't a fringe claim any more; Epoch AI's projections, presented at ICML 2024, estimate that frontier training runs will exhaust the effective stock of public human-generated text somewhere between 2026 and 2032, and the frontier labs' behaviour suggests they priced this in some time ago. The scaling curves that were sold as a road to general intelligence have flattened into diminishing returns.
Meanwhile, almost everything that actually happens on Earth is never written down. Every machine cycle, every flight, every cut, every grip, every pressure wave: it passes through instruments as signal, and then it's thrown away. The physical world is the largest untapped corpus in existence.
So far, so good. I've been making a version of this argument for years, and it's genuinely encouraging to watch the market arrive at it.
But look at what the industry does with the premise. Almost without exception, the new physical-world companies run the same playbook:
- Instrument the physical process: cameras, wearables, sensors, telemetry taps.
- Capture the signal and ship it away: to data centres, labelling pipelines, training clusters.
- Train a large model on the accumulated corpus: a policy, a surrogate, a simulator, a "foundation model for X".
- Deploy the frozen result back toward the world it was distilled from.
In other words: the correlational architecture that just hit the wall on text, pointed at atoms. Same mathematics, heavier data, higher stakes. The premise changed. The bet didn't.
I think it's the wrong bet, and I want to lay out why in architectural terms. Not because the people making it are foolish: they aren't; some of the sharpest technical founders I've read this year are in this wave, but because the failure mode is structural, it's knowable in advance, and the mathematics that says so has been in the literature for decades.
The Industry's Own Best Insight Refutes Its Own Bet
Here's my favourite detail from this wave, because it's an argument I could have written myself.
Some of the new data-capture work has noticed something profound: a video of a hand picking up an egg and a video of a hand picking up a rock are nearly identical in pixel space, and nearly identical in joint angles. They differ almost entirely in a quantity nobody has been recording: force. A hand squeezing hard and a hand at rest can occupy the identical pose. This isn't just a founder's pitch line; it's now an active research finding. Recent robot-learning work has shown that grip force cannot be recovered from vision or kinematics downstream, and must be sensed at the source (from the muscle signal itself) a channel that Meta's neuromotor-interface research in Nature demonstrated can be read non-invasively at the wrist.
It's worth sitting with what's actually being said there. The observable stream is not the state. Two situations that look identical to any camera, at any resolution, with any amount of compute applied afterwards, are causally different, and the difference is precisely the thing that determines what happens next. You cannot recover the cause from the correlate, no matter how large the model you train on the correlates. Judea Pearl spent an entire research programme formalising why not.
And this is not a hardware insight. It is the founding insight of a whole branch of physics and mathematics (computational mechanics) formalised by James Crutchfield and Karl Young in a 1989 Physical Review Letters paper and developed rigorously by Shalizi and Crutchfield in 2001. The state of a process is not what you observe. It is the minimal sufficient statistic of the observation history (the causal state) the thing that screens off the past and determines the distribution over futures. Crutchfield's 2012 Nature Physics overview is the accessible way in, if you want one.
So the industry is rediscovering the founding theorem of computational mechanics, one sensor at a time. And then it does something strange with it: captures the newly-sensed channel, ships it to a data centre, and trains a correlational model on it anyway. The insight (observables are confounded, causes must be recovered) gets applied to the sensor and abandoned at the model.
If the observable is not the state at the sensor, it is not the state in the training set either.
Why the Bet Fails: Three Structural Problems
1. The economics of watching
Physical processes don't emit tokens on demand. They emit signal continuously: hundreds to thousands of events per second, per machine, forever. An architecture that prices every inference as a forward pass through billions of parameters cannot economically sit on that stream. And it doesn't matter whether the model runs in the cloud or on an embedded board: the cost is a property of the architecture, not the deployment location.
So the industry's models get used before the process runs (simulation, planning) or after it has run (training on captured data, batch analysis): because per-event, at stream rate, the arithmetic simply doesn't price out. The live process, the machine actually running right now, is left unwatched by the very systems built in its name.
2. The state-tracking ceiling
This one is a theorem, not an opinion. The transformer is a fixed-depth circuit: however large it grows, its computational depth per step is constant. Merrill and Sabharwal's work in TACL places log-precision transformers inside the complexity class TC₀: and by Barrington's classic result, tracking the state of even simple systems (composing permutations of five elements, the algebraic skeleton of "follow what this machine is doing") is complete for a strictly harder class, under standard complexity assumptions. Nor do the newer state-space architectures escape: "The Illusion of State in State-Space Models" (ICML 2024) shows the same ceiling applies to them, their recurrent framing notwithstanding. And when these architectures appear to learn state tracking, they learn shortcuts: approximations that hold on the training distribution and break off it.
Read those results against the physical-world bet and the conclusion is uncomfortable. Following a physical process (carrying and updating its state indefinitely, at the rate it evolves) is exactly the problem these architectures provably sit below. It's not a data problem. It's not a scaling problem. It isn't fixed by capturing better signal. A system that cannot, in principle, track state at stream rate can't be the layer that knows what a machine is: only, at best, a layer that predicts what similar machines tended to do in the training distribution. And physical operations live and die on the difference: the failures that matter (the drift, the regime change, the precursor) are precisely the departures from the training distribution.
3. Data gravity
Physical signal is heavy, continuous, private, and frequently safety-critical or sovereign. The capture-and-centralise playbook asks factories, fleets, hospitals, and defence operators to export the most sensitive telemetry they possess to third-party training clusters: at exactly the moment the market has begun demanding the opposite. Look at what actually sells in enterprise AI right now: on-premise, own-your-stack, nothing-leaves-the-building. A physical-world architecture that structurally requires the data to leave is swimming against its own customers.
Notice what all three problems share: they're invariant to model size, data volume, and capital. They don't soften as the bet gets bigger. They are the wall, relocated.
The Tell: Everything Happens Before or After. Never During.
Look at the shape of the new wave through this lens and a pattern jumps out. The simulation companies predict outcomes before reality: the design before it's built, the user before the feature ships, the physics before the prototype. The data companies capture reality to train models after the fact. The policy companies deploy behaviour distilled from the past. Every position on the board is before or after.
Nobody is during. Nobody sits on the live stream of a specific machine, while it runs, and maintains a current, causal account of what state it's in: on the machine, at the rate the machine speaks, without a data centre in the loop. That absence isn't an oversight by any individual company. It's the architecture's silhouette. The correlational stack can't occupy that position, for the three reasons above, so the entire industry routes around it.
And here's the irony that should bother the people writing the cheques: every success of the before-and-after bet enlarges the hole. Every autonomous machine deployed, every policy shipped, every simulated design built is a new live physical process that then needs to be tracked, understood, and caught when it departs from expectation: during operation, where the before-and-after stack cannot follow.
The Bet I'd Make Instead
The alternative isn't a secret and it isn't new. It's the road the field declined to take when compute got cheap: recover the causal states of the process from its own stream, at the source.
The mathematics is proven and decades old. Causal states are provably the unique minimal sufficient statistic of a process's history. Shalizi and Crutchfield's optimality theorems show that no alternative representation predicts better, and none as small predicts as well. The construction that computes them, the ε-machine, updates its state incrementally as each observation arrives. Take that seriously as an engineering foundation and the consequences are exactly the inverse of the three failure modes above. State is maintained per-event at constant, small cost: watching becomes affordable, on-device, at stream rate. The representation is the state of this machine, learned from this machine's history, so departure from expectation is first-class, not an out-of-distribution embarrassment. And because learning happens at the source, nothing needs to leave the building. Sovereignty becomes a property of the architecture instead of a compliance document.
This is the layer we build at Senua AI, and I'll be candid about scope. It doesn't generate poetry. It doesn't replace the planning and simulation layers the new wave is building: it's the layer beneath them, the one their own success makes necessary. They decide what the machine should do. Something still has to know what the machine is.
The industry got the premise right: the physical world is the substrate, and signal must be captured at its source, because causes can't be reconstructed from correlates downstream. What it needs now is the courage of that insight applied one level deeper: to the model, not just the sensor. The world is not a corpus to be harvested and imitated. It's a running process to be understood: causally, at the source, while it runs.
That's the bet we've made. The next few years will referee it.
References and Further Reading
The Mathematics: Computational Mechanics and Causal States
- Inferring Statistical Complexity. Crutchfield & Young, Physical Review Letters 63, 105 (1989). The founding paper: causal states and ε-machines.
- Computational Mechanics: Pattern and Prediction, Structure and Simplicity. Shalizi & Crutchfield, Journal of Statistical Physics 104 (2001). The optimality theorems: causal states as the unique minimal sufficient statistic.
- Between Order and Chaos. Crutchfield, Nature Physics 8 (2012). The accessible overview.
- Causality: Models, Reasoning, and Inference. Pearl (2nd ed., 2009). Why causes cannot be recovered from correlates alone.
The Ceiling: What Fixed-Depth Architectures Cannot Track
- The Parallelism Tradeoff: Limitations of Log-Precision Transformers. Merrill & Sabharwal, TACL (2023). Transformers sit inside TC₀.
- Bounded-Width Polynomial-Size Branching Programs. Barrington, JCSS (1989). Why state tracking is complete for a strictly harder class.
- The Illusion of State in State-Space Models. Merrill, Petty & Sabharwal, ICML 2024. The same ceiling holds for SSMs.
- Transformers Learn Shortcuts to Automata. Liu et al., ICLR 2023. Apparent state tracking is distribution-bound approximation.
- What Formal Languages Can Transformers Express? A Survey. Strobl et al., TACL (2024). The field's map of these results.
The Premise: Data Exhaustion and the Missing Channels
- Will We Run Out of Data?. Villalobos et al. (Epoch AI), ICML 2024. The projected exhaustion of public human-generated text.
- Force-Annotated Demonstrations from Wearable sEMG: recent robot-learning work showing grip force cannot be reconstructed from vision or kinematics downstream.
- A Generic Non-Invasive Neuromotor Interface: Nature (2025). Muscle signal read non-invasively at the wrist, at scale.
This piece continues an argument from "The Fallacy of Agentic AI" and "The Orchestration Fallacy": that the industry keeps building around the same architectural gap instead of closing it. At Senua AI I work on the closing side: causal-state cognition that learns the states of physical processes from their own telemetry, on-device, at stream rate.