Brain the Size of a Planet, Asked to Watch the High Street

There is a particular kind of robot that every engineer secretly fears building. It is not the one with red eyes and a grudge. It is the one with a brain the size of a planet, a working knowledge of every branch of mathematics yet devised, and a job description that reads, in its entirety, "open the door when someone approaches." It does the job. It does it perfectly. And it sighs, audibly, every single time.

Marvin, the chronically depressed android of the Sirius Cybernetics Corporation, has been a running joke among technologists for nearly half a century, mostly because he is funny and partly because he is a little too accurate. We keep building extraordinarily capable machines and then pointing them at extraordinarily small problems. I was reminded of him recently while thinking about the question that tends to arrive at the end of every serious conversation like a dessert trolley: "What about AI?" There is an argument doing the rounds among geopolitical analysts that contains three ideas deserving more room than they usually get, and one of them is genuinely uncomfortable.

 

The Argument, in Three Moves

Compressed, it goes like this:

• Technology equals specialisation multiplied by globalisation. Advanced technology is the product of thousands of highly specialised suppliers spread across the planet. Break the global trade system (which, on this view, is already breaking) and you break the multiplier. In a world of war and collapsing supply chains, AI "cannot make that leap into AGI."
• So AI gets repurposed. Instead of reducing scarcity, curing disease, or fixing the climate, it becomes "a control grid": a system to monitor and surveil all of us. Our cities are already full of cameras; they are just a little bit dumb, because they don't talk to each other. With AI, they will. Crises will accelerate the trend, and companies like Palantir will use them to develop AI-controlled autonomous drone warfare.
• But surveillance is not the big risk. The big risk is that AI cheapens and dehumanises the human experience. When the most powerful people in technology tell us computers are smarter than we are, they may even be right about the mathematics and the medical diagnostics. But we are not here to out-calculate a datacentre. We are here to love, to imagine, and to be kind, and an obsession with AI distracts us from that.

 

I think the first point is more right than most people realise, the second point is more right than most people would like, and the third point is the one that should keep CIOs awake at night, though not for the reason usually given.

 

Move One: The Most Fragile Machine Ever Built

"Technology equals specialisation times globalisation" sounds like the kind of formula that gets written on a whiteboard and photographed for LinkedIn. It is also, in the specific case of AI, an almost literal description of the bill of materials.

Consider what has to go right for a single frontier AI chip to exist. The extreme ultraviolet lithography machines that print it come from exactly one company, ASML, in the Netherlands. Each machine is roughly the size of a bus, contains well over a hundred thousand parts, and ships in dozens of crates across several cargo flights. Its mirrors come from Zeiss in Germany, polished to a smoothness where, scaled up to the size of a country, the largest bump would be a fraction of a millimetre high. Its light source involves firing a laser at droplets of molten tin, fifty thousand times a second, twice per droplet, to make a plasma hotter than the surface of the Sun. This is not an exaggeration for comic effect. Reality has simply got there first.

The chips themselves are overwhelmingly fabricated by TSMC in Taiwan. The specialised photoresists come largely from a handful of Japanese chemical companies. Before 2022, around half of the world's semiconductor-grade neon was purified in Ukraine. The high-bandwidth memory comes from South Korea. The design software is American. The power to train the models comes from grids that are, themselves, built on globally sourced transformers that currently have lead times measured in years.

Now place that supply chain on a map and draw a circle around the regions that keep geopolitical analysts awake at night. Taiwan, Japan, the South China Sea, Eastern Europe. The overlap is not subtle. It is, if anything, a little rude.

Where I part company with the argument slightly is on the word "cannot." I have argued before that AGI is still a long way off regardless of geopolitics, and that the frontier is getting expensive while the rest of the field catches up. A fractured world doesn't uninvent the models we already have. It freezes them. Capability stops climbing and starts spreading sideways, into cheaper, smaller, good-enough systems that can run on the hardware you already own. Which brings us neatly, and rather ominously, to the second point.

 

Move Two: Surveillance Is the Killer App Because It Needs No Genius

Here is the part the argument gets exactly right, though it rarely spells out why. If AI stalls short of anything resembling general intelligence, its most immediately useful application to a nervous state is surveillance, precisely because surveillance doesn't require intelligence. It requires correlation.

You do not need a mind that can prove new theorems to notice that the same face appeared on camera 412 at 09:14 and camera 1,876 at 09:31. You need pattern matching, a large database, and a budget. Current AI is spectacularly good at pattern matching. It is the one party trick it never fumbles. Asking a frontier model to watch a city is like hiring Marvin to monitor a car park: the job is beneath it, it will complain, and it will not miss a single number plate.

That observation about dumb cameras is the important technical point, and it is worth unpacking. Britain, my own country of origin and a nation with a long and enthusiastic relationship with the CCTV camera, has an estimated five million or more of them. For decades they were mostly harmless in practice, not because anyone decided they should be, but because they were islands. Thousands of separate owners, incompatible formats, footage overwritten after thirty days, and nobody with the time to watch any of it. The privacy we enjoyed was not a right. It was an integration problem.

AI solves integration problems. That is, in a sense, all it does. Computer vision turns pixels into identities, a data fusion layer joins those identities to transactions, phone locations, and licence plates, and suddenly millions of dumb islands become one very attentive continent. The Metropolitan Police already deploys live facial recognition vans on London streets. China's "Sharp Eyes" programme was explicitly designed to connect public and private cameras into a single grid. Nobody needed AGI for any of this. They needed a decent model and a procurement framework.

The mention of Palantir is apt, though perhaps not for the drones. Palantir's real business, and its real significance, is exactly this fusion step: taking data that lives in a hundred separate systems and making it queryable from one screen. It is enterprise integration, which is the least glamorous job in IT and also, it turns out, the most consequential.

And it is right that crises accelerate it. Every emergency power ever granted to a government has come with a sunset clause, and almost every sunset clause has been quietly renewed at dusk. Systems built "temporarily" for a war or a pandemic tend to discover a second career once the war or the pandemic is over. In IT we have a name for this. We call it scope creep, and we write it on the risk register, and then we watch it happen anyway.

 

Move Three: The Real Dehumanisation Is Not the Robot's Fault

Now for the part of the argument that is usually framed as spiritual and that I want to treat as an operational risk, because I suspect it is both.

The big danger, the argument goes, is that AI cheapens the human experience, and that people like Musk, Thiel, and Altman encourage this by telling us the machines are smarter than us. I agree with the conclusion but I think the mechanism is more mundane, and more damning. Machines don't dehumanise us by being clever. We dehumanise ourselves by deciding that the clever bits were the only bits that mattered.

The Sirius Cybernetics Corporation, you may recall, had a product line called Genuine People Personalities. The idea was that doors, lifts, and kitchen appliances would be fitted with cheerful, simulated emotions, so that every door would be thrilled to open for you and would tell you so. The joke was never that machines might have feelings. The joke was that a corporation would decide feelings were a feature, manufacture them at scale, and fit them to the doors.

We are now, quite seriously, doing this. We have models that write birthday messages, condolence notes, wedding speeches, and the first draft of an apology. We have AI companions marketed to the lonely, and AI "empathy layers" bolted on to customer service queues so that the system can sound deeply sorry while declining your refund. Each of these is individually defensible. Collectively they amount to outsourcing the precise list of things we are supposedly here for: love, imagination, and kindness. The machine is not taking those from us. We are handing them over, because they are slow, and the machine is fast, and fast is the only thing the dashboard can measure.

I wrote recently about the beige wall of AI productivity: the strange flattening that happens when everybody uses the same model to produce the same competent, frictionless, forgettable output. That is the organisational version of the same warning. It is what dehumanisation looks like when it turns up in a quarterly report rather than a sermon. Nobody is harmed. Everything is fine. And somewhere in the building, the one person who used to write the oddly memorable line in the proposal, the one that actually won the contract, has stopped bothering.

 

So What Do We Actually Do?

For those of us who run technology inside organisations, there is a more practical list, and it maps neatly onto the three moves.

• Treat the supply chain as a strategic risk, not a procurement detail. If your AI strategy depends on a single hyperscaler, a single model vendor, and an uninterrupted flow of chips through the Taiwan Strait, you don't have a strategy. You have a hope with a budget line. Keep capability you can run on hardware you control, and know which of your systems would still work if the frontier froze tomorrow.
• Be very careful what you integrate. Every time you join two datasets "because we can," you are building a small piece of the grid this argument is worried about. Data minimisation is not bureaucratic fussiness. It is the only engineering control that actually works against function creep, because data that was never collected cannot be repurposed later by someone with different intentions.
• Decide, on purpose, which human tasks are not for automating. Not because the AI can't do them, but because the doing is the point. A manager's handwritten note to a grieving colleague is not an inefficiency. If it is ever on a list of things to optimise, the list is wrong.
• Measure the things the dashboard can't see. Customer trust, staff loyalty, the quality of an original idea. If your AI programme is only evaluated on hours saved, it will relentlessly save hours from exactly the activities that made the organisation worth working for.

 

The Answer Is Not the Problem

There is a famous story about a supercomputer that spent seven and a half million years calculating the answer to life, the universe, and everything, and arrived at a two-digit number. The answer was perfectly correct. The problem was that nobody had properly understood the question.

That is where we are with AI. The technology is very good at answers. It will become a frighteningly efficient surveillance tool not because anyone has thought hard about whether that is wise, but because it is an answer that happens to fit a great many governments' questions. And it will flatten our working lives into competent beige not because it wants to, but because we never asked what the humans in the system were actually for.

A period of crisis may force that question on us, whether we like it or not. I would rather we asked it voluntarily, while the supply chains are still running and the cameras are still a little bit dumb. Somewhere, a very clever robot is staring at a door, waiting to be asked something worthy of it. The least we can do is not join him.

 

And if the anthropomorphisation of AI concerns you, if you find yourself lying awake wondering whether the machine really understands you, really cares, or is quietly miserable about the doors, just ask yourself one question.

Can a submarine swim?

All Posts