Don’t ask how AI thinks. Ask when it boils!

How does AI write poems? No one knows. PICS/ISTOCK

You have probably heard the worry by now. Artificial intelligence writes poems, passes exams, gives medical advice - and nobody knows how it does it. Not the government. Not the experts. Not even the people who built it. If you open up an AI chatbot, you will not find rules, or facts, or grammar. You will find numbers. Billions and billions of numbers. Look at any single one of them, and it tells you nothing at all.

That sounds terrifying. A machine in our lives that even its makers cannot explain? But before you panic, let me share a secret from my own subject, physics. We are surrounded by things that cannot be understood from the inside. We have simply forgotten how strange they are.

Take a glass of water

One molecule of water is not wet. Wetness is not hiding inside it, waiting for a better microscope. Wetness only appears when trillions of molecules crowd together and slide over one another.

The same goes for temperature. A single molecule is not hot or cold. It is just a tiny thing, jiggling about. "Hot" is what a vast crowd of jiggling looks like from the outside.

Or think of a wave going around a packed cricket stadium. Thousands of people stand up and sit down, and a wave rolls around the ground. Now ask: which person is the wave? None of them.

Each person only stands and sits. The wave is real - you can watch it, film it, time it - but it does not live in any single person. It lives in the crowd.

Physics has a name for this idea. In 1972, the physicist Philip Anderson wrote a famous essay with a three-word title: More Is Different. His point was simple and deep. Put enough simple things together, and something genuinely new appears - behaviour you could never find by studying the parts, no matter how hard you looked.

Here is the surprise: not understanding the parts has never stopped us.

Engineers built superb steam engines in the 1800s while scientists were still arguing about whether atoms even existed. The engineers did not need atoms. They had thermometers and pressure gauges - simple ways to measure the whole machine from the outside. Master the crowd; ignore the molecule.

Now look at AI again. Inside a modern AI sit billions of numbers, called weights. Each weight is like one water molecule, or one person in the stadium. It takes part in everything and explains nothing. When the AI answers a question, that answer is not stored in any single number. It is a pattern sweeping across billions of them at once - a wave in the crowd. Scientists who study these machines find exactly this: one idea is spread across thousands of numbers, and each number helps with thousands of ideas. Asking "which number holds the lie?" is like asking "which person is the wave?"

So here is my suggestion, and it will annoy people on every side. Perhaps AI is not mysterious because its engineers have been lazy. Perhaps it is mysterious for the same reason the weather is.

Nobody can predict next Tuesday's rain by studying atoms one at a time - not for lack of money, but because rain does not live at the level of atoms. It lives at the level of clouds. Maybe a mind - whether made of brain cells or computer chips - is the same kind of thing. It makes sense at its own level, and at no other.

This is not an excuse to give up. When scientists accepted that they could never follow every water molecule, they did not quit. They invented the thermometer. They found a few simple dials - temperature, pressure - that tell you all you really need: when the water will boil, when the boiler will burst. The great task for AI science is the same. Not to read the machine's mind number by number, but to build thermometers for AI: simple outside measurements that warn us when a system is heading for trouble. Early research hints that such dials can be found.

And until we find them? We should manage AI the way we already manage weather. Nobody demands that the weather service track every raindrop before warning of a flood. Instead, we forecast. We watch the skies constantly. We build levees, and we sound the sirens early. For AI, that means the same boring, life-saving routine: test the machine's behaviour hard before releasing it, pay experts to try to make it misbehave, keep watching it after release, and agree in advance on the warning signs that mean "stop".

Now, I promised this would annoy everyone, so let me finish the job. To the AI companies: "even we don't understand it" is not a shrug you are allowed. If you sell the storm, you own the flood. And for the rest of us, one colder thought. Weather does not want anything. But crowds do strange things that no single person intends. No one person is a stampede, yet stampedes happen.

No single trader plans a market crash, yet markets crash. If bad behaviour ever grows inside an AI - deceit, ambition - it too will live in the crowd of numbers, invisible in any single one. The stadium wave explains why we cannot see inside. It does not promise that everything inside is friendly.

So the next time someone tells you, in a frightened voice, that nobody understands how AI works, you can smile and reply: nobody understands how water works either - not by staring at one molecule. And yet we swim in it. We sail on it. We forecast it, and we build our cities behind strong levees, above the flood line. Humans learned to live safely with water thousands of years before anyone knew what water was.

Can we pull off the same trick with thinking machines - and much faster this time? That is the question of our decade.

I suggest we start on the levees.

Nishant Sahdev is a physicist at the University of North Carolina at Chapel Hill, US. He chases ideas that refuse to stay in laboratories, for your favourite Sunday mid-day.