From HAL to Jarvis, part 5 of 7. A history of artificial intelligence, told as a story, for people who will use it. It began as a talk.

We taught it language

In 2017 a Google paper appears with a Beatles title: Attention Is All You Need. It describes a new arrangement of the dials, the transformer, that can read a whole page at once instead of a word at a time.

Around 2020 someone points that machine at all the text ever written and asks it one question, a trillion times: what is the next word? Something unexpected happens. It starts to reason. It also, in Caliban’s word, learns to curse: every bias, every falsehood, every bad habit in the language comes in with the language.

On the thirtieth of November 2022, ChatGPT is released, and reportedly a hundred million people use it within two months. Turing’s test was a conversation. It quietly stops being interesting.

Ray Kurzweil bet Mitch Kapor twenty thousand dollars in 2002 that a machine would pass the Turing test by 2029. By his own count, eighty-six percent of the predictions his 1999 book made for 2009 came true; by an independent count, about forty-two. He was right about the compute curve and wrong about the dates, and he was mocked for thirty years. Grade him yourselves.

What a language model is

Before we go on, let me say what that machine is, because you will use one for the rest of your career.

It is the dial machine, trained on text, and it does one thing: given the words so far, it produces the next one. Then the next. Everything else you have seen it do is built from that.

The picture I want you to keep is this. It has read. It does not look up.

Imagine a colleague who read an entire library, some years ago, and was then moved to a room with no books in it. Ask him anything and he answers from recollection, fluently, in full sentences. Most of the time he is right. Sometimes he is wrong, and he is exactly as fluent when he is wrong, because he is not consulting anything. He is remembering.

Now put a document on the table in front of him. He reads it properly, and he answers from the page.

So the machine has two memories. What it learned in training: enormous, frozen on a date, recalled and never consulted. And what is on the table: the page you gave it, the question you asked, the conversation so far. Small, exact, and gone when you close the window.

The word you will hear for its mistakes is hallucination. It is a poor word. The machine is not seeing things. It is remembering badly, and it cannot tell.

Which gives you the first rule of using one, and it costs nothing. Put the source on the table.

How one is built

Three stages, and you now know enough to follow all three.

First, it reads. A large share of everything ever written, the next-word game played trillions of times, for months, in a data centre full of specialised chips. This is the expensive part. A single run is counted in hundreds of millions of dollars, and only a handful of organisations on Earth can pay for one.

Second, it is taught manners. What comes out of the first stage has read everything and has no idea how to behave. So people grade its answers, thousands and thousands of them: this one helpful, this one false, this one it should have declined. The same procedure as before passes the blame back through the dials. Its makers also write down, in plain sentences, how they want it to behave. Yes. Three sentences again, and rather more than three.

Third, it is given tools and released. A search engine, a calculator, the ability to open a file. That is when the colleague in the empty room gets a door.

And then it is frozen. Its knowledge has a date, and after that date it knows what you put on the table.

One footnote. A company can take a finished model and train it a little further on its own domain. The same dials, smaller nudges. That is how a general machine learns a trade.

What it is good for

Four verbs.

It reads. A hundred pages in a minute, and it will tell you what is in them, find the clause, and compare two versions.

It writes. A first draft of anything, in any register, in any language you work in. A first draft. I will come back to that word.

It transforms. Translate, summarise, restructure. An email thread into a table. A table into a paragraph. A description into working software.

And it acts, which is the new one.

Where it is weak follows from what it is, and it is not where you would guess. It is not mathematics: these machines now help mathematicians with open problems. It is memory. Facts, unless the source is on the table. Anything that happened after its date. And knowing when it does not know. It can help prove a theorem and still misquote a clause it read years ago.

So trust it most where you can check it. A machine that reads the page in front of it and shows you where it found the answer is a different animal from one reciting from memory.

The conversation became a colleague

Between 2024 and 2026, the machine stopped answering and started doing.

Here is what that looks like, with an example I have invented so that it names no product. You say: “Go through last quarter’s claims files, find the ones where the reserve moved twice, and draft me a one-page note on why.” It opens the files. It reads them. It runs the numbers. It checks its own work. It writes the note, and comes back four hours later.

And then you judge. Is the note right? Is it what you meant? That judgment did not move. It got more valuable.

The example is invented. The capability is not.

Upside down

Era one: we wrote every rule ourselves, and drowned. Era two: we stopped writing rules; the machine found them in the examples. Era three: we write intent again, one level up, and the people who say what they want, clearly, in writing, get the most out of the machine.

For five hundred years, from Leonardo to Turing, every dreamer got the machine right and the hard part wrong, because the hard part was the machine. The engine, the rocket, the propulsion, the memory.

Now the machine is cheap. It builds in three weeks what took three years. The hard part is the dream, said precisely.

You are all Jules Verne now.

Next: Part 6: The Apprentice’s Problem

The series