Machines That Learned · Part 1

1943: The Year a Neuron Became Arithmetic

Two men, one a neurophysiologist and one a homeless teenage logician, wrote a paper claiming that a nerve cell could be described completely by a sum and a threshold. They were wrong about the biology and right about something much larger.

The question: How did a simplified neuron become a tool for computation rather than biology?

Big idea: The logical neuron made networks computational, but learning, behavioral tests, and the later AI agenda were separate additions.

The paper is called "A Logical Calculus of the Ideas Immanent in Nervous Activity". It appeared in the Bulletin of Mathematical Biophysics in 1943, which was not a journal anyone read for excitement, and its argument is contained almost entirely in its first move.

Take a neuron. Ignore everything about it that is biological — the chemistry, the timing, the fact that it is alive. Keep three facts: it receives signals from other neurons, it fires or it does not, and whether it fires depends on whether the incoming signals add up past some threshold.

What remains is not a cell. It is an arithmetic operation.

The unlikely authors

Warren McCulloch was a neurophysiologist and a philosopher by temperament, in his forties, interested in a question that sounds more like theology than science: how does matter know anything?

Walter Pitts was still a teenager when they met. He had run away from home at fifteen. He had no degrees and, for stretches, no address. The often-told version of his origin — hiding in a library for three days reading Bertrand Russell's Principia Mathematica, then writing to Russell about errors he had found, and receiving a reply — has the shape of a story that improved in the telling, but the underlying fact is not disputed: Pitts was a logician of extraordinary ability who arrived at the University of Chicago with no formal standing whatsoever.

McCulloch took him in. The 1943 paper was the result.

What the paper actually claims

The claim is stronger than "neurons are like switches", and it is the reason the paper matters.

McCulloch and Pitts showed that networks of these threshold units could compute logical propositions — and that, conversely, any proposition expressible in their calculus could be computed by some network of them. The move is to put neural activity and formal logic in the same notation.

Once they are in the same notation, a question that had been philosophical becomes technical. If thinking is a kind of logical operation, and logical operations can be performed by networks of threshold devices, and threshold devices can be built out of anything at all, then thinking is not intrinsically biological. It is a pattern that biology happens to implement.

You can dislike this conclusion — many people did and do — but it is worth appreciating how much work one modelling decision did. They got there by throwing away almost everything true about neurons.

Which brings up the honest caveat. As biology, the model is wrong in ways that were already suspected then and are undisputed now. Real neurons are not binary. They integrate over time, they fire in rates and patterns, their thresholds move, they are modulated chemically, and a single cell can perform computations far beyond a weighted sum. The McCulloch–Pitts neuron is not a simplified neuron. It is a different object that borrowed the name, and the borrowing has caused eighty years of confusion in both directions.

Hebb, and the part the 1943 paper was missing

The 1943 networks could compute. They could not learn. The weights were set by whoever built the network.

Six years later, Donald Hebb's The Organization of Behavior supplied the missing half with a proposal about how connections change. His actual sentence is more careful than the slogan it became:

When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased.

The slogan — "cells that fire together wire together" — is not Hebb's phrasing and flattens the directionality he was careful about, but the idea is his. Learning is a local, physical change at a connection, driven by correlated activity. Nothing in it requires a designer to know what the network should compute.

Put the two together and you have the entire programme of the next eighty years.

Turing's redirection

Meanwhile Alan Turing was working on a different problem in the same territory, and in 1950 published "Computing Machinery and Intelligence" in Mind.

Turing's contribution is usually summarised as the imitation game, which undersells what he actually did. Faced with "can machines think?", he observed that the question is unanswerable as stated, because it depends on definitions of "machine" and "think" that nobody agrees on. So he replaced it with an operational question about behaviour in a specific game.

This is a philosophical move, and a slightly slippery one — it sets aside the original question rather than answering it, a fact Turing was open about. It also worked. It gave the field a target it could aim at, and a way to argue about progress that did not require settling metaphysics first.

The 1950 paper is also where Turing discusses learning machines and raises, briefly and casually, the idea of building a child-machine and educating it rather than programming an adult one directly. That paragraph has aged remarkably well.

The name arrives late

The phrase "artificial intelligence" was coined in 1955, in a funding proposal. John McCarthy, with Marvin Minsky, Nathaniel Rochester and Claude Shannon, proposed a two-month workshop at Dartmouth College for the summer of 1956.

The proposal's central sentence is worth having in mind for the rest of this series, because it set the field's tone permanently:

The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.

That is a strong conjecture stated as a working assumption. The proposal went on to suggest that significant advance could be made on several of its listed problems by a group of ten people over two months.

Seventy years later, several of those problems are open.

McCarthy reportedly chose the term partly to distance the work from cybernetics and from Norbert Wiener personally. The field's name, in other words, was chosen for political reasons in a grant application — which is a fitting origin for a discipline whose funding history is the subject of part 3.

The workshop itself was not the intense collaboration the proposal imagined; people came and went. But Newell and Simon arrived with something real: the Logic Theorist, a program that proved theorems from Principia Mathematica — the same book the teenage Pitts had read in a library a decade and a half earlier. It found a proof of one theorem that was shorter than Russell and Whitehead's own.

The field had a name, a founding myth, and its first demonstration.

Part 2 is about the machine that followed, the press conference that greeted it, and the book that is usually blamed for killing it.