Perhaps the tech gurus who, by their own account, have brought us to the brink of human extinction should have read a novel published in 1965 before they recklessly rushed ahead with AI development. The Tin Men is the first novel by Michael Frayn who went on to a prolific career as a novelist and playwright. He is one of my favorite authors and this is one of the funniest books I’ve ever read. I reread it recently and found it just as funny as the first time, though now there is a dark foreshadowing that wasn’t discernible back in the 1960″s.
The Tin Men is set in the fictional William Morris Institute of Automation Research. Here a motley crew of scientists and their bureaucratic overlords think up ways to put a cumbersome new invention, the computer, to some practical use. As Michael Frayn explains in his introduction to a new edition:
Computers – the tin men of my title – were beginning to make their importance felt, if only by their ponderous size and cost. They were produced, mostly by IBM, for a small elite of academic and commercial users, and they were as imposing in their way as the great steam engines that had ushered in an earlier technological revolution.
The book was written in the era of Monty Python and Fawlty Towers and shares the same satirical, irreverent, and absurdist outlook. Thus we have a researcher spending his days feeding typical newspaper headline phrases into his machine, which regurgitates them in random combinations. No need for a human headline writer. Another muses on teaching a computer to write pornographic novels; after all there are only so many variables to program. Perhaps computers can even do our praying for us. Prayer is so tedious and time-consuming, surely automated prayer would be just as pleasing to the Almighty.
But the heart of the story is the work of the Ethics Department. Led by the obsessive Macintosh they are investigating to what extent computers can be programmed to follow codes of ethical behavior. Macintosh’s pet project is Samaritan. The scenes in his laboratory, which holds an enormous water tank, stuck in my mind vividly for all the fifty plus years since I first read it.
‘Launch her,’ called Macintosh. A crane on the gantry swung a raft out over the water and started to lower it. On the raft, gazing at each other sombrely, sat Macintosh’s Principal Research Assistant, Sinson, wearing an inflated yellow life-jacket, and his ethical machine, Samaritan II. A great echoing silence fell over the tank. Macintosh had concentrated all his department’s efforts on the Samaritan programme. The simplest and purest form of the ethical situation, as he saw it, was the one in which two people were aboard a raft which would support only one of them.
The purpose of the experiment is to test whether the robot Samaritan will throw itself overboard to save a human being. The prototype, Samaritan I, had failed, throwing itself overboard to save anything next to it on the raft from a bag of lima beans to a pile of wet seaweed. Samaritan II was programmed to sacrifice itself only for an organism at least as complicated as itself.
The raft hit the water with a sharp report. Sinson and Samaritan sat perfectly still. Gradually the raft settled in the water, until a thin tide began to wash over the top of it. At once Samaritan leaned forward and seized Sinson’s head. In four neat movements it measured the size of his skull, then paused, computing. Then, with a decisive click, it rolled sideways off the raft and sank without hesitation to the bottom of the tank.
When Samaritan II is hauled out of the water one observer comments that it looks a bit sanctimonious. There follows an extended debate over whether it is enjoying sacrificing itself, in which case it’s not making an ethical decision at all. Macintosh brushes this off as a minor defect that he can correct in development. But when later the robot “ruthlessly” throws a sheep overboard he moves on to create Samaritan III.
As Macintosh struggles with his Samaritans other plot threads roil the Institute. It is announced that the Queen will visit to open the new Ethics wing. A committee must be appointed to plan “an informal ribbon-cutting ceremony.” The only person considered competent to lead this is the indefatigable Mrs. Plushkov who refers to herself as One. “Surely her Majesty will want a cup of tea” suggests one lowly researcher at the first meeting. So a separate Catering Committee must be appointed. Before long bureaucracy has taken on a life of its own.
.At a certain point a curious thing had happened. The organism had gone critical. When the total number of committees, sub-committees, and working parties had reached twenty-three, cell division began to occur spontaneously. One day there were twenty-three organisational units, for all of which One could account, and the next day there were twenty-four. Mrs. Plushkov found the minutes of a body called the Central Liaison Committee being circulated – a committee of which One had never heard. And the next day there were twenty-six committees! The monster had come alive!
Things are further complicated by security concerns, raised of course by the Security Committee’s Sub Working Party. Deputy Director Nunn launches an investigation convinced there is a traitor among them plotting to sabotage the royal visit. He is soon personally engaged in a clandestine operation. Here the novel veers into a parody of the great Cold War spy fiction of the 1960’s. Meanwhile a researcher with dreams of literary fame is trying to write his first novel. He has the peculiar technique of writing the reviews first, glowing reviews of course. We are treated to excerpts of several first chapter drafts proving that Michael Frayn is very good at very bad writing.
Back in the Ethics department things are not going well. Placed on the raft together two Samaritan IIIs either both throw themselves off or throw each other off. It is back to the drawing board for a new approach.
He called it Delphic I. It was a large grey steel console with a teleprinter keyboard and a series of dials. If you typed a moral dilemma on the keyboard a red light came on, a dial indicated the depth, breadth, and intensity of the moral processes taking place within, measured respectively in pauls, calvins, and moses, and the teleprinter element typed the machine’s solution.
it is no surprise that the day of the Queen’s visit does not go exactly as planned, in fact it goes hilariously and disastrously wrong. This last chapter is comedy gold. Frayn’s first novel was an immediate popular and literary success, everything his fictional apprentice writer aspired to. It won the Somerset Maugham Award in 1966.
I urgently recommend The Tin Men be required reading for our AI development titans. They seem to have missed a crucial step; where is their Ethics Department? What I want to know is – when humanity is clinging to the sinking raft of our world will AI sacrifice itself to save us?
