ILLIAC IV Supercomputer : DARPA, SIMD, Fairchild and Stanley Kubrick's '2001'
ILLIAC IV Supercomputer : DARPA, SIMD, Fairchild and Stanley Kubrick's '2001'
And the spiritual ancestor of modern GPUs
In 2023, descendants of the architecture that the ILLIAC IV pioneered, using monolithic integrated circuits, are central to the development of modern AI, which looks like it could change the course of the human race, just as Stanley Kubrick’s monoliths did in ‘2001 : A Space Odyssey’
Too many tubes
John von Neumann
on Daniel Slotnik’s original idea for a parallel computer
Stanley Kubrick’s computer in ‘2001 : A Space Odyssey’, HAL 9000, took its name from ‘decrementing’ ‘IBM’. But, there was also a strong connection with a computer built at the University of Illinois. According to the National Air and Space Museum:
As HAL’s circuits are disconnected, “he” recounts his creation in Urbana, Illinois. Why Urbana? In 1968, the University of Illinois at Urbana-Champaign was at the center of research into what we now call “supercomputers.” There, Professor Daniel Slotnick was designing a computer that had not one but 64 separate processors, wired in parallel. It was designed to attack problems that ordinary, single-processor computers could not handle.
That supercomputer was ILLIAC IV.
And the architecture pioneered in the machine that helped inspire ‘HAL’ has become central to the development of modern AI, over fifty years later.
ILLIAC IV was the world’s first massively parallel computer. It was designed to perform many floating point operations at the same time, each run in distinct ‘Processing Elements’ (PE), but with each PE performing the same instruction on its own data. It was thus an early ‘Single Instruction Multiple Data’ machine.
It was an important and pioneering machine, but it was widely considered to be an expensive failure.
I’ve been planning a post on ILLIAC IV for a while. So, I was delighted when Eric Gilliam’s excellent Freaktakes Substack recently published a post containing extensive histories of both ILLIAC IV and a later parallel computer, the Connection Machine. Rather than write another history of ILLIAC IV, I’m going to use this post to link to Eric’s but also supplement Eric’s work with a short summary of ILLIAC IV’s story and then take a look at ILLIAC IV’s architecture.
Here are some of the highlights of the ILLIAC IV story.
(1952) The idea of a parallel computer was devised by Daniel Slotnik whilst working on the Princeton IAS Machine.
(1952) Slotink raises the idea with John von Neumann who rejects it with the comment ‘Too many tubes’.
(1958) Slotnik writes a paper with John Cocke, of IBM 801 fame, on the topic of parallel computers.
(c1962) Working at Westinghouse, Slotnik, designs SOLOMON, a machine with 1,024 processing elements.
(c1963) Slotnick moves to the University of Illinois and joined the Illinois Automatic Computer (ILLIAC) team.
(1964) University of Illinois signs contract with ARPA to build a new machine, which later became known as ILLIAC IV.
(1966) Burroughs Corporation wins the contract to build ILLIAC IV, working with Texas Instruments, and starts work on construction of the computer. By this time the ILLIAC IV has evolved to become a machine with 256 parallel 64-bit floating point units.
(1972) The first ‘quadrant’ of 64 floating point units was completed.
(1972) ILLIAC IV was moved to the NASA Ames Research Center in California to ensure the machine’s safety amid protests based on the belief that it was being used to design nuclear weapons.
(1972) By this point the ILLIAC IV had cost $31m for an incomplete build of a quarter of the design that had originally been budgeted at $8m.
(1973) NASA replaces the Burroughs 6500 machine that had been a front end for the ILLIAC with a PDP-10.
(1973) First programs run on the system, but the results are sometimes wrong!
(1975) The team finally get the ILLIAC IV working properly, albeit at a reduced clock frequency of 13 MHz compared to original specification of 25MHz.
(1975) It’s connected to ARPANET, the forerunner of the internet, becoming the first network connected supercomputer.
(1981) ILLIAC IV is turned off and it’s decommissioned a year later. Non-working sections are shipped to the Computer History Museum at Mountain View, close to NASA Ames.
The ILLIAC IV project was widely considered as a failure. It was wildly over budget. It took almost a decade to get working reliably. And it never came close to meeting its performance target of one GigaFLOP per second.
Slotnik himself said:
I'm bitterly disappointed, and very pleased... delighted and dismayed. Delighted that the overall objectives came out well in the end. Dismayed that it cost too much, took too long, doesn't do enough, and not enough people are using it.
It did have an impact beyond the work of those who used the machine. Crucially, Slotnik pivoted from thin film memories to semiconductor memories made by Fairchild Semiconductor. The success of semiconductor memory in the ILLIAC IV rendered other memory systems obsolete. According to Eric Gilliam:
Some believe that this project helped speed up the pace with which semiconductor memories became commercially available — although given the field was growing so quickly, it is hard to tell if such a claim is true.
Why do I say that it is a ‘spiritual ancestor’ of modern GPUs? Because, it was the first machine with multiple Single Instruction Multiple Data (SIMD) processors, just like a modern GPU.
We’ll look at the ILLIAC IV’s architecture later. But first, here is Eric Gilliam from FreakTakes on ILLIAC IV and the Connection Machine.
Stanley Kubrick and 2001 A Space Odyssey
Back to ‘2001 : A Space Odyssey’. The film starts with the appearance of a black slab, a ‘monolith’. Does the monolith in fact represent a ‘monolithic integrated circuit’ just like the Fairchild memories used in ILLIAC IV?
That’s the claim by Creation Machines : Stanley Kubrick’s View of Computers in 2001 by Mark Midbon. Kubrick had been briefed by engineers at Burroughs, knew about the ILLIAC IV and thus presumably the use and the potential of the integrated circuit. From Midbon’s paper:
… it shouldn't surprise us that Kubrick came to see computers as playing a climactic role in the evolution of the human race and that his imagery points to this role.
And more specifically he relates the references to Urbana, Illinois in the film to ILLIAC IV and how it was being used:
Just as antelope thigh bones have a good side and a bad side, so do computers. The good side is that they are helping the human race attain fulfillment. The bad side is exemplified by Illiac IV, whose first priority was nuclear weapons research. Stanley Kubrick had just finished making ‘Dr.Strangelove, Or How I Learned to Stop Worrying and Love the Bomb’, so we can guess his reaction when his consultants from Burroughs briefed him on the Illiac IV project. That is why the movie mentions Urbana not during the earlier scenes with HAL, while showing his good side, but only after revealing the computer's destructiveness.
In 2023, descendants of the architecture that the ILLIAC IV pioneered, using monolithic integrated circuits are central to the development of modern AI, which looks like it could change the course of the human race, just as Stanley Kubrick’s monoliths did in 2001 : A Space Odyssey
After the break, we’ll take a look at the architecture of the ILLIAC IV.
The rest of this edition is for paid subscribers. If you value The Chip Letter, then please consider becoming a paid subscriber. You’ll get additional weekly content, learn more and help keep this newsletter going!