Parallel computing history (version 2.1)

Parallel computing history (version 2.1)

Post by Greg Wils » Sun, 09 May 1993 03:01:32

The file included below is a brief timeline-style history of the
development of parallel computing and supercomputing more generally, with
some notes on the appearance of theoretical ideas and compilation
techniques.  It is still very incomplete, and may contain errors.

If you would like to help improve this document, please send formatted
entries to  I am looking in particular for:

1) notes on the development of vectorisation and parallelisation
   techniques and tools;

2) information about parallel logic languages (Parlog, Strand88, and
   the like);

3) for information about STRETCH, ATLAS, STARAN, the BSP, and other
   influential projects of the 1960s and 1970s; and

4) for more entries about *significant* events in the histories of key
   companies, particularly Meiko, NCube, Convex, Intel, Encore, CDC,
   Fujitsu, Hitachi, and NEC.

            Gregory V. Wilson       Computing Science
      University of Alberta
        "Things fall apart; it's scientific." (David Byrne)




This document is a timeline of major developments in parallel computing.
It will eventually appear as part of a textbook on parallel programming,
but may also be separately published if there is sufficient interest.
All contributions and corrections are welcomed,
and should be sent to \verb``.



\item   When submitting corrections or additions,
        please submit {\em{only}\/} the new material,
        not an edited copy of this whole document.
\item   Please include your initials in every entry you submit,
        and add them to every entry you change.
\item   Please remember to put each item in the keyword listing
        (field \#2 of each event entry) in braces,
        so that they may be extracted automatically.
\item   Please do not circulate this widely
        until it reaches a stable state.




BMB     & Bruce Boghosian   & \verb`` \\
BRC     & Brad Carlile              & \verb`` \\
WD      & Bill Dally                & \verb`` \\
JF      & Jim Flemming              & \verb`` \\
WH      & Willi Hasselbring & \verb`` \\
LSK     & Larry Kaplan              & \verb`` \\
YO      & Yoshio Oyanagi    & \verb`` \\
HSS     & Harold Stone              & \verb`` \\
PDT     & Philip Tannenbaum & \verb`` \\
GVW     & Greg Wilson               & \verb`` (to July 1993)\\
        &                   & \verb`` (permanent)\\
MW      & Mike Wolfe                & \verb``





\event{1956}{{IBM}, {Stretch}}{MW}
        {IBM starts Stretch project with the goal of producing a machine
        with 100 times the performance of the IBM 704,
        initiated by Atomic Energy Commission at Los Alamos.}

\event{1959}{{IBM}, {Stretch}}{MW}
        {IBM delivers first Stretch computer; less than 10 are ever built.}

\event{1960}{{CDC}, {Cray}, {CDC 6600}}{MW}
        {Control Data starts development of CDC 6600.}

\event{1962}{{Petri}, {Petri-Nets}}{WH}
        {C.~A.\ Petri describes Petri Nets.}

\event{1964}{{CDC}, {Cray}, {CDC 6600}}{GVW}
        {Control Data Corporation produces CDC~6600,
        the world's first commercial supercomputer.}
\event{1964}{{AEC}, {CDC}, {TI}, {Illiac}}{MW}
        {Atomic Energy Commission urges manufacturers to look at ``radical''
        machine structures.
        This leads to CDC Star-100, TI ASC, and Illiac-IV.}
        {Air Force signs Illiac-IV contract with University of Illinois.
        The project is led by Daniel Slotnick;
        primary subcontractors are Burroughs and Texas Instruments.}

\typeout{need project start date, delivery date for Star-100, TI ASC.}

\event{1966}{{Bernstein}, {compilation}, {data dependency}}{GVW}
        {Bernstein introduces Bernstein's Condition
        for statement independence,
        which is foundation of subsequent work on data dependency analysis.}
\event{1966}{{Flynn}, {taxonomy}}{GVW}
        {Flynn publishes paper describing architectural taxonomy.}

\event{1967}{{IBM}, {Tomasulo}, {IBM 360/91}}{MW}
        {IBM produces the 360/91 with dynamic instruction reordering;
        about 20 of these were produced over the next several years.}
        {Amdahl publishes paper questioning feasibility of parallel processing;
        his argument is later called ``Amdahl's Law''.}

\typeout{Significance of instruction re-ordering?? And who is Tomasulo?}

\event{1968}{{CDC}, {Cyberplus}}{MW}
        {Group formed at Control Data to study computing needs for image
        processing; this leads to AFP and Cyberplus designs.}
\event{1968}{{Dijkstra}, {semaphore}}{GVW}
        {Edsger Dijkstra describes semaphores.}

\event{1969}{{CDC}, {Cray}, {CDC 7600}}{GVW}
        {CDC produces CDC~7600 pipelined supercomputer.}

\event{1970}{{C.mmp}, {CMU}}{GVW}
        {Development of C.mmp begins at Carnegie-Mellon.}
        {Floating Point Systems Inc.\ founded
        by former C~N~Winningstad and Tektronix employees
        to manufacturer floating-point co-processors for minicomputers.}
\event{1970}{{DEC}, {asymmetric multiprocessor}}{JF}
        {PDP-6/KA10 master/slave (asymmetric) multiprocessor
        jointly developed by MIT and DEC.}
\event{197x}{{Goodyear}, {STARAN}}{MW}
        {Goodyear Aerospace produces several models of the
        STARAN associative processor.}

\typeout{Need details and dates for STARAN machine(s).}

        {Intel produces 4004 microprocessor (world's first single-chip CPU).}
\event{1971}{{CDC}, {Cyberplus}}{}
        {CDC delivers hardwired Cyberplus
        parallel radar image processing system to Rome Air Development Center,
        where it produces 250 times the performance of CDC~6600.}
\event{197x}{{TI}, {ASC}}{MW}
        {Texas Instruments delivers the Advanced Scientific Computer
        (also called Advanced Seismic Computer).
        Seven of these machines were developed.
        An aggressive automatic vectorizing Fortran compiler
        was developed for this machine.}
\event{1971}{{Dijkstra}, {dining philisophers}}{WH}
        {Edsger Dijkstra poses the dining philisophers problem which is often
        used to test the expressivity of new parallel languages.}

\typeout{Need date for TI ASCI.}

\event{1972}{{Illiac-IV}, {NASA Ames}}{GVW}
        {Quarter-sized (64 PEs) ILLIAC-IV installed at NASA Ames.}
\event{1972}{{BBN}, {Pluribus}}{GVW}
        {BBN builds first Pluribus machines as ARPAnet switch nodes.}
\event{1972}{{CDC}, {CRI}}{GVW}
        {Seymour Cray leaves Control Data Corporation,
        founds Cray Research Inc.}
\event{1972}{{ICL}, {DAP}}{GVW}
        {Paper studies of massive bit-level parallelism done
        by Stewart Reddaway at ICL.
        These later lead to development of ICL DAP.}
\event{1972}{{DEC}, {asymmetric multiprocessor}}{JF}
        {Asymmetric multiprocessor operating system TOPS-10
        developed by DEC for PDP-10 minicomputers.}

\event{1974}{{ICL}, {DAP}}{GVW}
        {Work begins on prototype DAP (Distributed Array Processor) at ICL.}
        {Design begins on Burroughs Scientific Processor (BSP).}
\event{1974}{{Denelcor}, {HEP}}{GVW}
        {Burton Smith begins designing context-flow
        Heterogeneous Element Processor (HEP) for Denelcor.}
\event{1974}{{Hoare}, {monitors}}{GVW}
        {Tony Hoare describes monitors.}

\event{1975}{{CMU}, {Cm*}}{GVW}
        {Work begins at Carnegie-Mellon University on Cm*,
        with support from DEC.}
\event{1975}{{Intel}, {iAPX 432}}{GVW}
        {Design of iAPX 432 (symmetric multiprocessor) begins at Intel.}
\event{1975}{{CDC}, {Cyber}}{MW}
        {Cyber~200 project begins at Control Data.}
        {Illiac-IV becomes operational at NASA Ames after concerted
        check out effort.}
\event{1975}{{Dijkstra}, {guarded commands}}{WH}
        {Edsger Dijkstra describes guarded commands.}

\event{1976}{{CRI}, {Cray-1}}{MW}
        {First Cray-1 delivered to Los Alamos National Laboratory.}
\event{1976}{{CDC}, {Flexible Processor}}{MW}
        {Control Data delivers Flexible Processor,
        a programmable signal processing unit.}
\event{1976}{{FPS}, {array processor}, {LIW}}{BRC}
        {Floating Point Systems Inc.\ delivers 38-bit AP-120B
        array processor that issues multiple pipelined instructions
        every cycle.}
\event{1976}{{FPS}, {software pipelining}, {array processor}}{BRC}
        {Floating Point Systems Inc.\ describes loop wrapping,
        later called software pipelining,
        to program pipelined multiple instruction issue processors.}

\typeout{Relevance of CDC Flexible Processor?}

\event{1977}{{CMU}, {C.mmp}}{GVW}
        {C.mmp hardware completed at Carnegie-Mellon University
        (crossbar connecting minicomputers to memories).}
\event{1977}{{Goodyear}, {MPP}}{MW}
        {Massively Parallel Processor project first discussed at NASA
        for fast image processing.}

\event{1978}{{BBN}, {Butterfly}}{GVW, HSS}
        {BBN begins design of distributed-shared memory machine
        based around ``butterfly'' switch,
        with its roots in work on perfect-shuffle networks by Stone (1972)
        and on Omega networks by Lawrie (1975).}
\event{1978}{{Fortune}, {Wyllie}, {PRAM}}{GVW}
        {Fortune and Wyllie publish paper describing the PRAM model.}
\event{1978}{{Lamport}, {virtual time}}{GVW}
        {Leslie Lamport describes algorithm for creating
        partial order on distributed events.}
\event{1978}{{Hoare}, {CSP}}{WH}
        {Tony Hoare describes CSP.}

\typeout{CHoPP developed by Sullivan??}

\event{1979}{{ICL}, {DAP}}{GVW}
        {ICL DAP delivered to Queen Mary College,

read more »


Parallel computing history (version 2.1)

Post by Paul Repacho » Mon, 10 May 1993 22:19:10

Quote:> \event{1970}{{DEC}, {asymmetric multiprocessor}}{JF}
>    {PDP-6/KA10 master/slave (asymmetric) multiprocessor
>    jointly developed by MIT and DEC.}

The multi-processor capability is documented in the PDP-6 hardware manual,
F-65, Aug 64 DEC. Nothing specific, as the software was pretty non-existant
at the time.

Quote:> \event{1972}{{DEC}, {asymmetric multiprocessor}}{JF}
>    {Asymmetric multiprocessor operating system TOPS-10
>    developed by DEC for PDP-10 minicomputers.}

A PDP-10 a minicomputer?!
Also, TOPS-10 was a re-write of 'the monitor' rather than a new exec.
( Chris White, pers comm, CW was one of the original tops-10 team )

\event{1976?}{{DEC}, {symmetric multiprocessor}}{pr}
      { Dec start on the PDP-11/74, 'Hydra' project. 4 11/70 CPUs on common
        memory system ( MKA-11 ) and an inter-processor interupt system.
        The original cpu was the KB-11cm, the KB-11E never being released.
        The RSX-11M-PLUS os was developed to support this. Was field tested,
        but the HW never made it to the market. M-PLUS did.}

Quote:> \event{1980}{{DEC}, {symmetric multiprocessor}}{JF}
>    {DEC develops KL10 TOPS-10 based symmetric multiprocessor
>    (up to three CPUs supported,
>    but a customer built a five-CPU system).}

DEC was marketing multi 10s in at least the KI-10 era. I have the bumph, and
will dig it out if you are interested. These systems seemed to be targeted
at the newpaper/typesetting industry.

Quote:> \event{1981}{{DEC}, {VAX}, {asymmetric multiprocessor}}{JF}
>    {DEC produces VAX~11/782 asymmetric multiprocessor.}

Note that CMU had the only 11/784. Same cpus, memory, but 4 cpus instead of

As a foot note query: The 10s and 11s had a IO bus per CPU, rather than
the current common IO system. Are these systems SMP? or should we call them
'shared memory' systems?

I will dig out the other bits of dec mp stuff I have as well



Parallel computing history (version 2.1)

Post by Eugene N. Mi » Wed, 12 May 1993 02:55:34

>> \event{1981}{{DEC}, {VAX}, {asymmetric multiprocessor}}{JF}
>>        {DEC produces VAX~11/782 asymmetric multiprocessor.}
>Note that CMU had the only 11/784. Same cpus, memory, but 4 cpus instead of

To my knowledge there were five 784s.  I saw the CMU one on a visit
before it was torn apart.  We had one (from four separate 780s),
LLNL had one (The Circus Vaximus, best name), I believe Manchester had one,
and one was at a classified location.  All used the MA780 shared memory.

  Resident Cynic, Rock of Ages Home for Retired Hackers
  {uunet,mailrus,other gateways}!ames!eugene
Second Favorite email message: Returned mail: Cannot send message for 3 days
A Ref: Mathematics and Plausible Reasoning, vol. 1, G. Polya


Parallel computing history (version 2.1)

Post by Paul Repacho » Wed, 12 May 1993 06:22:34

Quote:> \event{1976?}{{DEC}, {symmetric multiprocessor}}{pr}
>       { Dec start on the PDP-11/74, 'Hydra' project. 4 11/70 CPUs on common

Sorry, wrong monster. Should be "Cerberus" the 3 headed dog at the gates
of hell.