Draft History of early ARPANET Part 2/3

Draft History of early ARPANET Part 2/3

Post by Michael Haub » Thu, 30 Dec 1993 13:31:14

             Part II. The Network Working Group

        Once the initial sites were picked, representatives from
each site gathered together to start talking about solving the
technical problem of getting the hosts to communicate via
protocols. The ARPA Completion report tells us about this

        "To provide the hosts with a little impetus to work on
the host-to-host problems. ARPA assigned Elmer Shapiro of SRI "to
make something happen", a typically vague ARPA assignment.
Shapiro called a meeting in the summer of 1968 which was attended
by programmers from several of the first hosts to be connected to
the network. Individuals who were present have said that it was
clear from the meeting at that time, no one had even any clear
notions of what the fundamental host-to-host issues might be."
(AC Draft III-67

        Again, we see that this group, which came to be know as
the Network Working Group (NWG), was exploring new territory. The
first meeting took place several months before the first IMP was
put together and they had to think from a blank slate. Throughout
the existing recollections of the important developments the NWG
produced, (especially RFC 1000) the reader is reminded that the
thinking involved was totally original and thus
thought-provoking.  Steve Crocker remembers in the RFC Reference
Guide (RFC 1000) that the first meeting was chaired by Elmer
Shapiro, who initiated the conversation with a list of questions.
(Crocker, 1993b) Also present were Steve Carr from University of
Utah, Stephen Crocker from UCLA, Jeff Rulifson from SRI, and Ron
Stoughton from UCSB.  These attendees are the programmers referred
to in the ARPANET Completion Report.

        In the words of Steve Crocker, this was a seminal meeting.
The attendees could only be but theoretical, as none of
the lowest levels of communication had been developed yet. They
needed a transport layer or low-level communications platform to
be able to build upon.  BBN did not deliver the first IMP until
August 30, 1969. It was important to meet beforehand, as the NWG
"imagined all sorts of possibilities." (Rfc1000) Only once their thought
processes started could this working group actually develop anything.
These fresh thoughts from fresh minds help to incubate new ideas.
The ARPANET Completion Report properly acknowledges what this
early group helped accomplished: " Their early thinking was at a
very high level." (ARPA draft, III-67) A concrete decision of
the first meeting was to continue holding meetings similar to the
first one. This wound up setting the precedent of a holding exchange
meetings at each of the sites.

        Steve Crocker, describing the problems facing these

networking pioneers, writes:

"With no specific service definition in place for what the IMPs
were providing to the hosts, there wasn't any clear idea of what
work the hosts had to do.  Only later did we articulate the
notion of building a layered set of protocols with general
transport services on the bottom and multiple application-
specific protocols on the top.  More precisely, we understood
quite early that we wanted quite a bit of generality, but we
didn't have a clear idea how to achieve it.  We struggled between
a grand design and getting something working quickly."(Crocker,1993c)

        The initial protocol development lead to DEL (Decode-
Encode-Language) and NIL (Network Interchange Language).  These
languages were ahead of their time. The basic purpose was to form
an on-the-fly description that would tell the receiving end how
to understand the information that would be sent.  However, these
first set of meetings were extremely abstract as neither ARPA nor
the universities had deemed any official charter.  The lack of a
charter allowed the group to think broadly and openly however.

        BBN did submit details as to the host-IMP interface
specifications from the IMP side. This information provided the
group some definite starting points to build from. Soon after BBN
provided more information, on Valentine's Day, 1969, members of
the NWG, members of BBN and members of the Network Analysis
Corporation (NAC) met for the first time. [The NAC was contracted
by ARPA to "specify the topological design of the ARPANET and to
analyze its cost, performance, and reliability characteristics.
(ARPA not draft, III-30)] As all the parties had different
priorities on mind, the meeting was a difficult one. BBN was
interested in the lowest level of making a reliable connection.
The programmers from the host sites were interested in getting
the hosts to communicate with each either via various higher
level programs. And BBN also did not turn out to be the "experts
from the East" that Steve Crocker wrote the members of the NWG
expected. He continues by writing in RFC 1000 that they constantly
thought that "a professional crew would show up eventually to take
over the problems we were dealing with."

        A step of incredible importance and openness occurred as a
result from a "particularly delightful" meeting that took place a
month later in Utah. (RFC1000) The participants decided it was
time to start recording their meetings in a consistent fashion.
What resulted was a set of informal notes titled "Request for
Comments." Steve Crocker writes about their formation:

   "I remember having great fear that we would offend whomever the
   official protocol designers were, and I spent a sleepless night
   composing humble words for our notes.  The basic ground rules were
   that anyone could say anything and that nothing was official.  And to
   emphasize the point, I labeled the notes "Request for Comments."  I
   never dreamed these notes would distributed through the very medium
   we were discussing in these notes.  Talk about Sorcerer's Apprentice!"
        (Crocker, RFC 1000, pg 3, 1987)

        Crocker replaced Shapiro as the Chairman of the NWG after
the initial meeting. He describes how they wrestled with creation
of the host-host protocols:

        "Over the spring and summer of 1969 we grappled with the detailed
   problems of protocol design.  Although we had a vision of the vast
   potential for intercomputer communication, designing usable protocols
   was another matter.  A custom hardware interface and custom intrusion
   into the operating system was going to be required for anything we
   designed, and we anticipated serious difficulty at each of the sites.
   We looked for existing abstractions to use.  It would have been
   convenient if we could have made the network simply look like a tape
   drive to each host, but we knew that wouldn't do."
   (Crocker, RFC 1000, pg. 3)

        The first two IMPs were delivered to UCLA (number 1) and
SRI (Number 2). [Footnote 2] Once two IMPs existed, the NWG had
to implement a working protocol. This first set of host protocols
included a remote login for interactive use (telnet), and a way
to copy files between remote hosts (FTP). Crocker writes:

   "In particular, only asymmetric, user-server relationships
   were supported.  In December 1969, we met with Larry Roberts in Utah,
   [and he] made it abundantly clear that our first step was not
   big enough, and we went back to the drawing board.  Over the
   next few months we designed a symmetric host-host protocol,
   and we defined an abstract implementation of the protocol
   known as the Network Control Program. ("NCP" later came to be
   used as the name for the protocol, but it originally meant the
   program  within the operating system that managed connections.
   The protocol itself was known blandly only as the host-host
   protocol.)  Along with the basic host-host protocol, we also
   envisioned a hierarchy of protocols, with Telnet, FTP and some
   splinter protocols as the first examples.  If we had only consulted
   the ancient mystics, we would have seen immediately that seven layers
   were required." (RFC 1000, pg 4)

        After Robert's guidance, the Network Working Group went
forward in developing the protocols necessary to make the network
viable.  The group swelled in attendance as more and more sites
connected to the ARPANET. The group became large enough (around
100 people) that one meeting was held in conjunction with the
1971 Spring Joint Computer Conference in Atlantic City. A major
test of the NWG's work came in October 1971, when a meeting was
held at MIT.  Crocker continues the story,

   "[A] major protocol "fly-off" - Representatives from each site were on
   hand, and everyone tried to log in to everyone else's site.  With the
   exception of one site that was completely down, the matrix was almost
   completely filled in, and we had reached a major milestone in
   connectivity." (Crocker, RFC 1000, pg. 4)

        The NCP was created as what was called the "host to host
protocol." Explaining why this was important, the authors of the
ARPA draft write:

"The problem is to design a host protocol which is sufficiently
powerful for the kinds of communication that will occur and yet
can be implemented in all of the various different host computer
systems. The initial approach taken involved an entity called a
"Network Control Program" which would typically reside in the
executive of a host, such that processes within a host would
communicate with the network through this Network Control
Program. The primary function of the NCP is to establish
connections, break connections, switch connections, and control
flow. A layered approach was taken such that more complex
procedures (such as File Transfer Procedures) were built on top
of similar procedures in the host Network Control Program."
(Arpa draft, II-24)

        As the ARPANET grew, the number of Users bypassed the
number of developers. This signaled the success of these
networking pioneers. Steve Crocker appointed Alex McKenize and
Jon Postel to replace him as Chairmen of the Network Working
Group. The Completion Report details how this role changed:

"McKenzie and Postel interpreted their task to be one of
codification and coordination primarily, and after a few more
spurts of activity the protocol definition process settled for the
most part into a status of a maintenance effort."(ARPA draft,III-69)

        ARPA (Advanced Research Projects Agency) was a management
body which lent funding to academic computer scientists. ARPA's
smart management sense paved the way for these scientists to
create the ARPANET. BBN helped via developing the packet
switching techniques most suitable to passing a wide variety of
information. However, the most important development was that of
the "Request for Comments" documentation.

             3. About RFC's as "Open" Documentation

        The openness initiated from the very first meeting of the
Network Working Group continued on in a more informal formalized
manner in the Request For Comments. As meeting notes, the RFCs
were meant to keep members updated on the status of several
things. They were also meant to gather responses from people.
The Documentation Conventions RFC (RFC 3) documents the "rules"
governing the production of these notes. Topping the page were
the open distribution rules:

"Documentation of the NWG's effort is through notes such as this.
Notes may be produced at any site by anybody and included in this

The guide goes on to describe the rules concerning the contents
of the RFCs:

"The content of a NWG note may be any thought, suggestion, etc. related to
the HOST software or other aspect of the network.  Notes are encouraged to
be timely rather than polished.  Philosophical positions without examples
or other specifics, specific suggestions or implementation techniques
without introductory or background explication, and explicit questions
without any attempted answers are all acceptable.  The minimum length for
a NWG note is one sentence."

The RFC continues to explain the philosophy behind the
unprecedented amount of openness represented:

"These standards (or lack of them) are stated explicitly for two
reasons.  First, there is a tendency to view a written statement
as ipso facto authoritative, and we hope to promote the exchange
and discussion of considerably less than authoritative ideas.
Second, there is a natural hesitancy to publish something
unpolished, and we hope to ease this inhibition." (Crocker, RFC 3
- 1969) [The entire RFC is reproduced in the Appendix B.]

This openness led to the exchange of information. These open
principles are what made the development of the Net possible.

        Statements like the ones contained in RFC 3 are very
progressive in their openness. Late 1960's was a time awash in
popular protest for freedom of speech and demanding more of a say
of how the country is run. The openness contained in trying to
develop new technologies fits well with the cry for more
democracy which students demanded throughout the country and the
world. What is amazing is that the collaboration of the NWG
(mostly graduate students) and ARPA (a component of the
military), seems to be contrary to the normal atmosphere of the
times. Robert Braden of the Internet Activities Board reflects on
this collaboration:

          "For me,  participation in the development of the ARPAnet and
           the Internet protocols has been very exciting.  One important
           reason it worked, I believe, is that there were a lot of very
           bright people all working more or less in the same direction,
           led by some very wise people in the funding agency.  The
           result was to create a community of network researchers who
           believed strongly that collaboration is more powerful than
           competition among researchers.  I don't think any other model
           would have gotten us where we are today." (RFC 1336)

What is even more important is the work of these computer
scientists founded what has lead to the most amazing and
democratic body (i.e.: The Net and the culture attached to it) to
emerge in long time.  The community that has developed and the
tools which accompany it form an important democratic force. [See
footnote 3.]

        The idea of calling these notes a "Request for Comment"
is a fascinating tradition. It predates the Usenet Post, which in
a fashion could be called a "request for comment" as it is the
presentation of a particular person's ideas, questions or
comments, to the general public (of those who read that
newsgroup) for comments, criticism or suggestion, or just plain to
further the readers' knowledge. Other Early RFCs echo this
reality. There exist RFCs which are in response to previous RFCs.
Following are some examples, more are contained in the appendix.

65    Walden, D.   Comments on Host/Host Protocol document #1
1     Crocker, S.  Host software   1969 April 7

39    Harslem, E.; Heafner, J.  Comments on protocol re: NWG/RFC #36
38    Wolfe, S.    Comments on network protocol from NWG/RFC #36
36    Crocker, S.  Protocol notes  1970 March 16

47    Crowther, W. BBN's comments on NWG/RFC #33  1970 April 20
33    Crocker, S.  New Host-Host Protocol  1970 February 12

                       Part IV: Conclusion

        The Network Working Group's development of open technical
documentation - the RFC - was a necessary step to technical
advancement.  Steve Crocker explains the importance of openness
in a developmental situation:

"The environment we were operating in was one of open research.
The only payoff available was to have good work recognized and
used.  Software was generally considered free.  Openness wasn't
an option; it just was." (Crocker, 1993c)

        The NWG's work was important (THE?) to the development of
the ARPANET. Their work paved the way for the development of
TCP/IP, when more capacity was needed and other problems arose.

        I would call the RFC one of the Heralding Achievements of
the NWG. It represents the forward looking view which these
people had and it proved to succeed. The principles which embody
RFC 3 foreshadowed the success of TCP/IP from NCP's influence.
Both TCP/IP and NCP were developed in the field. A version of the
protocols would be released for experimentation and use. Also all
specifications were available free and easily available for
people to examine and make comments about. Only through this
early release were the problems and kinks found and worked out in
a timely manner. This bottom-up approach is substantially
different than the top-down approach which other protocol suites
have been developed under. The top-down idea comes from figuring
everything out as a standard on paper, or behind closed doors and
then releasing it to be used. The bottom-up (and free
accessibility of protocol documentation and specifications) model
allows for a wide-range of people and experiences to join in and
perfect the protocol and make it the best possible. (Check email
in TCPIP.MAIL file to provide quotes.)

        In summing up the achievements of the process that
developed the ARPANET, the ARPANET Completion Report draft

      "The ARPANET development was an extremely intense activity in
which contributions were made by many of the best computer
scientists in the United States. Thus, almost all of the "major
technical problems" already mentioned received continuing
attention and the detailed approach to those problems changed" [II-24]

The computer scientists and others involved were encouraged in
their work by the ARPA philosophy of gathering the best computer
scientists working in the field and supporting them:

"IPT usually does little day-to-day management of its contractors.
Especially with its research contracts, IPT would not be
producing faster results with such management as research must
progress at its own pace. IPT has generally adopted a mode of
management which entails finding highly motivated, highly skilled
contractors, giving them a task, and allowing them to proceed by
themselves." (III-47)

The result, explained by the Completion Report was a new way of
looking at computers as communications devices rather than as
arithmetic devices. Yet many computer science department still do not
understand this significance today.

 | Michael Hauben    CC '95    |  E-mail me for sample copies of       |
 | hau...@cs.columbia.edu      |  The Amateur Computerist Newsletter   |
 | hau...@columbia.edu         | & read the alt.amateur-comp newsgroup |