CHAPTER 3 The Social Forces Behind the Development of Usenet By Michael Hauben hauben@columbia.edu Right at this moment someplace in the world, someone is being helpful (or someone is being helped.) At the same time, others are participating in various discussions and debates. A new communications medium is currently in its infancy. Over the past two decades the global computer telecommunications network has been developing. One element of this network is called Usenet (also known as Netnews). The original carrier of this news was called UUCPnet (or just UUCP). The rawest principle of Usenet is its importance. In its simplest form, Usenet represents democracy. The basic element of Usenet is a post. Each individual post consists of a unique contribution from some user placed in a subject area, called a newsgroup. In Usenet's very beginning (and still to some extent today), posts were transferred using the UUCP utility distributed with Unix. This utility allows the use of phone lines to transmit computer data among separate computers. Usenet grew from the ground up in a grassroots manner. Originally, there was no official structure. What began as two or three sites on the network in 1979 expanded to 15 in 1980. From 150 in 1981 to 400 in 1982. The very nature of Usenet is communication. Usenet greatly facilitates interhuman communication among a large group of users. Inherent in most mass media is central control of content. Many people are influenced by the decisions of a few. Television programming, for example, is controlled by a small group of people compared to the size of the audience. In this way, the audience has very little choice over what is emphasized by most mass media. However, Usenet is controlled by its audience. Usenet should be seen as a promising successor to other people's presses, such as broadsides at the time of the American Revolution and the Penny Presses in England at the turn of the 19th Century. Most of the material written to Usenet is by the same people who actively read Usenet. Thus, the audience of Usenet decides the content and subject matter to be thought about, presented and debated. The ideas that exist on Usenet come from the mass of people who participate in it. In this way, Usenet is an uncensored forum for debate - where many sides of an issue come into view. Instead of being force-fed by an uncontrollable source of information, the participants set the tone and emphasis on Usenet. People control what happens on Usenet. In this rare situation, issues and concerns that are of interest and thus important to the participants, are brought up. In the tradition of Amateur Radio and Citizen's Band Radio, Usenet is the product of the users' ideas and will. Amateur Radio and CB, however, are more restricted then Usenet. Currently the range of Usenet connectivity is international and quickly expanding around the world into every nook and cranny. This explosive expansion allows growing communication among people around the world. In the 1960s, the Advanced Research Projects Agency (ARPA) of the Department of Defense began research of fundamental importance to the development and testing of computer communications networks. ARPA research laid the groundwork for the development of other networks such as UUCPnet. ARPA conducted an experiment in attempting to connect incompatible mainframe computers.(1) This experimental connection of computers was called the ARPA Computer Network (ARPANET). ARPA's stated objectives were: "1) To develop techniques and obtain experience on interconnecting computers in such a way that a very broad class of interactions were possible and 2) To improve and increase computer research productivity through resource sharing."(2) ARPA was both conducting communications research and trying to study how to conserve funds by avoiding duplication of computer resources.(3) Bolt, Beranek and Newman, Inc. (BBN), a Cambridge, Massachusetts company, was chosen to construct the network, and AT&T was chosen to provide the communications lines. The ARPANET was needed because it was found that a data connection over existing telephone voice lines was too slow and not reliable enough to have a useful connection.(4) Packet switching was developed for use as the protocol of exchanging information over the lines. Packet switching is a communications process in which all messages are broken up into equal size packets which are transmitted interspersed and then reassembled. In this way, short, medium and long messages get transferred with minimum delay.(5) The ARPANET was a success. ARPA provided several advances to communications research. ARPANET researchers were surprised at the enthusiastic adoption of electronic mail (e-mail) as the primary source of communication early on. E-mail was a source of major productivity increase through the use of the ARPANET.(6) By 1983, the ARPANET officially shifted from using NCP (Network Control Program) to TCP/IP (Transmission Control Protocol/Internet Protocol.) A key point to TCP/IP's success is in its simplicity. It is very easy to implement over various platforms, and this simplicity has accounted for its continued existence as a de facto standard of the Internet up to the present. ARPANET's lasting contribution was demonstrating how a backbone infrastructure can serve as a connection between gateways. A gateway is a computer or part of a computer programmed to receive messages from one network and transfer them onto another network. ARPANET grew quickly to more than 50 nodes between Hawaii and Norway.(7) However, it did not extend to all who could utilize it. Computer scientists at universities without Department of Defense contracts noticed the advantages and petitioned the National Science Foundation (NSF) for similar connectivity. CSnet was formed to service these computer scientists. CSnet was initially financed by the NSF. Very quickly the desire for interconnection spread to other members of the university community and CSnet grew to serve more scientists than just computer scientists at universities. CSnet became known as "Computer 'and' Science Network" rather than just "Computer Science network."(8) By the mid 1980s, ARPANET was phased out by the Department of Defense, and was replaced by various internal networks (such as Milnet). The role of connecting university communities and regional networks was taken over by the NSF funded NSFnet, which originated as a connection for university researchers to the five National Supercomputer Centers. CSnet and NSFnet were made possible by the research on ARPANET. The NSFnet became the U.S. backbone for the global network known as the Internet. ARPANET research was pioneering for communications research.(9) Researchers discovered the link between computer interconnection and increased productivity from human communication. The sharing of resources was proven to save money and increase computer use and productivity. The development of packet switching revolutionized the basic methodology of connecting computers. The source of these discoveries were the people involved. The personnel involved in the ARPANET project were very intelligent and forward looking. They recognized their position of developing future technologies, and thus did not develop products that commercial industry could (and would) develop. Instead they understood that the communications technologies they were developing had to come from a not-for-profit body. ARPA researchers had no proprietary products to support, and no commercial deadlines to meet. Either would have tainted, or made developing networks of incompatible computers impossible or limited. Current users of international computer networks are in debt to the pioneers of ARPANET. The ARPANET was successful in its attempt to connect various spatially remote computers, and thus more importantly the people who used those computers. However, these people were either professors at universities that had Department of Defense research grants or employees of a limited number of Defense Industry companies. There were still a mass of people who wanted a connection, but were not in a position to gain one. Duke University and the University of North Carolina at Chapel Hill were two such locations. It was in these underprivileged fertile grounds where the grassroots computer communications breakthrough of Usenet originated and developed. The Unix operating system provides the basic tools needed to share information between computers. Unix(10) was developed as "a system around which a fellowship would form."(11) One of the programmers of Unix, Dennis Ritchie, wrote that the intended purpose of Unix was to "encourage close communication."(12) Unix's general principles thus conceptually foreshadowed the basic tenet of Usenet. How else should one go about designing communications programs, but on an operating system which was designed with a basic principle of encouraging communication? The Unix utility UUCP was created at Bell Labs in 1976 by Mike Lesk. It was further developed by David Nowitz and later by Nowitz, Peter Honeyman and Brian E. Redman. UUCP provided a simple way of passing files between any two computers running Unix and UUCP. One of the motivations for AT&T developing Unix was to make software production cheaper in order to bring down the cost of telephone service. Unix's popularity also arose from AT&T's prohibition to profit from other than their main business, phone services, under the terms of the 1956 Consent Decree. Unix was thus available on a "no cost" (or very low cost) basis. The operating system was seen as an "in-house" tool on DEC and other computers and was in use throughout Bell Labs. Many universities used the same type of computer and were licensed by AT&T to utilize Unix. It was thus easily accessible. Schools picked it up, and computer science students used it to learn about operating systems, as Unix was a model of elegance and simplicity compared to most operating systems of the time. Unix became a widely used operating system in the academic world. This paved the way for an international public communications system to form. When Usenet was developed in 1979, it was created to form a `Unix Users Network'. The developers thought Usenet would be used to discuss problems and to share experiences about Unix. Usenet provided a forum for people to solve problems with Unix, as AT&T initially provided little external support for Unix. In an early handout, Usenet was originally referred to as a "poor man's ARPANET."(13) In an email message, Stephen Daniel explained that people who didn't have access to the ARPANET were hungry for similar opportunities to communicate.(14) Usenet has been full of surprises from the beginning. The originators of Usenet underestimated the hunger of the people. As the initial intentions were to produce an easy method of communicating with other users at the same site, the programmers thought people would want to have local bulletin boards.(15) However, people were attracted by the possibility of communicating with others outside the local community as well. Even today, the global communication it makes possible is part of what makes Usenet so enticing. It was also thought Netnews would be useful as a method of communications at individual locations, and between sites close to each other.(16) Usenet grew as a grassroots connection of people. The people who utilized Netnews wanted to communicate, and communicate they did! People have a fundamental need to communicate and Usenet aptly fills the bill.(17) Early in 1981 the gap between ARPANET and Usenet was bridged. The University of California at Berkeley had connections to both ARPANET and Usenet. This allowed Usenet pioneer, Mark Horton, to bring mailing list discussions from ARPANET mailing lists into Usenet newsgroups.(18) This was a significant achievement. Communities other than ARPA sponsored researchers were finally able to see what the ARPANET had made possible. The gatewaying of ARPANET mailing lists into Usenet attracted a wave of people. These people became attracted to Usenet when two ARPANET mailing lists (SF-LOVERS and HUMAN-NETS) began to appear on Usenet. These lists provided interesting material and discussions. The size of the news feed (i.e., the raw data of Usenet) thus became larger and provided more for people to read. Later other sites would serve as gateways to even more discussion lists from the ARPANET. Netnews was also seen as a superior method of holding discussions. Gatewaying these FA (From ARPANET) newsgroups proved to be politically courageous. The ARPANET was accessible by only a certain group of people, and these gateways challenged that notion. The effect on the ARPANET was important as Steve Bellovin, another of the Usenet pioneers, wrote: "The impact of Usenet on the ARPANET was more as a (strong) catalyst to force reexamination (and benign neglect) on the strict policies against interconnection. Uucp mail into the ARPANET became a major force long before it was legit. And it was obviously known to, and ignored by, many of the Powers that Were."(19) Usenet, a network made possible by UUCP, expanded to connect people across the entire continent. Rather early UUCP expanded across the continent when the University of Toronto Zoology Department joined the Net in May of 1981.(20) Two companies proved helpful to this communication by distributing Netnews and electronic mail long distance. Each UUCP site had to either pay the phone bill to connect to the next system, or arrange for the other system to make the phone call. System administrators at AT&T and DEC did the footwork in order to take e-mail and news where it might not have reached. These people went through the trouble in order to try to see the system work. However, easy connections were not always available. In one example, Case Western Reserve University graduate students had to route mail across the continent twice in order to send mail through UUCP to reach their professors who were connected to the ARPANET next door.(23) Usenet encouraged the idea of connectivity to the ARPANET. Gradually the ARPANET was interconnected with other networks eventually functioning more as a backbone to other networks than a self-contained network.(22) Contributed effort is the crucial foundation of UUCPnet and Usenet. On one side, there are those who donate time and energy by contributing to Usenet's content - writing messages and answering messages or participating in a debate. Without the time and effort put in by its users, Usenet would not be what it is today. Also important to Usenet's success are the system administrators who make the functioning of Usenet possible. Resourcewise, Netnews takes up disk space on computers throughout Usenet, and phone calls in some cases must be made to transfer the raw data of the news. In particular, system administrators at AT&T and DEC found it worthwhile to transport Netnews across the country. Certain sites emerged as clearing houses for Usenet and UUCP e-mail.(23) These machines served as major relay stations of both news and e-mail. A structure grew that was considered the "backbone" of "the net." Backbone sites formed the trunk of the circulatory system of news and e-mail. A backbone site would connect to other central distribution computers and to numerous smaller sites. These central backbone sites provided a crucial organization to the Usenet communications skeleton. People formed the center of these connections. For example, 'ihnp4' at AT&T existed mainly because of Gary Murakami's effort and only partially from management support. Usenet services and support were not officially part of Murakami's job description. After Gary left Bell Labs Indian Hill Laboratory (Naperville, Illinois), Doug Price put time and effort to keep things running smoothly. Certain system administrators in universities also picked up the responsibility for distributing Netnews and e-mail widely. Often these individuals would find ways of having their site pick up the phone bill. Sometimes sites would bill the recipients. Also, those who received a free connection were obliged to provide the same to others for no charge.(24) Initially, expansion of sites receiving Usenet was slow.(25) Why did this happen? Initially Usenet was only transported via UUCP connections. Soon, besides UUCP, other resources were used, such as weekly airmailing of magtape data to Australia to provide connectivity.(26) Today, Usenet travels over all types of connections. The evolving ARPANET (and now the Internet) provided a faster way of transporting Netnews. However, a large number of Usenet recipients only have connectivity via UUCP. Universities and certain businesses can afford to connect to the Internet, but many individuals also want a connection. Even as late as 1992 when 60% of Usenet traffic was carried over the Internet via the instantaneous Network Transport Protocol (NNTP), 40% of Usenet was still carried through the slower UUCP connections. There are still many examples of various types of connections using UUCP. These representatives of the "fringe" give a clue to what the origins of this communication must have been like.(27) The number of sites receiving Usenet continually increased, demonstrating its popularity. People were attracted to Usenet because of what it made possible. People want to communicate and enjoy the thrill of finding others across the country (or today across the world) who share a common interest or just to be in touch with. Besides the common thrill, it is possible to form serious relationships online. Usenet makes this discovery possible because it is a public forum. People expose their ideas broadly. This wide exposure makes it possible to find compatriots in thought. The same physical connections which carry Usenet often also transport electronic mail. Interactions and discoveries are only made possible by the public aspect of Usenet. Mailing Lists have as wide a range of discussion, but are available to a much smaller sized group. The appeal of Usenet can become tiresome at times(28), but it is rare that anyone leaves Usenet permanently. Unless, of course, someone can't find the time to fit Usenet into his or her life. As more universities, schools, libraries, businesses, and individuals connect, the value of Usenet grows. Each new person eventually can add his or her unique opinion to the collection of thoughts and information that Usenet already has. Each new connection also increases the area where new connections can be made through cheap local phone calls. The potential for inexpensive expansion is limited only by the oceans, other natural barriers or perhaps mistaken government policies. The ARPANET was supplemented by CSnet and eventually replaced by U.S. government funding of its successor, NSFnet. Both CSnet and NSFnet were created by the U.S. Government in response to research scientists' and professors' pleas to have a similar connection to the ARPANET. The NSFnet was also created to provide access to the five supercomputer computing centers around the country. The NSFnet, as the backbone of the U.S. portion of the Internet, provided another route for the distribution of Usenet. Similar to the ARPANET, NSFnet was a constant connection run over leased lines. One of the ways Netnews is distributed is using the NNTP protocol over Internet connections. This allows for Netnews and e- mail to be distributed quickly over a large area. Internet connections also assist in carrying Usenet and e-mail internationally. The Internet class networks and connections include the established government and university sponsored connections. However some of the way individuals are connected at home is via phone lines and various versions of UUCP. There are also commercial services that, for a fee, provide connections for electronic mail and Usenet access, as well as access to the Internet. Much of the development of Usenet owes a big thanks to the early restrictions on commercial uses. Where else in our society has the commercial element been so clearly separated from any entity? Forums of discussion and communication become clogged and congested when advertisements use space. Because of the voluntary actions of those who use and redistribute Netnews and e-mail, people on Usenet feel it wrong to assist commercial ventures. When people feel someone is abusing the nature of Usenet, they let the offender know through e-mail and in public messages. In this manner users work to keep Usenet as a forum that is free from commercial exploitation. Usenet is not allowed to be abused as a profit making venture for any one individual or group. Rather, people fight to keep it a resource that is helpful to society as a whole. On what was the ARPANET and what was afterward the NSFnet portion of the Internet, there were Acceptable Use Policies (AUP) that existed because these networks were initially set up, founded and financed by public money. On these networks, commercial usage was prohibited, which meant it was also discouraged on other networks that gatewayed into the NSFnet. Unfortunately, the NSF encouraged privatization of the NSFnet backbone.(29) However, the discouragement of commercial usage of the global Usenet is separate and developed differently from the AUP. The social network that Usenet represents supersedes the physical connection it rides on. The current Netnews rides on many of the physical networks that exist today. However, if ever there were the need, Usenet could reestablish itself outside of the current physically organized networks. Usenet's quality is such that it will survive because of its users' determination. Usenet draws its strength from being a peer to peer network. People who use Usenet do so because they wish to communicate with others. This communal wish means that people on Usenet find it in their own and in the community's interest to be helpful. In this way, Usenet exists as a worldwide community of resources ready to be shared. Where else today is there so much knowledge that is freely available? Usenet represents a living library. Usenet is an important part of the worldwide computer networks. The very nature of Usenet promotes change. Usenet was born outside of established "networks", and transcends any one physical network. Currently, at this time, it exists of itself and via other networks. It makes possible the distribution of information that might otherwise not be heard through "official channels." This role makes Usenet a herald for social change. Because of the inherent will to communicate, people who don't have access to Usenet will want access when they become exposed to what it is, and people who currently have access will want Usenet to expand its reach so as to further even more communication. Usenet might grow to provide a forum for people to influence their governments allowing for the discussion and debate of issues in a mode that facilitates mass participation. This becomes a source of independent information. An independent source is helpful in the search for the truth. Administrators and individuals who handle the flow of information have been predicting the "imminent death of the net" since 1982.(30) The software that handles the distribution of Netnews has gone through several versions to handle the ever increasing amount of information. People who receive Netnews have either had to decrease 1) the number of days individual messages stay at the site or 2) the number of newsgroups they receive; or they have had to allocate more disk space for the storage of Netnews. Despite all the predictions and worries, people's desire for this communication have helped this social network develop and expand. Brad Templeton once wrote, "If there is a gigabit network with bandwidth to spare that is willing to carry Usenet, it has plenty more growth left."(31) Various research labs have been working on producing usable gigabit networks. Usenet is a democratic and technological breakthrough. The computer networks and Usenet are still developing. People need to work towards keeping connections available and inexpensive, if not free, so as to encourage the body of users to grow. There are several cities and governments across the world where the public has access to network services as a civic service. This direction is to be encouraged. Exclusive arrangements for access are to be discouraged. The very nature of Usenet means people are going to be working for its expansion. Others will be working for the expansion for their own gain, and some forces will be an active force against expansion of Usenet. I can only ask that people attempt to spread this book in an attempt to popularize and encourage the use and fight for Usenet. -------------------------- Notes for CHAPTER 3 (1) "In September 1969, the embryonic one-node(!) ARPANET came to life when the first packet-switching computer was connected to the Sigma 7 computer at UCLA. Shortly thereafter began the interconnection of many main processors (referred to as HOSTs) at various university, industrial, and government research centers across the UnitedStates." (Leonard Kleinrock, "On Communications and Networks," IEEE Transactions on Computers, vol C-25 no 12, December, 1976, p. 1328). (2) F. Heart, A. McKenzie, J. McQuillan, and D. Walden, ARPANET Completion Report, Washington, 1978, p. II-2. (3) Alexander McKenzie and David C. Walden, "ARPANET, the Defense Data Network, and Internet" in The Froehlich/Kent Encyclopedia of Telecommunications, vol. 1, New York, 1991, p. 346. (4) Lawrence G. Roberts, "The ARPANET and Computer Networks," in A History of Personal Workstations, ed. Adele Goldberg, ACM Press, New York, 1988, p. 145. (5) Leonard Kleinrock, "On Communications and Networks", IEEE Transactions on Computers, vol C-25 no 12, December, 1976, p. 1327. (6) Alexander McKenzie and David C. Walden, p. 357. (7) F. Heart et al, p. II-25. (8) Alexander McKenzie and David C. Walden, p. 369. (9) "For many of the people in government, at the major contractors, and in the participating universities and research centers the development of the ARPANET has been an exciting time which will rank as a high point in their professional careers. In 1969 the ARPANET project represented a high risk, potentially high impact research effort. The existence of the net in practical useful form has not only provided communications technology to meet any short term needs, but it represents a formidable communications technology and experience base on which the Defense Department as well as the entire public and private sectors will depend for advanced communications needs. The strong and diverse experience base generated by the ARPANET project has placed this country ahead of all others in advanced digital communications science and technology." (ARPANET Completion Report, p. II-109.) (10) Unix was born in 1969, the same year as the ARPANET. (11) Dennis. M. Ritchie, "The UNIX System: The Evolution of the UNIX Time-sharing System," Bell Systems Technical Journal, vol 63 no 8, October, 1984, p. 1578. (12) Ibid. (13) Stephen Daniel, James Ellis, and Tom Truscott, "USENET - A General Access UNIX Network," Duke University, Durham, North Carolina, Summer 1980. (14) Stephen Daniel, a personal communication, November 1992. (15) Steve M. Bellovin and Mark Horton, "USENET - A Distributed Decentralized News System", an unpublished manuscript, 1985. (16) Ibid. (17) See, e.g., Gregory G. Woodbury's "Net Cultural Assumptions," reprinted in Amateur Computerist, vol 6 no 2/3, Winter/Spring, 1994-5. (18) Comment from Steve Bellovin, October 10, 1990, Usenet History Archive: "Correct. The original concept was that most of the traffic would be the form now known as UNIX wizards (or whatever it's called this week). Growth was slow until Mark started feeding the mailing lists in because there was nothing to offer prospective customers. Given a ready source of material, people were attracted." (ftp://weber.ucsd.edu/pub/usenet.hist/nethist.901010.Z) (19) Steve Bellovin, October 10, 1990 - Usenet History Archive. (ftp://weber.ucsd.edu/pub/usenet.hist/nethist.901010.Z) (20) Henry Spencer, Usenet History Archives, ftp://weber.ucsd.edu/pub/ usenet.hist/history.Z. (21) From Amanda Walker, Tue, Oct. 16, 09:11 PDT, 1990, Usenet History Archives, ftp://weber.ucsd.edu/pub/usenet.hist/nethist.901016.Z, (22) Alexander McKenzie and David C. Walden, "Indeed, during a typical measurement period in June 1988, over 50% of the active ARPANET hosts were gateways, and they accounted for over 80% of the traffic." p. 369. (23) At AT&T, the computers 'research', then 'allegra', then 'ihnp4' served as major mail and/or news distribution sites. At DEC - 'decvax' gradually increased its role (e.g., 'decvax' in New Hampshire would call long distance to San Diego across the country.) (24) E.g., Duke University fed Usenet files to Greg Woodbury who in turn gave "feeds" as they are called to others who requested them from him. See "Net Cultural Assumptions." (25) See table of site growth in Chapter 2. (26) Andrew Tanenbaum is quoted as saying something similar to "Never underestimate the bandwidth of a station wagon full of 9 track tape (or magnetic tape)." (27) Usenet began with the spirit that still exists today. On several newsgroups I posted a message with the following subject: "I want to hear from the four corners of the Net - That means YOU!" In return I received numerous wonderful answers. One new pioneer was going to use packet radio to send e-mail up to the CIS's orbiting Mir Space Station in the heavens. Others in Krakow, Poland, in Australia and in the ex-USSR sent me information about their connection. Some told me of how they made other connections possible. One user in South Africa told me how he distributed news and e-mail and was trying to gain access to a satellite in order to set connections up with the interior of Africa that lacks the otherwise needed infrastructure. The world is still in the infancy of this communications interconnectivity! See description of this result in Chapter 1. (28) "Flame Wars" (highly emotional attacks) can become annoying. There are ebbs and flows of interesting posts. Even though Usenet is addicting, it can also be overwhelming. (29) See e.g., the U.S. Office of Inspector General's Report of April, 1993, for documentation of the process set in motion to implement the privatization of the NSFnet. (30) From the Usenet History Archives, ftp://weber.ucsd.edu/pub/usenet.hist/ (31) From Usenet History Archives, ftp://weber.ucsd.edu/pub/usenet.hist/ posthist.Z ----------- Special Thanks to Bruce Jones for establishing and archiving the Usenet History Archives at ftp://weber.ucsd.edu/pub/usenet.hist/ Also thanks to the Usenet Pioneers for getting Usenet off to the right start. Last Updated: October 15, 1995 =============================================================== This article is a draft chapter from Michael Hauben's and Ronda Hauben's netbook titled "Netizens: On the History and Impact of Usenet and the Internet." *Commercial use is prohibited* Please send us any comments about this draft. Send comments to both hauben@columbia.edu and rh120@columbia.edu.