History of Packet Switching (how the Internet works)

Generally speaking communication networks are one of two types: circuit switched or packet switched. Circuit switching is like the traditional telephone network, each telephone line (circuit) is physically connected (switched) to another for the duration of the call. In the early days this switching was done by human operators physically putting in plugs to connect lines together. Later on automatic switching used electromechanical switches controlled by the number dialled and then solid state electronics, but the end result was the same - a physical dedicated electrical circuit between the two endpoints for the entire duration of the call. Circuit switching is great for real time communications like voice or video because it provides guranteed dedicated bandwidth at all times but for transmission of computer data it is quite inefficient.

Data transmission tends to be bursty - a burst of data is sent then everything goes quiet for a while. Dedicated bandwidth such as that provided by circuit switching is wasted during these quiet periods. With packet switching a single physical circuit between two points can be used to carry data from other users during the quiet periods. Also by splitting the data up into small chunks (packets) and switching each one seprately, fault resilience is built in as each packet can take a different route bypassing problems before the whole lot is re-assembled at the distant end.

Donald Davies & The National Physical Laboratory

The history of packet switching goes back to UK National Physical Laboratory (NPL). Donald Davies who was in charge of the Computer Science Department at NPL realised that a better way of sending data between computers than the circuit switched network. Davies proposed splitting the data into packets with each packet addressed with its source and destination address and its sequence number so that all the packets could be put back in order at the destination. At each switching point on the network, known as a switch (often called a router), the switch or router would examine the destination address and send the packet out on the appropriate line, interwoven with completely unrelated packets from other users. If the router detected a problem with one of its lines it would consult its software based routing table to see if it had an alternative route to get it towards its destination.

Davies wrote a paper describing his packet switching ideas in 1965. In it he envisioned a national network of switching centres (routers) like the phone network but for carrying computer data.

NPL launched its own internal packet switched local area network (LAN) in 1971. This was the first practical use of packet switching techniques. A mark 2 version of the network was operational 2 years later. In the late 60's when the NPL LAN was first being designed there was no off the shelf networking hardware to use as building blocks. Everything down to the circuit board level had to be deisgned and build from scratch, along with the software in each router. The routing decisions were carried out on a Honeywell DDP516 mini computer.

Accross the pond

Also around this time the US Department of Defense's Advanced Research Projects Agency (ARPA, later re-named DARPA) was carrying out research into how to build communications networks which would allow sharing of computer power scattered around the country at research sites and universities.. Packet switching with its efficiency of line usage and its 'self healing' properties - being able to dynamically re-route packets around a failed router or line soon caught their attention. Building on the ideas from NPL ARPA launched its own packet switched network ARPANET. Like NPL, ARPANET also used the Honeywell DDP516 with custom hardware to form a router referred to as an 'Interface Message Processor' (IMP). Each IMP could connect to upto 6 others as well as local computers which needed to send / receive data. ARPA initally connected 3 university sites in California and one in Utah.

The first message sent over the network was on October 29th 1969. A student at UCLA sent the message 'login' to a computer at Stanford. The letters 'lo' were sent before the system crashed. Recovering from the technical problems later that day the first full message was sent and before the end of 69 the complete network of 4 IMPs was up and running. Between 1970 and 75 the network grew to have 57 IMPs spanning the US. At this point the network was deemed operational rather than a research project and so control was handed over to The Defence Communications Agency.

As ARPANET grew it connected together government agencies and universities and research centres throughout the US and in Europe. It enabled researchers everywhere to exchange information and share computing resources with unprecedented speed, boosting collaboration and fostering a culture of open collaboration between software developers which continues to this day in the numerous open source projects which create and maintain key pieces of software which power todays Internet. As well as scientific discussion, ARPANET also catered for more cultural and social collaboration through mailing lists covering a wide variety of interests.

Many of these centres had other networks of differing sorts and the communication protocol governing the format of the packets on ARPANET (known as Network Control Protocol - NCP) was starting to show its weaknesses as the network expanded far beyond the size and boundaries originally envisaged. In 1973 two DARPA scientists Vint Cerf and Bob Kahn started to work on a design for a new protocol. One of the goals for the new protocol was to be able to interconnect many subnetworks of differing types so that they could all communicate as one big interconnected network or 'internet'. The idea Kahn & Cerf came up with was to have a device similar to an IMP called a router at the boundary between each network. To simplify the design the new protocol was split into a Transmission Control Protocol (TCP) and an Internet Protocol (IP). The first test of TCP/IP was carried out in 1975 and as the system evolved through versions 1,2,3 & 4 between 75 and 83, successive testing was carried out with the core network still running the old ARPANET protocol. On January 1st 1983 the ARPANET was completely switched over to IPv4.

The US Department of Defense declared TCP/IP as the standard for all military computer networking and companies such as IBM,DEC and AT&T adopted it for their products. Version 4 introduced in 1983 continues to be the most wideley used packet switching protocol both on the public internet and on private local area networks. Version 4 uses 32 bits for it's addresses, giving 4,294,967,296 (4.29 billion) unique addresses, which back in 1983 with the then projected growth of ARPANET seemed plenty. However with the explosion in growth of the Internet over the subsequent decades and the huge number of devices now connected, that address pool is starting to run out. The Internet Society predicts that by 2016 there will be 20 billion devices on line each needing an address. Technologies such as Network Address Translation have come along and reduced the urgency of the problem by allowing subnetworks such as your home/office LAN to sit behind one public IP address, with all the internal devices (PCs, tablets etc) using an internal private address, however, the number of spare public addresses is still running out.

A new version of IP, version 6 was standardised in the mid 90's to address the shortage of addresses by increasing the number of bits used for the address from 32 to 128 (meaning 340 trillion trillion trillion possible addresses), and to introduce other features such as multicasting (ability to send packets to multiple destinations simultaneously) and security. However despite compaigns over the years to encourage companies to move to IPv6 it still only carries a minority of net traffic - in September 2013 the amount of traffic reaching Google over IPv6 reached 2% for the first time according to The Internet Society. Whilst this may be a small amount of the total net traffic, it is double that using IPv6 when a major campaign was launched on 6th June 2012 to encourage adoption.

IPv6 is the Internet Protocol of the future because without it there just arent enough Internet addresses to go around. As billions more devices and people are Internet connected, IPv6 momentum continues to grow. The Internet Society

Berkeley Unix & the implementation of TCP/IP

One of the collaborative software projects helped by ARPANET was the Unix operating system. Originally created by AT&T&'39;s Bell Research Labs it pioneered a numer of unique ideas for the time. One of these was that by writing the bulk of the code in a high level language (C, which Bell Labs created for the task), Unix was easily portable to different makes / models of computer - providing a common environment for users, and a platform on which to build more easily portable applications further easing the collaborative research between universities. AT&T at the time was prohibited from competing in the computer business as it already had such a large monopoly on the communications market. Unable to sell Unix for profit, Bell Labs licensed it at cost to universities. One such university which adopted Unix was The University of California, Berkeley. Berkeley soon became heavily involved with Unix development, creating enhancements and bug fixes and feeding them back to Bell Labs. Berkeley's enhanced version of Unix, which had become known as BSD (Berkeley Software Distribution) caught the attention of the guys at DARPA in the late 70's.

As ARPANET approached ten years old in 1979, many of the computers connected to it were reaching the end of their life. Replacing them with newer machines meant the need to port the applications over to the new proprietary and incompatible operating systems shipped with each one. Unix, with its cross platform portability, reliability and active development community looked a promising solution. Professor Bob Fabry of Berkeley proposed that they develop an enhanced version of BSD Unix for the DARPA community. A contract was signed in 1980 for Berkeley to develop BSD on DARPAs behalf. Initial work focussed on improving speed and and general features. The result, BSD version 4 was released in October 1980. A followup version 4.1 improved performance further and its success and popularity led to a further contract being signed between DARPA & Berkeley for a further two years work.

One of the major features to come as a result of this contract was the implementation of the new TCP/IP packet switching protocols into the operating system. DARPA had previously contracted out development work for ARPANET to a company called BBN and they had produced some early prototype code implementing TCP/IP which Berkeley integrated into BSD as a stop gap measure until the TCP/IP protocol was finalised. This code, shipped in BSD4.1a,b&c and then BSD4.2 in 1983 enabled sites to begin moving their ARPANET connected computers over to the new Internet Protocol. As feedback came in Berkeley programmers refined and refactored the design to improve performance, evolve it as IP evolved in design, and make the code easier to maintain and extend in the future. When BBN delivered the final version of their TCP/IP code to Berkeley, Mike Karels, one of the Berkeley programmers studied it and decided it had some flaws in its design and that the earlier code which had benefited from several years of refinement as part of BSD was the better design. In the interests of fairness Berkeley proposed shipping both, and letting users choose which to use, but DARPA expressed concern that this would lead to confusion, software bloat and wasted effort fielding bug reports and design suggestions for two code bases. BBN were also unhappy as their plan had been to replace the entire body of existing TCP/IP code with their in house design.

To reach a compromise BBN & Berkeley agreed to allow DARPA to be an independent judge of the best approach and DARPA decided that the BSD codebase was indeed the better design so rather than rip out and replace the old IP code in BSD, they would instead take the best ideas from the new code and evolve the old code base which had benefited from several years testing, debugging and refinement already. Because of the open-source nature of the BSD license (link to Wikipedia) permitting widespread use of the source code in both open and closed source products the status of IP as the dominant networking protocol was further cemented and the BSD networking code became the defacto reference implementation for TCP/IP.

Berkeley continued to develop BSD Unix throughout the 80's and into the 90's By the time the official team of BSD programmers employed by the university was disbanded in 1995 BSD Unix had grown into a mature enterprise grade operating system powering the servers behind the Internet. Due to the open nature of the BSD license BSD Unix continues to evolve in the form of 3 open source projects which forked from BSD Unix in the early 90's These are FreeBSD (link to FreeBSD.org), NetBSD (link to NetBSD.org) &OpenBSD (link to OpenBSD.org) and these Unix derived operating systems, together with the newer Unix clone Linux continue to power the majority of servers on the net including popular services like Yahoo, Netflix, WhatsApp, Google, Facebook , Twitter and Wikipedia. In addition FreeBSD is used as the basis of the operating system on Sony' PS4, Juniper IP Routers and even Apple''s OS X and iOS.