Paper Summary: The Design Philosophy of the DARPA Internet Protocols

The Design Philosophy of the DARPA Internet Protocols

Maybe this wasn't as exciting a paper as the last: with that off my chest, I can talk about the compelling points it makes.  The perspective is historical rather than rhetorical; instead of proposing or criticizing a system, it takes a system that is as it is and describes broadly the goals that informed the design, attempting to dispel common misconceptions that might arise from an interpretation of the existing protocol implementations.

The paper's main agenda is to show that far from developing in a theoretical vacuum, the protocols were designed to fit an explicit, ordered set of goals.  Among these, the primary goal was reuse of existing systems, and the author points out that even the choice of packet switching (so often considered to be the theoretical foundation of networking) was made largely to satisfy the goal of support and reuse of existing networks and applications, and not because of any theoretical bias against circuit-based networks,  which were also considered.  Following integration of existing networks, obvious design goals include fault-tolerance,support for a diversity of networks and services, scalability, cost-effectiveness and monitoring.  The contents of this list should not surprise us, but the particular ordering (fault-tolerance at the top, cost-effectiveness and accounting at the end) reflect the origin of the requirements in the context of the military.  The initial design directly follows from these goals, and subsequent changes to the protocols, like the separation of the IP and TCP layers, came largely from empirical observations of applications within the framework.

Several details beg comparison to the End to End Argument.  The stateless design of packet-switched networks puts a high burden on the host joining the network, since it needs to implement all the protocols and support all the state: in some sense, rule #6 of the ordered list of goals is sacrificed to support rule #1.  But this architectural decision supports #1 in the most optimal way; having paid the cost of implementing the protocols, a host never needs to pay the cost of re-establishing a connection, except when the network becomes completely partitioned or an end-host fails.  The idea of "fate-sharing" between end hosts is a direct example of the end to end argument in practice, applied to the network domain rather than the software domain (that is to say, in the software model, the "ends" are applications and the "lower levels" are libraries or other subsystems, whereas in the network model the ends are hosts(each with the whole stack)  and the lower levels are the routing infrastructure connecting them).  

Although it is not discussed in detail, the decision to split TCP into a lower IP layer seems like evidence of a protocol rewrite influenced by the end-to-end argument applied to the software domain.  The initial design attempted to build a low-level TCP protocol to be used for all network communication, filled with useful features, using the datagram as transport but hiding this behind an ordered, correct abstraction.  But applications desiring more control or stability ended up reimplementing these features or seeking access to the raw interfaces (buffers, datagrams) behind the abstraction.

Paper Summary: End to End Arguments in System Design

End-to-End Arguments

"End to End Arguments in System Design" articulates, largely through example, a piece of experiential wisdom in systems design that in some ways runs counter to intuition.  Design principles encourage the use of abstractions to hide low-level details, and often this leads to layered systems like the internet protocols.  When designing such layered systems, instinct tells us to push down as much functionality as possible into the lower layers, as this will increase the level of abstraction and simplify the higher levels with which the applications interact.  The paper argues quite convincingly that in many cases following this intuition not merely limits the capabilities of the applications making use of the system, but also results in redundancy when applications at the "ends" are forced to re-implement functionality that has been pushed down, to accommodate potential failures in other subsystems.

The main vehicle for the end-to-end argument takes a theoretical file transfer protocol as the application, the underlying network as the subsystem and correctness of the transferred file (implemented with checksums, redundant copies, timeouts or some combination of these) as the functionality.    It is shown that this is ultimately a systems problem: because the application relies on multiple subsystems (network, file system, volatile memory, etc), any of which may fail, any measures that may be taken to increase reliability or correctness at the network level would have to be duplicated at the application level anyway.  Hence, it is both more flexible and simpler to apply this functionality at the end, and to push the desired functionality into lower subsystems only when failures in those subsystems are so common that overall performance is improved by doing so.  

The authors demonstrate, again convincingly, that the principle illustrated by the file transfer example applies to a range of other applications including message acknowledgement, encryption, sequenced delivery and duplicate elimination.  

If taken to extremes, the end-to-end argument clearly falls apart: we are left, for example, with a file transport protocol that implements the whole network stack itself, in order to preserve flexibility and avoid any redundancies in buffering, congestion control, acknowledgement or checksums.  Extreme as this might be in combination, examples abound of application-level implementations of these functionalities.

This is a great paper to read in a networking class because it provides a strong counter-argument to the prevailing philosophy of aggressive layering and encapsulation in networks.