Why Not ATM?

Terry Gray
6/1999; rev 11/2000

OUTLINE

OVERVIEW

From time to time (though ever less frequently), people ask why we have not used ATM technology for the University of Washington campus network backbone, nor for the Pacific/NorthWest Gigapop. Instead, UW has steadfastly advocated a "frame oriented" --rather than an ATM "cell oriented"-- approach and based our network infrastructure on Ethernet technology. Often I explain this simply by saying "We don't believe in packet shredding technologies", but sincere inquiries deserve a more in-depth response. This document will attempt to provide that.

Originally, ATM was conceived as a complete end-to-end replacement for all other networking technologies. It was expected by proponents to dominate in all three networking domains: desktop, campus backbone, and wide-area. Cost and complexity defeated ATM to the desktop; that's now old news. The same thing is happening today in the campus/enterprise backbone. That leaves the wide-area as the last bastion of ATM, and *for now* there are some good reasons to use ATM in certain wide-area situations --but as Internet Protocol traffic displaces or absorbs all other forms of communication, and methods for emulating or simulating virtual circuits in a pure IP environment are deployed, I predict the same thing will eventually happen in the wide-area domain as well: simpler alternatives (e.g. Packet-over-SONET and Packet-over-optical-lamda) will prevail as competitive pressures inevitably squeeze ATM's cost and complexity out of the picture. Because of enormous telco investments in ATM, it will be around for many years, but contrary to early claims, it will ultimately be superseded.

Do other universities share our view that ATM is not the future of networking? Certainly not all of them, but I believe that more each year conclude that Fast and Gigabit Ethernet represent a more cost-effective approach for enterprise backbone networks. I expect this trend to continue as the cost for Gigabit Ethernet continues to come down, and support for QOS (Quality of Service) in a pure IP environment improves.

Although five years ago my suspicions about ATM were indeed considered reactionary by many, I think it is fair to say my views have become unexceptional, indeed, mainstream. (However, I still manage to find inflammatory ways of expressing them, as when I refer to ATM as a "packet shredding" technology :)

KEY ISSUES: COST and QOS

The principal arguments between Ethernet and ATM have to do with cost and adequacy for the intended purpose, especially QOS support for multimedia applications. Let's first consider hardware cost:

ATM cards may have become a bit cheaper since my last datapoint, but they are way more than an order-of-magnitude higher than a Fast Ethernet card. The $10 100Mbps NIC is a value proposition that cannot be denied, and is precisely why end-to-end (or desktop) ATM is history. Moreover, the same ratio exists for the cost of switch ports in the supporting infrastructure, and this is why fast and gigabit Ethernet is now winning the war for the enterprise backbone. And that's just hardware cost... there are training and operational costs which also favor the (much simpler to manage) Ethernet family of technologies. Cost also takes the form of ATM's "cell tax", discussed later, which wastes a substantial fraction of the original channel capacity.

But what about "adequacy"? In particular, don't we need ATM to provide adequate QOS for advanced multimedia networks? Short answer: No. The basic question is whether the resource you are trying to manage costs more or less than the mechanism intended to manage it. With gigabit Ethernet now a proven technology with declining prices, and 10 gigabit Ethernet just around the corner, it is increasingly difficult to argue that bandwidth is scarce enough within an enterprise to justify complex bandwidth management techniques, such as ATM. Indeed, I have authored a white paper on why a simple DiffServ-oriented approach to QoS is appropriate within an enterprise; you don't even need to use "hard" Internet-oriented QOS strategies (e.g. RSVP).

The last bastion of ATM is in the wide-area, and there are several reasons for this. First is the predisposition of telcos to use a circuit-oriented technology, second is the legitimate requirement of telcos to be able to multiplex other layer 2 services (e.g. SMDS, Frame Relay, Transparent LAN Service, etc) over their networks, and finally, even within a pure IP environment, there is value in having a mechanism for creating virtual circuits. As Internet packet traffic absorbs everything else, issue two evaporates, though some form of virtual circuit capability will continue to be useful for QOS and routing management.

My conviction that ATM will ultimately lose momentum in the wide-area, just as it has in the desktop and enterprise backbone arenas, is based on the premise that it will be possible to provide suitable alternatives to ATM virtual circuits in a pure IP environment. Whether this will be done via source and policy-based routing, combined with more sophisticated queuing algorithms in routers, or by using tag switching (now called "Multi-Protocol Label Switching") is not yet clear. MPLS is generally viewed by Internet folks as less onerous than ATM, but it too adds complexity, by requiring the management of a distinct network topology over which the IP connections are overlaid. Since complexity is THE number one enemy in managing network infrastructure, the best (simplest) outcome is a traffic prioritization and routing solution based soley on IP headers. Whether DiffServ or policy routing or MPLS ultimately prevail in meeting real-world QoS requirements, the last nail will soon be in ATM's coffin --at least in terms of technical issues.

In addition, advances in optical communications such as DWDM (Dense Wave Division Multiplexing) promise to drastically reduce the cost of wide-area bandwidth, thus reducing the need to micro-manage small chunks of bandwidth via traditional virtual circuits and admission controls (the data equivalent of busy signals). Even today, a number of Internet Service Providers have opted for packet-over-SONET backbones rather than national ATM clouds in order to avoid the so-called "cell tax" that accrues when running IP packets over a cell-relay (ATM) network. The cell tax represents about a 20% loss of the available channel capacity. Ultimately decision makers tend to go with the least expensive option that meets their needs. A few years ago, the cheapest way to get national bandwidth was via ATM clouds, but that economic picture has changed as a result of a new generation of telcos (e.g. Quest, Level-3) who are creating their own national fiber backbones.

I should mention that there is one venue where ATM appears to be pretty firmly entrenched and not likely to be displaced anytime soon, and that is in transport for DSL services to homes. As far as I know, most telcos have chosen to use ATM at the transport level for their high-speed data services, attracted by the prospect of having multiple PVCs (Permanent Virtual Circuits) to each home for different kinds of services. It's not at all clear that this capability will prove to be important in our increasingly IP-centric world, especially since the in-home DSL devices being deployed today typically convert the DSL connection into Ethernet, but it should be mentioned for completeness. In contrast, high-bandwidth cable data service into the home is typically not ATM-based, and as mentioned in a later section, Milo Medin, @Home's CTO, considers this a feature, not a bug.

EARLY SUSPICIONS AT UW: June 1996

Date: Tue, 11 Jun 1996 13:34:27 -0700 (PDT)
From: Terry Gray 
Subject: Updated ATM summary

The future of ATM as seen from UW...

Observations:

 o The only thing growing faster than the Internet is hype about ATM.
   Many people actually believe that ATM is the only way to achieve
   dedicated, scalable bandwidth; indeed the only way to "go fast".
   (This was true until the ATM-96 conference, when a large dose of
   reality started setting in :)

 o Many of the original assumptions in ATM design are obsolete.
   Specifically:
	-the tiny cell size was chosen based on low-bandwidth links
	 instead of the fast links now used.  Also IP packet sizes
	 are increasing.
	-the assumption that most traffic will be constant-bit-rate
	 no longer holds.  Audio and video streams will be compressed,
	 turning them into bursty variable-bit-rate streams.

 o When speaking of ATM, one must distinguish two issues:
	-Is ATM the "universal bearer service" that applications
	 understand?  Or does sockets-over-IP continue to fill that role?
	-Is ATM a desirable link-layer technology underneath IP?

 o If one believes that the world will become native ATM, end-to-end,
   (that is, it becomes the universal bearer service) then many objections
   to ATM go away, but this is not likely to happen anytime soon.

 o The marriage of IP and ATM is one not made in heaven.
   Issues include:
	-Congestion collapse due to the mismatch between the size of
	 element being dropped and the size of the element being
	 retransmitted.
	-broadcast domains: Several IP protocols require broadcast
	 semantics, which are inefficient in a switched network.
	-LAN Emulation: "All the disadvantages of ATM with none of the
	 advantages"

 o Network management is a big concern.  We expect that managing large ATM
   nets will be much more complex than current IP nets (which are bad
   enough)  because of the difficulty in analyzing transient delay
   phenomena and tweaking the multitude of ATM network parameters.

 o ATM has some inherent inefficiencies.
	-The cell tax: breaking packets into 48+5 byte cells can result
	 in significant inefficiency, a particularly important issue
	 for expensive wide-area links.  Many Internet service providers
	 are looking forward to using a simpler and more efficient
	 approach, for example, PPP over SONET.
	-Segmentation/Reassembly overhead on end-systems can be
	 significant.
	-The one-to-many ATM multicast routing model is less efficient
	 than that used in the Internet MBONE, resulting in much larger
	 multicast routing tables in the ATM case.

 o ATM also has some potential advantages.
	-Partially offsetting the cell tax is the fact that each cell has
	 a small virtual circuit ID as opposed to a frame-based system
	 having two full IP addresses on every packet.  If you can send
	 large packets, the cell tax is still much worse, however.
	-There is an argument that a pure TCP/IP environment will not
	 scale to very high speeds because TCP's end-to-end congestion
	 control won't be responsive enough to the dynamic peak loads
	 in the Internet; the claim is that hop-by-hop congestion feedback
	 and control --as promised in *next* generation ATM switches-- is
	 going to be essential.  On the other hand, even if this proves to
	 be true in practice, TCP has a history of adaptation to new needs,
	 and alternatives such as RTP already happily coexist with TCP in
	 the existing IP infrastructure.

 o ATM products are improving, but still immature.
	-The ATM Forum standards are still evolving.
	-Only recently have ATM vendors started providing adequate
	 buffering to deal with delay-bandwidth product
	-Only recently have ATM vendors started providing solutions
	 to IP congestion collapse (e.g. Early Packet Discard).
	-ATM systems that incorporate hop-by-hop congestion control
	 for ABR service are reportedly not available until late 1996.

 o Successful high-capacity IP-over-ATM solutions are rare;
   successful examples of using ATM for data, voice, and video are even
   more rare.  (MCI's use of Fore ATM switches for InternetMCI gives
   credibility to the IP-over-ATM use; the fact that no telcos use ATM
   for voice undermines the credibility of the multimedia claim.)

 o Use of IP-oriented approaches to multimedia support, e.g. RTP, IGMP,
   RSVP, if successful, will avoid the ATM problems mentioned above.

 o Notwithstanding the enormous amounts of money being invested in ATM,
   the world is now based on IP, and unless IP-mixed-with-ATM gets a lot
   better, we would not expect ATM to dominate anytime soon.

 o Because of the inherent problems with IP-over-ATM, ATM dominance will
   only occur if either of the following propositions proves to be true
   over the next few years:
	-ATM is demonstrably superior to IP/RTP/RSVP for multimedia.
	-ATM price/performance is demonstrably superior to IP routing.

 o Ipsilon's hybrid approach is quite interesting.

 o Multimedia support is going to require enormous investments, regardless
   of which technology is chosen.

Summary: It is possible that ATM may become the best available choice for
a high-speed IP multimedia backbone, but because of its problems (some
inherent, some related to IP support) we would be much happier with a more
native IP approach.  However, neither approach to supporting multimedia is
mature or proven in high-volume networks.  Finally, we are very skeptical
that the vision of ATM domination (end-to-end, completely replacing IP)
will ever come about, so the question will be whether ATM offers
significant advantages as an alternative link-layer technology in an IP
environment.

INDUSTRY REPORT: June 1997

Here is a useful reality check from June 1997: "The shrinking world of ATM" by Jodi Cohen, published in Network World on 6/30/97.

The opening lines are "ATM, once considered the ultimate end-to-end enterprise network technology, is getting pushed out of the campus network picture.

ATM's world is shrinking because of the emergence of Gigabit Ethernet, frame-based quality-of-service (QoS) features and IP switching, according to a sampling of the 4,000 attendees at this week's ATM Year '97 show here."

INDUSTRY REPORT: June 1999

More recent press reports suggest that the jury is in. For example, consider Tim Wilson's 6/3/99 article in InternetWeek entitled: "Gig Ethernet Surpasses ATM As Preferred Backbone".

Not everyone is sold yet, but as time goes on the inevitability of Ethernet-uber-alles will become clear.

OTHER VOICES

Here are some comments from other folks I think have credibility on this subject. Of course, I don't mean to imply that there are not credible players who still believe that ATM is the future, but while ATM will not completely disappear right away, I think these comments are indicative of the current trends.

Javad Boroumand: Until recently, Javad was a program manager for NSF's networking initiatives. He got people's attention at a 1999 meeting when he announced that the National Science Foundation would no longer fund proposals that called for end-to-end ATM. He made it clear that this particular battle was over.

Irv Blythe: Irv is CIO of Virginia Tech, a university that has been on the leading edge of networking technology. He has made substantial investments in ATM over the years, and so it was particularly dramatic at the April 1999 Internet 2 meeting when, in his plenary address, Irv referred to ATM as a "legacy technology".

Milo Medin: Milo is CTO of @Home, the major cable data service provider in the U.S. Prior to joining @home, he led NASA's Internet efforts. At the same April '99 Internet 2 meeting, Milo referred to ATM as a "reason resistant" technology. Milo's basic argument is that complexity costs money, and in the long run the simplification resulting from eliminating the ATM layer will lead to irresistable cost savings. This is a notable observation, since Milo has plenty of experience building ATM-based national backbones.

Scott Brim: Scott was a senior network architect at Cornell University for many years, until joining Newbridge Networks a couple of years ago. (I understand that he's now with Cisco.) Cornell was one of the few universities that did believe in desktop ATM, and developed the "Cells-in-Frames" technique for trying to leverage Ethernet during a transition to ATM. Both of these initiatives have since been abandoned. I think the following comments Scott made to me in December 1997 are very significant, coming from someone who had been an extremely strong advocate for ATM... The jist was: "ATM has a useful life of two years, beginning a year from now." The "year from now" represented how long he thought it would take Microsoft to improve Windows enough that a low-latency network technology would make any difference. (For interactive network applications, OS latency is *still* a problem, by the way, regardless of which network technology is used.) The "useful life of two years" after that was based on how long Scott thought it would be before IP or frame-based QOS alternatives would become available, rendering ATM unnecessary.

Hank Zanini: Hank is a founder of Avici Systems, a terabit/sec IP router startup that Nortel recently invested in. At the Nov 1997 "Next Generation Networks" conference I asked him what kind of reception he got when he went to major telcos and tried to sell them on Avici's non-ATM approach to backbone networking. He told me that he had been very warmly received. In fact, he claimed that a senior official of a major telco told him "We are politically, religiously, emotionally, and technically committed to ATM... and this is a mistake." Of course, within every large telco it is likely that one will find both pro-ATM and anti-ATM partisans. ATM was, after all, invented by the telcos, and many of them have already committed billions of dollars to it. So even if ATM is not the *best* long-term choice in an Internet-centric world, it will not disappear in the wide-area for quite awhile.

John McQuillan: John is creator of the original "ATM" trade conferences who has always had a warm spot in his heart for ATM... so it was significant that he announced at his 1997 "Next Generation Networks" conference that he had sold the ATM conference... He also made it clear in his tutorial on the current state-of-the-art that he had become a convert to the view that ATM was not needed in a pure IP environment, given the new multiple-queue network silicon chips coming along from MMC semiconductors and others. (By the end of the week, after getting push-back from various ATM advocates, he had retrenched a bit, and emphasized that the two technologies would coexist for a long time, which I believe is true.)

SUMMARY

Here are my key points:

Why?

In all three cases it is because there are, or soon will be, simpler and cheaper alternatives that meet contemporary networking requirements. In the case of desktop and enterprise-wide connectivity, the simpler and cheaper alternative is Ethernet (Fast, Gigabit, and Ten Gigabit).

ATM equipment vendors should understand that ATM has become a transitional technology as more IP-centric approaches take hold. They should be preparing for the inevitable assimilation of voice and video communication into Internet packet-oriented technologies, such as IP over gigabit Ethernet.

All network equipment vendors should play a leadership role in helping the industry move to the most cost-effective data communication mechanisms possible, especially in the context of the Internet tidal wave. ATM is definitely not the answer for the future I see coming.


TEG HOME