NGI White Paper Research Directions for the Next Generation Internet: Considerations for a Switched WAN Backbone with Congestion Control and Higher Bandwidth Stephen Tenbrink, Don Tolmie, Mike Boorman Dave DuBois, Gene Dornhoff, Andy DuBois Richard Thomsen, Ian Philp Network Engineering, Group CIC-5 Los Alamos National Laboratory Contact: Stephen Tenbrink Network Engineering, Group CIC-5 PO Box 1663, M/S B255 Los Alamos National Laboratory Los Alamos, NM 87545 Phone: 505-667-4935 FAX: 505-665-7793 email: sct@lanl.gov ----------------------------------------------------------------------- Considerations for a Switched WAN Backbone with Congestion Control and Higher Bandwidth Abstract: This white paper addresses the benefits for utilizing better flow control in switching the Next Generation Internet (NGI) backbone to avoid congestion and speed up data transfers utilizing multi-gigabit/sec ports on such switches. The obvious growth path for Internet networking today is via ATM in the wide area network (WAN). One potential problem with ATM, however, is its use of rate based flow control which can cause loss of data on switches that are heavily congested. Another concern is the availability of OC-192 ATM switches in the foreseeable future. The high-end supercomputer environment has for years dealt with the increasing demand for bandwidth for super computer applications by the development of gigabit networks, in particular the HIPPI, Fiber Channel, and SCI technologies. While these technologies usually address local area network (LAN) requirements, some aspects have been applied to WANs as well and may be useful to the development of the next generation Internet. Background: In 1987, network researchers at Los Alamos National Laboratory (LANL) developed the first gigabit network link proposal and submitted it to the American National Standard Institute as a candidate for standardization. This eventually became the High Speed Parallel Interface (HIPPI-800) ANSI Standard (1991) for delivering data at 0.8 gigabits/sec. In 1992, as part of the CASA Gigabit Testbed, LANL developed a HIPPI/SONET Gateway for transmitting data at HIPPI rates over a striped OC-3c SONET infrastructure. In 1995, LANL, along with Silicon Graphics Inc, and other commercial computer manufactures, started the development of a new ANSI standard for the next generation HIPPI called HIPPI-6400 for delivering user data rates of 6.4 gigabits/sec. Similar developments for the Fiber Channel Standard (FCS) were done at Livermore National Laboratory during the same time frame. NGI Technology Thrust: One of the chief concerns with the Internet today is congestion, especially at the routers. The advent of a new Internet infrastructure certainly implies higher bandwidth but should also deal with performance issues as well. Performance on the Internet today is severely affected by congestion at the router ports causing, in some cases, buffer overflow. When this happens, the TCP packet must be retransmitted which lowers effective performance. If the incoming router ports from the Internet had a more robust flow control at the physical layer, data buffers would no longer overflow and effective performance would improve. What is needed is a flow control that insures the buffers will never overflow. While there are certain benefits to rate based flow control in the WAN, especially to avoid the problem of back pressure extending to a "gridlock" stage, the judicial use of credit based flow control might benefit the overall performance of the WAN for NGI. The ANSI standard HIPPI-800 and the evolving HIPPI-6400 specifications deal directly with congestion by using credit based flow control, which is not available in ATM. HIPPI-800 and HIPPI-6400 were not designed for wide area networks (WANs) due to their high speed and cable limitations, but it has been shown, via the HIPPI/SONET Gateway in the CASA Gigabit Testbed, that HIPPI data can be sent over long distances (2000 Km) at the full 800 megabit/sec data rate. With the advent of HIPPI-6400 and its more robust error control, small fixed-size micro-packets (equivalent to cells in ATM) and use of multiple virtual channels, it should be possible to build a regionally switched backbone for the NGI that addresses both the bandwidth and congestion issues. The CASA Gigabit Testbed was a successful effort to interconnect supercomputers between California and New Mexico at the full 800 megabit/sec HIPPI raw data rate. Since HIPPI switches are readily available that support a rigid flow control, a scheme of switching SONET links carrying HIPPI on multiple OC-3c channels is possible today that will provide credit based flow control. With the growing concern of congestion on the Internet, such a flow control could be useful when deployed in a regional switched backbone. It may not be feasible to have a globally (or nationally) switched backbone since back pressure from heavily used destinations may be difficult to control. However, deploying a cluster of regionally switched backbones that are interconnected with high-performance routers may provide a simple solution to the congestion problem. HIPPI-6400 is an upgrade path for HIPPI-800 that is being implemented with features not found in the older HIPPI-800 ANSI specification. Some of the benefits that HIPPI-6400 brings to switched backbone are: 1. Increased bandwidth (6.4 gigabits/sec) 2. Virtual channels that enhance congestion avoidance 3. A possible solution to switching OC-192 SONET by carrying HIPPI-6400 on a OC-192 payload and using HIPPI-6400 switches at the switching nodes. Based on the success of the original HIPPI/SONET Gateway, a new gateway for HIPPI-6400 to SONET OC-192 based on striping OC-48c streams is possible. Another component that would be required is a router that could handle HIPPI-6400 ports and route to legacy 100baseT, FDDI, ATM, and even Gigabit Ethernet networks. Precedence for such a router has been set by the NetStar GigaRouter with its HIPPI-800, ATM, and FDDI ports and the DARPA funded router development at BBN. If such networks are to grow with the expected demand for service that is prevalent today, the issue of scaling such networks is also important. SONET and ATM have been specifically designed for scaleability, but other switched network technologies can also scale. If credit-based flow control and other features found in HIPPI are found to be beneficial, a new WAN standard based on the best parts of HIPPI-6400 and ATM could be developed using SONET as the physical transport layer. Striping across multiple SONET links to produce a higher bandwidth channel (e.g., the HIPPI/SONET Gateway) is another way to scale network bandwidth and can be economically desirable since bandwidth can be added incrementally using less costly lower speed links. Finally, this paper is not suggesting that HIPPI is the best choice for a switched backbone in the NGI although it is a technology that is available for such use today. The real thrust is to consider switching segments of the backbone with a better flow control than ATM. Some hybrid of HIPPI and ATM may be a better choice than either. There are advantages to both types of flow control and the possible use of dynamic credit assignment or converting between credit based flow control and ABR flow control should be addressed in research areas for the NGI. Further information on HIPPI-800, HIPPI-6400, and other network technologies can be found at http://www.cic-5.lanl.gov/lanp/lanp.html . ************************************************** * Stephen Tenbrink email: sct@lanl.gov * * Los Alamos National Lab Phone: 505-667-4935 * * PO Box 1663, MS B255 Fax: 505-665-7793 * * Los Alamos, NM 87545 * **************************************************