Effective Communications on the Next Generation Internet John Naegle and Steve Gossage {jhnaegl,sagossa}@sandia.gov Sandia National Laboratories Effective communication at high speeds on the next generation Internet will require much more than just high-speed links across the country. The following is a list of several features that must be present for the next generation Internet (NGI) to be a success: 1. High-speed and flexible long distance links. 2. Applications that can actually use fat pipes with high delay. 3. Protocol tuning and host modification to enable the above applications. 4. High speed local infrastructures that efficiently integrate with the long distance links. 5. Methods to allow secure interconnection. Sandia National Laboratories has been working intently on each of these areas for the last 7 years. In 1990, budget pressures forced Sandia to consolidate its supercomputing facilities to a single location in Albuquerque, New Mexico. A requirement for this consolidation was to create a communications and computing environment for the Livermore, California supercomputing users that had the same perceived network performance as when the supercomputers were located at their local campus. In order to achieve this requirement, research and improvements in all of the above mentioned areas were necessary. This project was successfully completed in May of ‘93. Since then, Sandia has continued working to scale all of these areas to build balanced production systems. The network infrastructure in place at Sandia models one of the most promising architectures for the NGI. This discussion will present Sandia’s networking environment and its relevance to the NGI. Sandia has partnered with many different long distance carriers and service providers to demonstrate and deploy long distance communications links that are both high-speed and flexible. The initial supercomputer consolidation link used the first two DS-3 links deployed by FTS2000. Sandia demonstrated the first long distance ATM OC-3c communications link outside of the RBOCs at the ‘93 Supercomputing Conference. Several different applications were demonstrated, such as high-speed data transfer, desktop video conferencing, and distance learning. The emerging cell switching technologies were used on both of these networks. Native ATM near broadcast quality video was also demonstrated from Albuquerque to Pittsburgh at the ‘95 Supercomputing Conference. ES Net, Sandia, and Oak Ridge National Laboratories have demonstrated high-speed long- distance communications between massively parallel computers. Since the early days of supercomputer consolidation, Sandia has deployed much more flexible links that have allowed even more consolidation. There is now an AT&T Globeview 2000 ATM switch in Albuquerque with a single DS-3 link to a Fore BX-200 ATM switch in Livermore. This single link is passing all of the different type of traffic that previously required several dedicated links. For TCP/IP data traffic, there are several different router interconnects as well as native ATM hosts sharing part of the DS-3 bandwidth. We also have demonstrated and will soon have production T1s using Circuit Emulations for part of the bandwidth. These circuits will be used to trunk our 5ESS switches and for PictureTel video conferencing equipment. Another piece of the bandwidth will soon be dedicated to a native ATM video conferencing "video-hallway" application for collaboration between the networking groups at the two sites. Sandia now has the flexibility to easily use the bandwidth for whatever type of service is currently required. The routers are used for low-bandwidth, short connect-time applications such as E-Mail and HTML where conectionless networking is most efficient. Native ATM can be used for high-speed data connections for large file transfers where connection setup time and volume is insignificant. Bandwidth can be dedicated for Circuit Emulation to handle our various dedicated T1 communications needs. All of this is managed through software provisioning rather than dealing with physically provisioning circuits each time our needs change. This is a model that could work very well for the NGI. All of the high-speed interconnects are of little use if applications can not use the available bandwidth. Although aggregation of many low-speed connections is a difficult problem, one of the goals of the NGI should be to enable single applications to use large amounts of bandwidth. Sandia has several examples of such applications. Our partnerships with the Oil and Gas industry require processing of huge amounts of seismological data. Interconnecting supercomputers to work on problems that are beyond the capabilities of a single machine will help achieve the ASCII goals. Sandia received an R&D 100 award for developing an ATM high-speed interface in the Intel Paragon that will enable such collaboration. Sandia also moves a vast amount of satellite data. High-quality video applications also require much larger bandwidths than typically available today. All of these applications require a tremendous amount of bandwidth between single end points, not just aggregate bandwidth. The current Internet can not meet the requirements for these applications. Sandia needs the NGI to effectively meet these requirements. Most applications can not effectively utilize high-speed and high-delay communications lines. As part of the supercomputer consolidation project, Sandia demonstrated the protocol tuning and host modifications that were necessary to utilize the high-speed, 1100 mile links. Research efforts have continued to demonstrate the maximum performance achievable with standards based equipment. Although proprietary solutions are capable of much higher local network performance, they are useless in a large-scale environment, such as the Internet, where standards are required. Sandia has deployed end hosts that achieve TCP/IP (Not UDP) data transfers of 500 Mbs. Even NT based systems, with proper tuning, are now capable of TCP/IP transfers of 120 Mbs. To utilize these kinds of application bandwidths over the NGI, the local network infrastructure must also be vastly improved. Sandia has invested heavily in its communications infrastructure. The Livermore site has multimode fiber to all 2000 desktops, and the Albuquerque site has almost 10.6K desktops with multimode fibers. Several multiplex systems, including an OC-48 ring, have been deployed to efficiently utilize the campus single mode backbone while allowing very high speed communications across the campus. A fully OC-3 ATM meshed, 60 router backbone has several Gigabits of bandwidth capacity. By mid-summer ‘97, over 150 native ATM and over 300 switched Ethernet connected hosts will be served by a campus wide native ATM and LAN Emulation network. This infrastructure will allow Sandia to bring the high-speed NGI directly to users’ desktops. New security mechanisms will need to be developed to reach the desktops at the data rates promised by the NGI. Sandia was one of the initial founders of the Security working group in the ATM Forum, whose task is to develop secure signaling extensions for authenticating connections. Sandia has also received two R&D 100 awards for scaleable, reliable encryption development. These efforts, coupled with our experience building production classified networks for DOE, has given Sandia a distinguished international reputation for network security. Since early 1990, Sandia has been committed to the concept of effectively utilizing high-speed long-distance secure communications. This commitment extends beyond just data networking but also to the full spectrum of voice, data, and visual communications requirements. All of these efforts have generated real user requirements for, and have given Sandia an in-depth perspective of the capabilities required from the next generation Internet. Thanks, ------------------------------------- John Naegle, Sandia National Laboratories E-mail: jhnaegl@sandia.gov Phone: (505) 844-8044 Date: 3/27/97 Time: 11:58:03 AM --------------------------------------