Steve Batsell, Network Research Group Leader MS 6367, Bldg 6012, Oak Ridge National Laboratory, Oak Ridge, TN 37831 423-574-7401 (phone) 423-574-0680 (Fax) sgb@ornl.gov http://www.epm.ornl.gov/~sgb/net.html Helen Chen, Senior Member of Technical Staff Sandia National Laboratories, Livermore, CA 94551-0969 510-294-2991 (phone) 510-294-1225 (fax) hycsw@ca.sandia.gov ---------------------------------------------------------------------------- Performance Measurements and Analysis of IP-over-ATM Architectures on the ESnet and ACTS ATM Networks Steve Batsell, Tom Dunigan, and Lawrence MacIntyre {sgb,dunigan,lpz}@ornl.gov Oak Ridge National Laboratory Helen Chen and Bruce A. Mah {hycsw,bmah}@ca.sandia.gov Sandia National Laboratories, CA Background Over the past decade, the Internet has grown to include approximately five million hosts on over 45,000 interconnected networks, on all seven continents and in outer space. This unprecedented growth, together with the introduction of multimedia workstations, has enabled the development of innovative applications that require high speed, low latency, and real-time transport. Today's Internet can neither scale in its bandwidth nor guarantee the Quality of Service (QoS) necessary to meet these performance requirements. Many network researchers propose to use Asynchronous Transfer Mode (ATM) technology as the underlying infrastructure for the next generation of Internet with the support for Integrated Internet Services. However, ATM is significantly different from existing networking technologies: 1) ATM is connection- oriented, whereas IP uses a connectionless paradigm, and 2) ATM networks lack an analog to the multicast and broadcast capabilities that are inherent to shared-medium technologies. Therefore, a simple and efficient integration of IP and ATM is especially challenging and as a result, there are many IP-over-ATM architectures under study [Chen97] [Woundy96]. Both the IETF and the ATM Forum propose an overlay architecture (Classical IP over ATM and Next Hop Resolution Protocol from the IETF, and LAN Emulation and Multiple Protocol over ATM from the ATM Forum) which require duplicate address spaces and routing protocols, hiding ATM's physical topology from the higher IP layer. Using a server-based mechanism and complex protocols, this architecture models a logical Non-Broadcast Multiple-Access (NBMA) medium. This approach presents implementation complexities, as well as possible performance bottlenecks and single points of failure at the servers. Three proprietary IP-over-ATM solutions (IP Switching from Ipsilon, Tag Switching from Cisco, and Aggregate Route-Based IP Switching from IBM) model the ATM infrastructure as point-to-point networks. Since today's protocols already work well with point-to-point networks, they will continue to work over these proprietary solutions without any modifications. By running standard routing packages in the ATM switches, this approach preserves IP's connectionless paradigm while potentially achieving ATM's hardware speed. Moreover, in most cases, the need for the complex ATM signaling and routing functions are eliminated. Because of switch forwarding table limitations, however, some of these architectures may have scalability problems in large Internet environments. Research Testbed Due to the complexity in IP-over-ATM integration, we believe that a research testbed is necessary to test and verify prototype IP-over-ATM equipment, during their research and development cycles. We propose to construct a research testbed by connecting Oak Ridge National Laboratory (ORNL) and Sandia National Laboratories (SNL) at California using a 45-Mbps Permanent Virtual Path (PVP) over the Energy Science (ESNET) as well as the ACTS ATM networks. This configuration, in effect, provides a logical T3 trunk for our evaluation, but will otherwise consume zero bandwidth in the absence of experimental traffic. With increased latency and error rates over the wide-area terrestrial and satellite links, this testbed can reveal potential performance issues inherent in these IP-over-ATM solutions. In addition, they can stress the underlying ATM technologies, such as ATM signaling, Private Network to Network Interface (P-NNI) routing protocol, ATM-payload encryption, as well as Available Bit Rate (ABR) and Early Packet Discard (EPD) congestion control algorithms. In particular, we are interested in the effect of delayed feedback to the efficiency of the rate-based, closed-loop ABR flow control mechanism. Due to limitations in quantity and availability of prototype equipment, some of our performance evaluation under heavy load may rely on simulation studies, as in [Billhartz97] and [Fang94]. Measurement and Analysis We define an IP flow as the base unit for establishing an ATM cut-through path. Because ATM connection setups consume significant amount of network resources, we will experiment with varying IP flow granularity of application-pairs, host-pairs, and network-pairs. Within each flow categories, the granularity can be further refined with respect to different QoS requirements. The effects of flow granularity on throughput, response time, and delay jitter will be measured and analyzed. Furthermore, we will develop an Application Programming Interface (API) which will allow an application to make Quality of Service requests through the Integrated Internet Services. In particular, the mechanism of mapping application's realtime requests via RSVP, ATM's Q.2931, and P-NNI to ATM's QoS will be investigated [Bord95]. The next generation Internet protocol, IPv6, provides flow identifier and priority fields which can be used to identify a realtime flow to receive the pre-reserved network resources. We will compare the realtime performance of the alternative IP-over-ATM transports using the distributed applications now being developed for distributed teraflop as well as cluster computing between ORNL and Sandia. While existing benchmarks can be used to evaluate effective throughputs and delays along network paths, these measurements may be difficult to translate into metrics that are meaningful at the application layer. By examining and analyzing packet traces of actual network traffic, it is possible to construct mathematical models of Internet applications and their network activity. A benchmarking tool or network simulator can use these models to mimic the behavior of real applications. The resulting network traffic, with statistical properties similar to the original trace, can then be used to drive an actual or simulated network. For example, [Mah97] describes such a model for the HyperText Transfer Protocol used by World Wide Web clients. We believe that such techniques can also extend to other types of applications such as multicast, multimedia conferencing or distributed, parallel computing. With traffic sources that are aware of the demands of specific Internet applications, it is possible to measure performance in terms of those same applications. For example, a tool generating a stream of Web requests can measure and record the response time of a single request, or of a set of requests making up a Web page. This approach yields much more concrete results than more traditional network benchmarks, because it takes into account the performance requirements of individual types of applications. Moreover, it can be used in concert with other measurement techniques, such as kernel statistics, which are necessary to determine details of any undesirable interactions between TCP and ATM flow control. References [Billhartz97] T. Billhartz, J. Cain, E. Farrey-Goudreau, D. Fieg, and S. Batsell, Performance and Resource Cost Comparisons of CBT and PIM Multicasting for DIS Environment, IEEE Journal of Selected Areas in Communications, April 1997, to appear. [Bord95] M. Borden, E. Crawley, B. Davie, and S. Batsell, Integration of Real-time Services in an IP-ATM Network Architecture, IETF RFC 1821, Available from ftp://ds.Internic.net/rfc, August 1995. [Chen97] Helen Chen, Rose Tsang, Jim Brandt. and Jim Hutchins, A Survey of IP-over-ATM Architectures, ConneXions, to appear. Available from ftp://ca.sandia.gov/pub/papers/IPoverATM.ps. [Fang94] Chien Fang, Helen Chen, and Jim Hutchins. A Simulation Study of TCP Performance in ATM Networks, Proceedings of IEEE GLOBECOM '94, San Francisco, CA, November 1994, pp. 1217-1223. Available from ftp://ca.sandia.gov/pub/papers/globecom94.ps. [Mah97] Bruce A. Mah. An Empirical Model of HTTP Traffic, Proceedings of INFOCOM '97, Kobe, Japan, April 1997. Available from http://www.ca.sandia.gov/~bmah/Papers/Http-infocom-release.ps. [Woundy96] Richard Woundy, Arun Viswanathan, Nancy Feldman, Rick Boivie, ARIS: Aggregate Route-Based IP Switching, IETF draft , November 1996. Available from http://ds.internic.net/internet-drafts.