nat'l laboratory for applied network research nsf-funded uc, san diego postal: ucsd mail code 0505 la jolla ca 92093 0505 (619) 534 8333 fax 5113 The NGI's Role in Facilitating Internet Data Acquisition and Analysis by k claffy, UCSD/NLANR/CAIDA --------------------------------------------------------------- Introduction Discussion Recommendations for the NGI Related URLs / Papers --------------------------------------------------------------- Introduction The U.S. government has played an important historic role in defining and nurturing the development of the global Internet. Since the transition of infrastructure stewardship to the commercial sector in early 1995, numerous user communities have called upon federal agencies to resume active involvement in next generation internetworking. This paper supports the position that the government has a limited, but critical role in facilitating development of advanced networking technologies and protocols that are beyond the realm of what is currently commercially viable or beyond the scope of individual service providers. We suggest that the government foster the acquisition and analysis of traffic data associated with individual and interconnecting network performance and data, focused on developing insights into general infrastructure-wide traffic patterns and behavior. Engineering Models and methodologies developed from such data are fundamental to the continued development of a scalable, robust global network. We also encourage the government to leverage its pivotal role as a major Internet customer, providing leadership in implementing practices and technologies that encourage development of sustainable business models. --------------------------------------------------------------- Discussion Most large Internet service providers currently collect basic statistics on performance of their own infrastructure, typically including measurements of utilization, availability, and possibly rudimentary assessments of delay and throughput. In today's commercial Internet, the only baseline against which these networks can evaluate performance is their past metrics. 1/ No data or even standard formats are available against which to compare performance with other networks or against any baseline. Nor are there reliable data with which users can assess or compare the performance of providers. Data characterization and traffic flow analysis are also virtually non-existent at this time, yet they remain essential for understanding the internal dynamics of the Internet infrastructure. Given the dynamic nature of the Internet environment, collected traffic data will be of primarily historical interest unless we tangibly improve our ability to analyze and predict network behavior. Without the necessary and fundamental understanding that traffic modeling and simulation offers, practitioners will continue their skepticism toward empirical measurement studies, which are largely of no realistic help in instrumenting large Internet backbones. Yet there is still no consensus on how to model IP traffic or incorporate real time statistics into such analyses. Telephony models developed at Bell Labs and elsewhere rely on queuing theories and other techniques that are not readily replicable to Internet style packet-switched networks. While gaps persist, the mutual inter-dependence of the research, provider and end-user communities and the growing requirements to assess Internet performance suggest a strong need to identify common ground. Toward this end, NLANR/UCSD is creating the Cooperative Association for Internet Data Analysis (CAIDA). CAIDA is a collaborative undertaking to promote greater cooperation in the engineering and maintenance of a robust, scalable global Internet infrastructure. It will address problems of Internet traffic measurement and performance and of inter-provider communication and cooperation within the Internet service industry. 2/ While CAIDA is industry-focused, it is receiving its initial seed funding as a project of the NSF-supported National Laboratory for Applied Network Research (NLANR) within the University of California, San Diego. Complementing an industry-wide effort with government support in this manner will promote balance among the needs of the various communities (private, research, government, and users) and facilitate the near term development and deployment of critical measurement technology and techniques. CAIDA's initial goals include: * working with industry, consumer, regulatory, and other representatives to assure their utility and universal acceptance of Internet performance metrics; * creating a collaborative research environment in which commercial providers can share performance and engineering data confidentially, or in desensitized form; and * fostering the development of advanced networking technologies such as... - traffic performance and flow characterization tools and analysis - bandwidth reservation and related QoS - traffic visualizations, simulations and analyses - BGP instability diagnosis - mechanisms for responding/adapting to improper end host flow control/congestion response mechanisms (e.g., misbehaving TCPs, UDP) - multicast and the Mbone - web caching protocols/hierarchies - "next generation" protocols/technologies, e.g. IPv6 --------------------------------------------------------------- Recommendations for the NGI As the NGI initiative is further refined and implemented, issues relating to measurement and performance will be paramount. As part of this process, NGI participants should strive to: * partner with industry to define priorities and implement NGI efforts; * encourage, through project funding, collaborations or other means,... - new and proposed Internet infrastructure measurement initiatives, such as those sponsored by UCSD/NLANR, PSC/LBL, CSG, Terena and others; - development and deployment of both end-to-end performance tools and traffic flow characterization monitors by organizations receiving federal support, including at Internet-2 and other NGI sites; - data analysis, simulation, and visualization efforts -- with a focus on support / endorsement by industry; * systematically make publically available details on performance, traffic workloads, and related analyses on federally-supported networks and exchange points; and * develop model service agreements and practices for federal agencies, incorporating service level specifications and monitoring/reporting requirements into contracts with commercial Internet service providers. --------------------------------------------------------------- Related URLs / Papers www.caida.org www.nlanr.net/INFO Internet Data Acquisition & Analysis: Status and Next Steps, Tracie Monk and k claffy (UCSD/NLANR/CAIDA), INET'97, Kuala Lumpur, June 1997. Cooperation in Internet Data Acquisition and Analysis, Tracie Monk (DynCorp) and k claffy (UCSD/NLANR), Harvard Workshop on Coordination and Administration of the Internet, Boston, MA, September 1996. Towards a Framework for Defining Internet Performance Metrics, Vern Paxson (DOE/LBL), INET'96, Montreal, CA, June 1996. --------------------------------------------------------------- 1/ Metrics of delay, packet loss, flow capacity, and routing behavior are fundamental to measurement and comparability of path and network performance. The Internet Engineering Task Force (IETF) IPPM working group is working to provide a more rigorous theoretical framework and guidelines for designing measurement tools robust to the wide variety of disparate signal sources in the frictionful Internet. Tools supporting measurement of such metrics are emerging and include efforts to measure both TCP goodput, dynamics indicative of misbehaving TCP implementations, and end-to-end delay distributions. The community is still only at a very rudimentary stage with respect to tools that can isolate traffic bottlenecks and congestion points and visualize traffic flows. Merit has taken extensive measurements of routing instabilities at the original NSF-chartered NAPs, but the underlying meaning and implications of these measurements has received little attention from the community. Several organizations are also initiating infrastructure performance measurement efforts in early 1997, including PSC and LBL, the Common Solutions Group, and Europe's Terena. 2/ In May 1997, NLANR/CAIDA will host its second Internet Statistic and Metrics Analysis (ISMA) workshops, www.nlanr.net/ISMA. During 1997, NLANR/CAIDA will work with participating companies to define the goals, priorities, and desired membership of CAIDA.