Internet traffic engineering has several characteristics that set it apart from other parts of the NGI initiative:
Internet "operations research" is perceived as neither, and thus does not receive intellectual or fiscal attention from either providers or research funding agencies. After more than two decades of research, there is nothing approximating a methodology or set of (intellectual or software) tools for characterizing Internet workload or performance, much less engineering a large-scale, multiprovider Internet infrastructure. While commercial ISPs continue to run their ever-thicker clouds by guesswork and intuition, they would welcome a set of tools that would facilitate their interaction with the rest of the Internet. There is no indication that such tools will emerge on their own in industry. On the contrary, competitive market pressures and antitrust sentiments actually create disincentives for ISPs and vendors to cooperate and collaborate to develop the necessary tools. This is a research area where the government can play the best of all possible roles. It can leverage modest funding to sanction or encourage industry to invest more funding to achieve cooperative ends.
ISPs struggle to keep pace with a workload that is increasing in several dimensions: the number of end-users, bandwidth per user, the range of new and future applications, the demand from customers for stronger predictability, and the internal need for greater efficiency as raw fiber capacity is squeezed to its limits.
To encourage commercial collaboration with NGI-funded research efforts (and group members contend that government funding would be ill-advised without such commercial collaboration), research objectives must be applicable to the actual, evolving Internet. In particular, traffic engineering research must create the means for making the future ubiquitous Internet efficient and robust for the next generation of transport technology, application requirements and constituents.
For the purposes of this report, traffic engineering will be discussed in terms of two principal components: 1) measurement (how do we know, what do we know), and 2) network modification and evolution in response to measurement (what do we do with what we know).
Group members identified two underlying principles that are expanded in the next section:
The roles of traffic engineering that jointly support the effective, cost-efficient provision and expansion of Internet service include:
To perform these roles, Internet providers, network managers and the research community must have consistent, accurate measures of various metrics of Internet performance and traffic. These measures cover not only the properties of individual flows and paths, such as throughput, loss, delay and jitter, but also broader scaling properties such as availability, reliability, fault resilience and resistance to intrusion or attack.
The measurement of performance and traffic serves multiple roles beyond conventional traffic engineering in supporting the service received by applications and users. It is important to keep this in mind when selecting and designing specific metrics and the procedures for applying them. Examples of these additional roles include: provider rating, provider and access selection, pricing, advertising, contract definition and enforcement, quality-based routing, and research on the adaptation of network behavior applications to network performance.
The dissemination of measurement information is as vital as its acquisition. An infrastructure is needed to get the right information to the right places, such as operations centers, network planners, adaptive applications and end-users. Despite the wide scope of the measurements needed, and the equally wide variety and location of uses (with widely varying timeliness constraints), it is essential that the information be gathered and distributed in a way that ensures minimal impact on the network and the applications.
NGI-sponsored research in traffic engineering should espouse certain overriding principles:
To realize this goal in the face of unanticipated future technologies, applications and user needs, the following requirements must be met:
--Tools and data formats must be extensible.
--Data selection and analysis must be integrated with routing, configuration and policy specification.
--Traffic measurement and analysis should be coordinated across the current Internet and the networks described in Goal 1 of the NGI initiative.
Guiding principles for the two major elements of traffic engineeringÑmeasurement and engineeringÑfollow.
To engineer the network, its behavior must be observed; measurement techniques must meet realistic goals:
To realize these goals, a measurement infrastructure is needed that supports:
To be practical and widely usable, tools must have available a uniform, standard, basic set of measurements and a means of parsing and interpreting them.
The current Internet environment is notable for its inability to perform in a consistent, stable and reliable manner. While the details of the future environment are impossible to predict, researchers must anticipate severe demands and steer NGI-developed traffic engineering to meet them. To this end, research must explore the boundaries of application behavior: new applications will deploy whether or not they are anticipated. The measurement infrastructure must be able to detect the effects of new applications and technologies; network alteration and evolution must be able to respond flexibly to the observed effects.
Some recent examples of rapidly expanding traffic demands that raise complex issues and are likely to have consequences for traffic engineering include:
-- highly diverse and dispersed.
-- alternative technologies are simultaneously significant, due to differing modes of use:
-- interactive vs. passive end-user involvement
-- smart peripherals vs. smart servers
-- rapid evolution of adaptive applications that attempt to measure and respond to network and server performance. For example, tens of millions of people who use interactive games already affect packet size distributions and traffic focus; providers must be able to adapt their networks to the resulting traffic.
The research questions listed below address specific concerns and opportunities for building the measurement and engineering foundation for the NGI. What should we observe in order to evolve access and transport technologies? How do we model, plan, provision and manage the NGI in a stable, consistent fashion, despite the evolution of these underlying technologies?
Traffic engineering must be concerned with a broad definition of service that includes network availability, reliability and stability, as well as traditional traffic data on loss, throughput, delay and jitter. Questions to be considered include:
Analysis is the art and science of interpreting measurements; developing the discipline of analysis is a significant research objective. The following points are relevant:
Data interpretation would be expedited by the development of tools or libraries to parse and interpret the standard data obtained, which requires use of a consistent format for data across all network components. While some of this work is more pragmatic than research, it will be essential to accelerating the widespread use of and benefit from the NGI traffic research. Gaps like this must be filled.
Tools are needed to determine what action to take based on traffic behavior, and to enable networks to recover from failures quickly, stably and automatically. Requirements include:
Over the longer term, the following tools are needed:
If the NGI initiative is to achieve its performance goals and foster the widespread availability of a cost-effective, high-performance information infrastructure for the national commercial, education and research communities, a few critical requirements are essential:
5.5.1 The NGI, and other government-sponsored testbeds, must be open to appropriate traffic measurement and performance monitoring by the research and user communities. They must be instrumented to support the goals of NGI traffic engineering research.
5.5.2 A basic set of standardized, unambiguous performance metrics must be defined that can support objective study and comparisons across networks throughout the Internet. To allow the evolution of increasingly accurate, low-impact techniques, the metrics themselves should not require a specific tool or procedure. However, each metric must have at least one feasible measurement technique.
5.5.3 The NGI initiative must promote the use and availability of such metrics and tools throughout the existing Internet, as well as within the networks defined in Goal 1 of the NGI initiative. Wide-scale experimentation is needed, with consistent study of what measurements matter in current and next generation environments. In addition to a common framework, NGI tools and techniques will require testing and verification using legitimate core backbone data from multiple providers and infrastructure components.
5.5.4 To allow consistent analysis over a long time frame, as well as comparisons of performance and traffic behavior, the measurements taken must be preserved in archives available to the research and user communities, subject to as few privacy and nondisclosure constraints as necessary. The NGI must support the development of:
The research will follow a spiral evolution: only by implementing prototype architectures for measurement, analysis and near- and long-term responses can the experience be gained to improve each of these areas to better serve the othersÑand the overall goals of the NGI initiative.
The research goal of Internet traffic engineering is to foster competition for reliable, available, efficient, high-quality service. A common structure of measurement and tools is needed to maintain international leadership and critical support for NGI research, as well as commercial Internet activities.
5.6.1 As the Internet becomes more essential to our national economic and educational competitiveness, the requirements for performance and reliability increasingly will outstrip the current Internet's abilities.
5.6.2 No amount of overcapacity alone can protect against failure modes that will become increasingly visible in such an environment, including network failures, traffic variability, feedback among adaptive applications, configuration errors and unintended interactions, and possible intrusion or attack. Thus, essential network reliability cannot be achieved without significantly expanding the data, tools and methods available for traffic engineering.
5.6.3 A networkÕs cost-effectiveness can only be assured by planning, provisioning and day- to-day traffic management, based on consistent objective knowledge of the networkÕs behavior and the traffic offered.
5.7.1 Historically, the definition of standards essential to commerce has required the active support of governments.
5.7.2 In today's highly competitive, free-wheeling commercial Internet, individual providers have little economic incentive to devote resources to developing consistent, universal standards.
5.7.3 Individual providers face strong marketing pressures to avoid exposing failures unilaterally.
5.7.4 Although there are independent enclaves of traffic engineering research, there is no motivation for coordinated study. The rapid and sometimes chaotic growth in the heterogeneous Internet requires objective, neutral-party solutions beyond the scope of any single provider or consortium of providers.
5.7.5 Many potential solutions, such as dynamic rerouting and load-balancing, could qualitatively improve the performance, efficiency and reliability of the Internet, but require longer-term research to create the underlying understanding and techniques. The industry, which is rapidly expanding and fragmented, cannot currently address these longer-term payoffs.
In the past, a trade-off has been made in precluding a government role in overall Internet architecture in favor of not having a single entity to coordinate and support long-term research. And, as it typically tends to do, competitiveness has reduced the chances for cooperation, common research, and consistent measurement and action.
With comparatively modest initial funding, the NGI initiative also provides the government with the opportunity to seed a prototype measurement infrastructure that can then harness market forces to expand its scope. Once such a prototype can demonstrate its benefits to a critical mass of providers, organizations and end-users, other providers will have greater incentives to contribute themselves.