Policy Architecture in Large Networks Paul Ferguson Consulting Engineer Cisco Systems, Inc. 400 Herndon Parkway Herndon, Virginia 20170 tel: +1.703.397.5938 fax: +1.703.397.5999 e-mail: pferguson@cisco.com [text] Introduction Scaling a large network is a principal concern, especially at high speeds, and an issue which needs to be carefully examined in the conceptual design and planning stages. Striking the right hierarchical balance can make the difference between a high-performance network and one which on the verge of congestion collapse. This paper discusses the architectural concepts of applying policy control in a large network to minimize the performance overhead associated different mechanisms that provide traffic and route filtering, bandwidth control (rate-limiting), routing policy, congestion management, and differentiated classes of service (CoS). Architectural Concepts The key to scaling very large networks is to maintain strict levels of hierarchy, commonly referred to as "core, distribution, and access" levels, which limits the degree of meshing between nodes. The "core" portion of the hierarchy is generally considered the central portion, or backbone, of the network; the "access" level represents the outer-most portion of the hierarchy, and the "distribution" level represents the aggregation points and transit portions which lie between the core and access levels of the hierarchy [Figure 1]. core --- core ---- core / | \ / | \ distribution distribution distribution /\ /\ /\ / \ / \ / \ / \ / \ / \ access access access access access access /|\ /|\ /|\ /|\ /|\ /|\ / | \ / | \ / | \ / | \ / | \ / | \ Figure 1. A high degree of meshing affects the ability to maintain stability in the routing system and degrades overall performance as the network grows larger. A distinction should be made between full-meshing and partial meshing; it stands to reason that partially-meshed networks scale better than fully-meshed networks, and for redundancy considerations, a partially-meshed network has acceptable scaling properties (in most instances). It is important to understand, however, the impact of applying policy at various locations with the network hierarchy. Depending upon where these policies are applied, it can have varying effects on the performance of the network. It is commonly accepted that the network core should never become the point of network congestion, but in reality, this is not always the case. The purpose of the network core is to forward traffic as quickly as possible to it's destination, therefore any type of policy which may affect forwarding performance should not be implemented in the network core. Types of policies which may fall into this category are traffic and route filtering. It is worthwhile to consider "pushing" these types of policy implementations to the lower portions of the hierarchy to avoid performance degradations which affect the entire network, instead of a smaller subset of the overall topology. If traffic congestion is a concern in the distribution and core levels of the network hierarchy, then one should consider the implementation of Random Early Detection (RED) as a congestion management mechanism in these portions of the network topology [1]. Implementation of policy lower in the hierarchy has a nominal impact on the overall performance of the network for several reasons. Some mechanisms, such as traffic filtering, have less of an impact on forwarding performance on lower speed lines. Since the speed of the inter-node connectivity generally gets faster as one goes higher in the network hierarchy, the impact of implementing policy in the higher levels of the network hierarchy increases. The same principal holds true for traffic accounting, access control, bandwidth management, preference routing, and other types of policy implementations. These mechanisms are more appropriately applied to nodes which reside in the lower portions of the network hierarchy, where processor and memory budgets are less critical. Another compelling reason to push policy out towards the network periphery is to maintain stability in the core network. In the case of BGP [2] peering with exterior networks, there are periods of route instability between exterior routing domains which can destabilize a router's ability to forward traffic at an optimum rate. By pushing exterior peering out to the distribution or access levels of the network hierarchy, this protects the core network and minimizes the destabilizing effect on the network's ability to provide maximum performance. This fundamental approach allows the overall network to achieve maximum performance and maintain stability, while accommodating the ability to scale the network to a much larger size and higher speeds. Summary It is critical to understand the impact of implementing policy in various points in the network topology, since mechanisms employed to achieve a specific policy may have an adverse effect on the overall network performance. Since high speed traffic forwarding is usually found in the network core, and conversely, lower speed forwarding is generally found lower in the network hierarchy, there can be varying degrees of performance impact depending on where these types of policies are implemented. In most cases, there is a much larger percentage of traffic which transits the network core than transits any one particular access point, so implementing policy in the network core has a higher degree of impact on a larger percentage of the traffic. By the same token, any adverse performance impact due to such policy implementations in the higher levels of the network hierarchy has a broader impact on a larger percentage of the overall network. Policy implementation in large networks should be done at the lowest portions of the hierarchy as possible, in order to avoid performance degradations which impact the entire network. ________________________ Footnotes: [1] Floyd, S., and Jacobson, V., "Random Early Detection gateways for Congestion Avoidance," IEEE/ACM Transactions on Networking, V.1 N.4, August 1993 [2] Rekhter, Y., and Li, T., "A Border Gateway Protocol 4 (BGP-4)," Internet Engineering Task Force (IETF) Request for Comments (RFC) 1771, March 1995 [end] -- Paul Ferguson || || Consulting Engineering || || Herndon, Virginia USA |||| |||| tel: +1.703.397.5938 ..:||||||:..:||||||:.. e-mail: pferguso@cisco.com c i s c o S y s t e m s