Essentials All Articles What is LAMP? Linux Commands ONLamp Subjects Linux Apache MySQL Perl PHP Python BSD ONLamp Topics All Articles App Development Database Programming Sys Admin

Traffic Engineering: Incoming Traffic
Pages: 1, 2

#### The Effect of AS Path Prepending

Suppose you multihome to two ISPs that are similar: they interconnect at mostly the same NAPs and Internet Exchanges, and they peer with mostly the same networks. Under these circumstances, other networks see two similar paths for the routes you announce. Figure 6-4 shows an example of this.

Figure 6-4. Multihoming to similar ISPs

If AS E (the example multihomed network, AS 60055) wants to receive more traffic over ISP A and thus makes the AS path longer over ISP B, the majority of traffic will flow over the ISP A, which now has the shorter path. In a situation with two similar ISPs, AS path prepending gives them only three choices:

• The default traffic distribution, which may or may not be balanced

• Longer path over ISP A: majority of traffic comes in over ISP B

• Longer path over ISP B: majority of traffic comes in over ISP A

Table 6-1 shows which route is preferred in the situation shown in Figure 6-4 without path prepending, with prepending the path to ISP A, and with prepending the path to ISP B.

Table 6-1: Prepended paths over similar ISPs

AS X AS Y AS Z Traffic distribution
Prepend to A AEE
BE
AEE
BE
AEE
BE
ISP A: 15%
ISP B: 85%
No prepending AE
BE
AE
BE
AE
BE
ISP A: 40%
ISP B: 60%
Prepend to B AE
BEE
AE
BEE
AE
BEE
ISP A: 90%
ISP B: 10%

For the purposes of calculating the traffic distribution, it's assumed that A always handles 15% of the traffic, B always 10%, and ASes X, Y, and Z are all the source of 25% of incoming traffic. ASes with "even" letters (X, Z) prefer to send traffic over ISP B when the paths are of equal length; "odd" ASes (Y) prefer ISP A in this example. The preferred path is listed in bold type in the table.

Figure 6-5. Multihoming to dissimilar ISPs

Table 6-2: Prepended paths over dissimilar ISPs

AS C AS V AS W AS X AS Y AS Z Traffic distribution
2 to ISP A BE BE BE
CBE
AEEE
CBE
AEEE
CBE
AEEE
BE
A: 15%
B: 85%
1 to ISP A BE BE BE
CBE
AEE
CBE
AEE
CBE
AEE
BE
A: 35%
B: 65%
No prepending BE BE BE
CBE
AE
CBE
AE
CBE
AE
BE
A: 55%
B: 45%
1 to ISP B BEE BEE BEE
CBEE
AE
CBEE
AE
CBEE
AE
BEE
A: 75%
B: 25%

The traffic distribution in this example is 15% from ISP A, 5% from ISP B and ASes V and W, 10% from AS C, and 20% from ASes X, Y, and Z.

TIP: All else being equal, it's a good idea to select dissimilar ISPs, for instance, one tier-1 ISP that peers with all the other large networks, and one tier-2 ISP that peers with many small networks. This way, you have a wide range of traffic engineering options.

#### Setting Outbound Communities

In many cases you'll want to prepend the path for certain upstream networks or peers of a transit ISP and not for others. For instance, if two of your ISPs have a transit network in common, you might want to have one ISP announce a prepended path to this transit network without changing the path that other transit networks and peers see over that ISP. To avoid spending a lot of time implementing this type of policy upon customer request, many ISPs provide their customers (and sometimes their peers) with a list of communities that trigger actions such as path prepending and setting the Local Preference. This can then be done for each route individually.

#### Well-Known Communities

Table 6-3: Well-known communities

Well-known community Action
`no-export` (`0xFFFFFF01`) Don't advertise this route to eBGP peers.
`no-advertise` (`0xFFFFFF02`) Don't advertise this route to any peers, iBGP, or eBGP.
`no-export-subconfed` (`0xFFFFFF03`) Advertise this route to iBGP peers with the same AS number, but not to other confederation members.

These communities can be useful under certain circumstances, for example, if an ISP wants to set the `no-export` community on routes it sends to a customer to make sure the customer doesn't accidentally announce the ISP's routes to another ISP. If the customer provides transit services to customers of his own, however, they won't receive the route unless the original customer of the ISP removes the `no-advertise` community.

WARNING: Setting the `no-export`, `no-advertise`, or `no-export-subconfed` communities can have the (possibly unwanted) side effect that no routes are announced, even if there are other routes that would otherwise be eligible for announcement.

For instance, if you set the `no-advertise` community on routes announced to ISP B, other customers of ISP B won't see these routes because they aren't advertised. This is as intended. But routes with the same NLRI that ISP B has learned from ISP A will not be advertised either, because ISP B considers the directly received routes with the `no-advertise` community best, and only the best route is eligible for further announcement over BGP.

#### Common Community Actions

Table 6-4: An example of communities an ISP accepts

Community Action
`50066:70` Set Local Preference to 70.
`50066:90` Set Local Preference to 90.
`50066:110` Set Local Preference to 110.
`50066:5010` Announce this route for transit to ISP C.
`50066:5020` Don't announce this route to transit ISP or peer C.
`50066:5040` Prepend AS path to C once.
`50066:5041` Prepend AS path to C twice.
`50066:5042` Prepend AS path to C three times.
`50066:10040` Prepend AS path once at interconnect point I.
`50066:10041` Prepend AS path twice at interconnect point I.
`50066:10042` Prepend AS path three times at interconnect point I.

Some ISPs require you to set a community indicating a route should be announced for transit, `50066:5010` in this example. This isn't common: most networks do the opposite and allow you to set a community indicating the route shouldn't be announced to transit networks. Be sure to notice the subtle difference between not announcing for transit and not announcing to transit networks: in the first case, the potential transit network still receives the announcement, but the route is treated as a peering route and not announced to peers and upstream networks of the transit network. In the latter case, the transit network doesn't get to hear the route at all.

#### Influencing the Local Preference in Upstream ASes

The MED is specifically intended to inform an upstream ISP that one connection should be preferred over another, but today this is often done with communities. Using communities instead of the MED may have benefits internal to the ISP network. For example, the ISP is then free to use the MED for another purpose, as we did for outbound traffic engineering in the beginning of this chapter. And, unlike the MED, using a community to set the Local Preference inside an ISP network also makes it possible to use a link to an ISP as a backup for a link to another ISP. When the Local Preference is set sufficiently low for the intended backup connection, the ISP it connects to will completely ignore the route and always send traffic over the other ISP as long as there is a route present over this ISP. This can't be accomplished with the MED; the AS path length overrides it, the MED isn't communicated from AS to AS, and by default, the MED is looked at only when two connections terminate at the same AS.

The impact of changing the Local Preference depends on the Local Preference values an ISP uses for routes learned from transit, peers, and customers. In this example, that would be 80 for transit, 100 for peers, and 120 for customers. If you have a fast main connection to this ISP along with a slower backup connection, you'll probably want to set community `50066:110` for routes announced over the backup connection. This makes sure all traffic flows over the main connection and the backup connection is used only when the main connection is unavailable. It's also possible to do this when you connect to two different ISPs: to ISP A with a main connection, and to ISP B with a backup connection. Then you'll want to set community `50066:90` or even `50066:70` so ISP B sends all traffic for you over a peering or transit connection to ISP A.

Example 6-13 shows a configuration for the router terminating the backup connection to ISP B. The main, high-bandwidth connection terminates at another router.

Example 6-13: Setting a community to indicate a backup route

``````!
router bgp 60055
neighbor 219.2.19.1 remote-as 50066
neighbor 219.2.19.1 route-map ispb-backup-out out
neighbor 219.2.19.1 send-community
!
route-map ispb-backup-out permit 10
set community 50066:5010 50066:70
!``````

The community `50066:5010` makes this route eligible for announcement as a transit route over AS C, but the community `50066:70` makes sure this route has the lowest possible Local Preference in ISP B's network. Thus, ISP B won't use it as long as there is any other route with the same NLRI (prefix), even if this means routing the traffic over ISP A.

TIP: By default, Cisco routers accept incoming communities but don't transmit them over iBGP or eBGP. The `send-community` command enables sending communities to a neighbor.

#### Prepending the AS Path

Some smaller ISPs have path-prepending communities for each peer, but even medium-sized ISPs peer with many networks, so this soon gets out of hand. More often, an ISP has communities to prepend the path to each of its transit ISPs individually, as well as communities to prepend the path for an entire interconnect point. Our example ISP B (AS 50066) accepts path prepending communities for ISP X and interconnect point I.

AS W in Figure 6-5 (shown earlier this chapter) connects both directly to ISP B and also over transit AS C and then ISP B. Supposing the peering link between AS W and ISP B is congested, we'll want incoming traffic from AS W to flow over ISP C. This is accomplished by prepending the path ISP B announces to AS W twice, as is done in the configuration in Example 6-14.

Example 6-14: Setting a community to prepend the path

``````!
router bgp 60055
neighbor 219.2.19.1 remote-as 50066
neighbor 219.2.19.1 route-map ispb-out out
neighbor 219.2.19.1 send-community
!
route-map ispb-out permit 10
set community 50066:5010 50066:10041
!``````

Unfortunately, it's not possible to do something similar for outbound traffic: this will still flow over the congested connection between ISP B and AS C. This isn't the case if it's routed over ISP A and not over ISP B, of course, as is done in Example 6-4 earlier this chapter (in a previous article). Also, setting community `50066:10041` means the path is prepended twice towards all peers at this interconnect point. This may be undesirable, for instance if AS Z connects to ISP B over the same interconnect point as AS W. AS Z now sees the path `C B B E` over ISP B and the much shorter path AS E over ISP A, so traffic from AS Z will now come in over the connection to ISP A.

In the next installment learn how to announce more specific routes. When prepending the path and setting communities for outbound routes aren't enough to balance incoming traffic, this is the only step left to take. However, because a more specific route always takes precedence over a less specific route, this technique always works.

Iljitsch van Beijnum has been working with BGP in ISP and end-user networks since 1996.