Have Your Layer Cake and Eat It Too
Pages: 1, 2

Decisions, Decisions

So what's in it for system administrators? What's the best course of action for your network? This article doesn't answer these questions; instead, it describes the factors to take into account when making network equipment assessments and decisions.

There are several benefits to having all of your networking functions performed by just one box (or with just a few boxes). You deal with a limited amount of vendors, which makes administration easier, as there is usually only one set of commands with which to become familiar. Warrantee and support issues are also simplified. Less equipment typically means less cost, so there is a financial advantage to choosing a more layered approach -- important in the world of the dwindling IT budget. For smaller shops, this type of consolidation can be the difference between implementation and a daydream.

Layer 2 redundancy is another benfit of feature consolidation. With multiple devices, configuring redundancy is often difficult, if not impossible (some devices, such as high-end Layer 3 routers, don't offer redundant Layer 2 interfaces). And multiple boxes can create a tangle of cross-connects and other redundancy nightmares. If you have just a pair of devices performing all of your networking functions, it's easy to implement Layer 2 redundancy.

Figure 1 illustrates the Layer 2 redundancy nightmare scenario -- all too common a challenge for site administrators. Each device has a separate task (Layer 2, Layer 3, firewall, and so on), and each device requires you to provide a redundant unit for fail-over. For complete Layer 2 redundancy, every device (including the servers themselves) needs to have a double connection, one into each redundant Layer 2 switch. You'll note the Layer 3 routers do not have such a double connection; this is because most Layer 3 routers do not have redundant Layer 2 connections per interface.

Figure 1: Layer 2 redundancy nightmare.

This type of wiring scenario can be complicated and difficult to administer and troubleshoot. The only things to do at this point are to tolerate the situation or sacrifice some of the Layer 2 redundancy for an easier and more elegant configuration.

In Figure 2, we see how using just a single pair of multipurpose devices simplifies redundancy for the installation. Only double Layer 2 connections to the individual servers themselves are required for full redundancy.

Figure 2: Single device pair redundancy scenario.

Redundancy issues, especially those in the Layer 2 realm, are complicated and would require a separate article (or even a book) to do them justice. We're just touching on some of the issues related to choosing what type of device -- either single-purpose or multipurpose -- is best for a site.


One of the greatest disadvantages to feature consolidation is that it's difficult to provide for all of your needs in just one box. As the saying goes, "a jack-of-all-trades is master of none," and this is certainly true of networking devices. You may find a Layer 2/3 switching platform that also provides SLB, but which lacks a few key features required for your specific site, such as cookie-based persistence or the ability to perform certain kinds of Network Address Translation (NAT) functions. Another Layer 2/3 switch might provide BGP service, but doesn't have enough RAM to pull down a full BGP session. The lack of a specific feature you need isn't always obvious, either, and can be a big gotcha you only discover after spending thousands on equipment.

The cost-savings argument can be flipped in favor of non-consolidation as well, such as the case of a large-scale site and Layer 2 port aggregation. An all-in-one device may perform all of the functions you need, but the cost per port may be significantly higher than that of a regular Layer 2 switch. It often makes much more sense to use a high-density Layer 2 switching platform (and thus relative low cost-per-port) to aggregate Layer 2 connectivity, while separate functions such as firewalls, load balancers, and so on are connected into the Layer 2 infrastructure. An example would be using a pair of Cisco Catalyst 6500s to aggregate traffic from a pair of F5's BIG-IPs to a large number of servers.

Performance may also be a factor in deciding whether to consolidate features. If the implementation of one feature is pegging the resources of a device, then other, unrelated features might likely suffer as well. For instance, it's theoretically possible to have a situation where the heavy use of the load balancing features of a device used in a small portion of a site may affect performance for the rest of the infrastructure. Of course, this depends on the architecture of such a device (some devices might have separate processing resources for various tasks), but it's something to bear in mind.

Hands-On Experience

I've set up a number of large, medium, and small-scale sites, and there is a difference in how the various consolidations of layers affect implementation based on the size.

Smaller-scale installations, such as from one- to ten-server configurations, can definitely benefit from feature consolidation. There are many vendors that provide Layer 2 connectivity, server load balancing, access lists, and routing in a single device (or with a pair of redundant devices). Less equipment usually translates into less cost, and usually, the basic functions required by a smaller-size site are met by general functionality. This can be critical for budget-starved installations looking to save money.

Larger-scale installations have different needs. Because there may be a need for a large number of Fast/Gigabit Ethernet ports, I usually recommend going with a large Layer 2 infrastructure, with the other network devices, such as load balancers and routers, hung off of the Layer 2 infrastructure. Getting a full-featured box can really jack up the per-port price, while using a basic, inexpensive Layer 2 platform plus additional devices to provide other required functionality can be more cost-effective. Performance issues also can be scaled more effectively this way. It's possible, depending on the product, that heavy use of one individual function might degrade the performance of the entire system.

One exception to this separation is the new breed of Layer 2/3 chassis switches. The cost per port for Layer 2/3 functionality is usually not all that much more than for just Layer 2 functionality. These devices can perform all of the necessary Layer 3 functions, including BGP routing, with either a software or a minor hardware upgrade (such as an additional blade on a chassis). Functions such as load balancing and SSL acceleration are usually best served with separate devices, however.


In the end, you have to realize that vendors are in business for revenue and market share, and that your ambition is contrary to theirs -- having a successful site, of course. Keep your needs in mind and realize that while a vendor may be pushing a miracle device, it may not suit your needs. That said, there are a number of solutions available that could provide for all of your needs in one set of devices, and several different vendors may have developed those solutions. Be sure to check for the features specific to your site, as some products may not have one particular and critical feature. Who knows, in the next five years there may be just one device that encompasses all network requirements, is the size of a toaster, and, well, also makes toast.

Tony Bourke is a private consultant specializing in Unix administration, networking, and load balancing.

Return to the O'Reilly Network.