25

I have a question about oversubscription in networking. I read a lot of documentation but I still don't understand what it means. I read the following on the Cisco website,

oversubscription of the ISL is typically on the order of 7:1 or greater.

  1. What does oversubscription mean?
  2. Where is it used? Where should it be avoided?
  3. How do we calculate this value?
  4. If this is a configuration parameter, which commands are used to set it? (Cisco or Juniper)
  5. If it is configuration parameter, which devices or which IOS version support it?
  • Cisco has some oversubscription recommendations. For example, the access to distribution oversubscription ratio is recommended to be no more than 20:1 (for every 20 access 1 Gbps ports on your access switch, you need 1 Gbps in the uplink to the distribution switch), and the distribution to core ratio is recommended to be no more than 4:1. – Ron Maupin Jun 24 at 15:20
  • @RonMaupin Thanks and highly appreciate your comment – infra Jun 24 at 15:25
33

Suppose you have a core switch that connect to several access switches (leaf and spine topology).

If your access switches have each 48 1Gbs ports, you can potentially aggregate 48Gbs of traffic to be passed to the core switch, so you would need a connection between the core switch and each access switches of at least 48Gbs.

Most often, this would be wasteful, because in practice you will never encounter a situation where all ports receive traffic at their maximum rate at the same time.

So we could have an access switch with 48 ports at 1Gbs and an uplink to the core switch at 10Gbs

We then have an over-subscription of 4.8:1

If we use a lag with 2 x 10Gbs ports, we can reduce it to:

48 x 1 Gbs / 2 x 10 Gbs = 2.4:1

When we use it and when not to use?

As you can see it is almost always used when you have several switch layers.

You don't use it:

  • when you have only one switch layer (very small networks)
  • when you have very specific requirements and wants the full bandwidth available on all ports at any time (and enough money to do so)

How do we calculate these Value?

As in the example above, the over-subscription ratio is the ratio between the upstream bandwidth and the downstream capacity.

As for how to decide which final ratio to attain when designing / upgrading a network, it can be tricky. This is why, from its vast experience and analysis of real networks, Cisco make some recommendation, such as the one you quoted, or the one quoted by @RonMaupin in a comment:

the access to distribution oversubscription ratio is recommended to be no more than 20:1 (for every 20 access 1 Gbps ports on your access switch, you need 1 Gbps in the uplink to the distribution switch), and the distribution to core ratio is recommended to be no more than 4:1

But the correct values for a given network highly depend on the traffic pattern.

For existing network, a close monitoring of the bandwidth used on each port should give enough insight. You can also use netflow / sflow to analyze further what use the bandwidth.

When designing a new network you need to assess the expected traffic.

If this is a configurable parameter, what are the commands which use to configure?(Cisco or Juniper)

You can see now that it is not something we configure, but it is a design choice.

Note:
The ports speed is not always the limiting factor. Most often the switch hardware is not capable of handling the full bandwidth on all its ports simultaneously; this is indeed a kind of internal over-subscription (once again mostly driven by real usage patterns and costs).

  • Highly appreciate your answer, I understand theory and concept behind oversubscription. – infra Jun 24 at 14:41
  • Given a network switch that is already oversubscribed, how can you calculate what you need in the switch you're going to buy to replace it with? (I'm thinking that the oversubscription itself borks most measurements/measurement tools but maybe I'm wrong?) (Thanks for a good answer, too!) – davidbak Jun 24 at 22:51
  • 2
    @davidbak you need to monitor the bandwidth used on the uplink. If it is frequently saturated, then you need to reduce the over-subscription ratio. You could either move some clients to another switch or use linkg-aggregation. But hard to give a full answer to a comment without much more details. Could worth asking it as a full question. – JFL Jun 25 at 6:49
  • 1
    @JFL, I think when the O.P. was asking how to calculate the oversubscription ratio, he might have been asking literally "how do you calculate it?" As in (max_possible_demand / max_actual_bandwidth). Your comment is great, so I wouldn't lose it, but you might do an edit where you explain the calculation as well. Even though the O.P. seems to have gotten this, other readers might benefit! – Forbin Jun 25 at 22:55
  • @Forbin I thought it was obvious enough but I edited to make it clearer. – JFL Jun 26 at 15:29
14

Oversubscription generally refers to potentially requiring more resources from a device, link, or component than are actually available.

Let me explain with an example:

Consider a typical 2-layer network topology, with access switches and a central core switch. The access switches have 24 user ports and one uplink port. The uplink port is connected to the core switch.

Each access switch has 24 1Gb user ports and a 10Gb uplink port. So, in theory, if all the user ports transmitted to a server at the same time, they would require 24Gb of bandwidth (24 x 1Gb). But the uplink port is only 10Gb, so that limits the maximum bandwidth to all the user ports.

We say the uplink port is oversubscribed, because the theoretical required bandwidth (24Gb) is greater than the available bandwidth (10Gb). Oversubscription is expressed as a ratio of required bandwidth to available bandwidth. In this case it's 24Gb/10Gb or 2.4:1.

There are many other examples of oversubscription in networking. They are all based on the idea that statistically, not all ports will require maximum bandwidth at the same time.

Oversupscription is not a configurable parameter per se, but it is a feature of some components and the topology. You can choose devices or design your network to adjust the oversubscription as needed.

  • Highly appreciate your answer, I understand theory and concept behind oversubscription – infra Jun 24 at 14:42
12

What is Oversubscription meaning?

Simply speaking, oversubscription is the concept of providing more downstream capacity than your infrastructure can actually provide. Let me give you a few examples to illustrate this concept:

  • As other answers point out and describe well, any time you have an access switch where the total possible combined bandwidth of all client connections exceeds upstream bandwidth from the access switch to distribution/core, you are oversubscribed.

  • Often there are more copper cables run to a closet than are cross connected to a switch port (or that can even be accommodated by the number of switch ports in a closet). In these cases you are also oversubscribed as you have more "capacity" than you have infrastructure to support.

  • An ISP has far less bandwidth available (or if you go back to dial up days, modems available) than the total amount that is sold to their customers. To be profitable, every ISP I have every known is/was oversubscribed.

When we use it and when not to use?

You make use of oversubscription anytime it makes sense (in many organizations, this means save money or reduce ) in your business model and design. From the examples above, clearly it doesn't make sense to assume that every client connected to the network will fully utilize their maximum available bandwidth 100% of the time. Some client devices may be powered off, not in use, only produce a small amount of traffic, etc. Exactly how far you choose to oversubscribe is determined by the expectations of the business need.

Or, it makes little sense to provide an actual switch port for every copper connection run to a closet. Most deployments run excess copper cabling (pulling two runs when you need one, adding runs to multiple locations in an office to allow for different furniture placement, etc) when they do the work as it is often far cheaper to do so than to only run what you actually need and add additional cabling based on changing needs after the fact. Providing switch ports for each cable run increases the cost of purchasing and operating (i.e. power, support, etc) equipment.

How do we calculate this Value?

This is going to be highly dependent on the exact type of oversubscription you are referring to and the actual need(s) of the business/organization in that environment. In some cases this may be quite high, in others you may not be able to oversubscribe at all.

Most organizations will have different needs in different areas of their network, so often have different ratios in different places. For example, an organization's office space and data centers are likely to have significantly different oversubcription ratios.

If this is a configurable parameter, what are the commands which use to configure?(Cisco or Juniper)

Not directly as this is more of a design concept. However there are features that are present to accommodate the use of oversubcription in a network. Here are a couple of examples:

  • QoS (quality of service) allows an organization to determine which traffic has priority when actual traffic exceeds capacity (if traffic must be queued, which traffic and which queues are cleared first).
  • Rate limiting and/or traffic shaping allows an organization to put limits on the amount of traffic going to/from a client device, and possibly limit the specific types of traffic going to/from a client device.

These types of features and the capabilities they provide is going to be highly device specific and may change from one version of software to another. Using QoS as an example the size and number of queues that a device has available will vary.

  • Great explanation, Thank you so much for your effort. Thanks again highly appreciate – infra Jun 25 at 3:25
  • 1
    "An ISP has far less bandwidth available [...] than the total amount that is sold to their customers." This is very true and can be noticed in real life by customers especially on evenings in the weekend! – Tobias Knauss Jun 26 at 4:53
  • @TobiasKnauss is it oversubcribe situation in real life? – infra Jun 27 at 11:24
  • @infra: yes, absolutely! The actual Internet speed goes down for the single customer because the available bandwidth of the ISP is exhausted. As the answers to your question say, oversubscription is usual, but normally it is not noticed. Here it is noticed! – Tobias Knauss Jun 27 at 11:39
  • @TobiasKnauss Thanks for your reply – infra Jun 28 at 11:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.