4

I am struggling with the concept of 'Fabric Plane' and 'Switch Fabric'? I haven't found any clear cut difference between them? Why fabric planes are different in MX960 compared with MX240 and MX480. How SCB effect throughput?

I am quoting the paragraph from the book Juniper Book MX series:

The MX240 and MX480 support two SCBs for a total of four switch fabrics and eight fabric planes. The MX960 supports three SCBs for a total of six switch fabrics and six fabric planes.

This begs the question, what is a fabric plane? Think of the switch fabric as a fixed unit that can support N connections. When supporting 48 PFEs on the MX960, all of these connections on the switch fabric are completely consumed. Now think about what happens when you apply the same logic to the MX480. Each switch fabric now only has to support 24 PFEs, thus half of the connections aren’t being used. What happens on the MX240 and MX480 is that these unused connections are grouped together and another plane is created so that the unused connections can now be used. The benefit is that the MX240 and MX480 only require a single SCB to provide line rate throughput, thus only require an additional SCB for 1 + 1 SCB redundancy.

I will be very happy if someone explain it. Many Thanks

2

EDIT: I believe I tried to remain a bit too abstract in my answer, I've updated my answer to more acutely answer your questions.

I am struggling with the concept of 'Fabric Plane' and 'Switch Fabric'? I haven't found any clear cut difference between them?

At a very high level, SCBs contain Switch Fabrics, and Switch Fabrics contain Fabric Planes. The collective role of these components is to provide connectivity between all of the PFEs on each FPC installed in the chassis.

Switch Fabric: Switch Fabrics physical chips that are contained within the SCBs. Each SCB will have 2 Switch Fabrics.

Fabric Planes: Fabric Planes are more dynamic in nature, the number of Fabirc Planes per Switch Fabric will vary depending upon the MX chassis model.

Why fabric planes are different in MX960 compared with MX240 and MX480.

As previously stated, the role of the SCBs is to provide connectivity between the PFEs on the FPCs. This connectivity, must be organized into a full mesh, this means that each FPC will have the same amount of throughput capacity to every other FPC. Every time you add another FPC, the SCBs must be able to support that full mesh requirement.

Using the book's example, MX240s and MX480s have 2 SCB slots, which will result in 4 Switch Fabrics and 8 Fabric Planes. But, they also have a smaller number of available FPC slots. It simply comes down to the fact that the MX240 and MX480 only need half of the Fabric Planes in order to accomodate the full mesh requirement, leaving the remaining half to be used as spares (backup).

NOTE: To answer the comment you left on Benjamin's answer, yes, those grey boxes are the spare planes in that particular example, be aware, it can vary per FPC type.

If we look at the book's example of the MX960, even though it has an additional SCB slot, it also has more FPC slots (and therefore more PFEs). In order to maintain that full mesh requirement for the increased number of PFEs, half of the Fabric Planes would not be enough. So, it will "partition" the Fabric Planes a bit differently. Now, we end up with 6 total Fabric Planes (3 SCBs, with 2 Switch Fabrics each, and 1 Fabric Plane per Switch Fabric).

How SCB effect throughput?

There are additional complexities to complement what I mentioned above. First, is the fact that different model SCBs support different levels of maximum bandwidth. Second, FPCs also support varying levels of bandwidth. Finally, there are your redundancy and throughput requirements.

Using an MX960 as an example, utilizing fabric redundancy will keep 1 SCB as a hot standby in case that one of the others fail. This means that only 2 SCBs of capacity will ever be used at any one time. You have to account for your network capacity in that those 2 SCBs must be enough to handle the traffic on your router. On the other hand, if you would rather used "increased-bandwidth" mode, you can do so, which will enable all 3 SCBs to be utilized. However, if a SCB fails during a period of high network traffic, you may experience network congestion and performance impact.

MX Series Routers Fabric Resiliency

All of these factors must be considered when deciding what hardware to use.

One final note, there is a newer "2nd Edition" of the MX book that you may find helpful as well.

  • Thanks Jordan for the detailed reply. Interestingly Juniper does try to mention difference between Fabric Plane and switch fabric. But for us its not clear. Here is next page from the same book about Switch Fabric and Fabric Plane. – NABEEL NASIR Jan 10 at 6:52
  • Given that the MX240 and MX480 only have to support a fraction of the number of PFEs as the MX960, we’re able to group together the unused connections on the switch fabric and create a second fabric plane per switch fabric. Thus we’re able to have two fabric planes per switch fabric, – NABEEL NASIR Jan 10 at 6:53
  • so in case of MX240 / MX480 it says each CB has 2 Switch Fabric and each Switch Fabric has 2 x Planes. So for 2 x CBs , we have total 4 Switch Fabric and hence 8 x Fabric Planes. This is really confusing ... – NABEEL NASIR Jan 10 at 7:03
  • @NABEELNASIR I'll update my answer to account for your questions soon. – Jordan Head Jan 10 at 14:35
  • 1
    That really helped. Really appreciate ur detailed reply. – NABEEL NASIR Jan 10 at 17:45
5

The wording in that document isn't great - IMO it should read

What happens on the MX240 and MX480 is that these unused connections are grouped together and another fabric is created so that the unused connections can now be used

However Juniper terminology-wise, the switch fabric really equates to the sum of all fabric planes.

These diagrams may explain it a little better (retrieved from https://kb.juniper.net/InfoCenter/index?page=content&id=KB23065):

MX480/MX240 Switch Fabric connectivity

MX960 Switch Fabric Connectivity

  • Thanks. But the thing that is not clear from the above image is that why in case of MX240 / MX480 we are left with spare fabric connections (called backup plane) shown in grey while in case of MX960 that not the case. They say 2 x SCBs are needed for the MX960 while only 1 needed for MX240 / MX480 while the other one acting as back Up. – NABEEL NASIR Jan 10 at 6:58
  • I've updated the answer with the MX960 Fabric diagram from the link I provided – Benjamin Dale Jan 11 at 0:03

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.