I'm trying to come up with a plan to get our core network design a bit more resilient to failure.
Here is a drawing of what I would like to achieve:
The final design would probably involve one more instance, but the drawing got way too cluttered with a third line all around the place.
What I've figured out on my own is how to set the MST instance root for each instance. What I don't quite get is how I'm supposed to design this that RACKSW02 takes over root for MST2 if RACKSW01 dies etc. Is it just a matter of setting the correct MST root priorities around the setup?
Do I need to set custom link costs if I want to override the default metrics, as RACKSW01<->RACKSW02 is 20gbps, while CORESW01/02 only has 10gbps?
The design goal behind all this is to balance out the load across the links, as 99.9% of the MST2 instance is only between the two top-of-rack switches.