How to design IXP with redundancy?

Question

With two route servers, one of them for redundancy, making peering for ISP's Do I have to make peering between the route servers? Despite of each route server would have all the prefixes for each ISP's.

Answer 1

From a design standpoint, specifically referring to the route servers, the answer is no, they should not peer with each other via an iBGP session [disclaimer: I've never personally attempted to do this, but I might now for fun]. The reason why is because route servers behave similary to route reflectors, only it's eBGP rather than iBGP. The most important part to remember though, is that a route server is not a router, it is a "BGP message broker". This means that to all of its route server clients, it does not participate in any routing decisions. Because of this, route server clients will configure their sessions to the IXP's route servers as a standard eBGP session, however, because the route server does not insert its ASN onto the AS_PATH of NLRI received from its clients before it advertises to its other clients, the client configuring the eBGP session will have to ensure that its eBGP config on whatever device they're configuring allows for a way to disable enforcing the first ASN in a path from an eBGP neighbor (see Cisco IOS implementation notes), otherwise (depending on the vendor of course) the session will not come up, or will come up but no NLRI is exchanged, etc. Keeping that in mind, by creating an iBGP session between the two route servers, every route server client will have to drop an NLRI update message from itself, that was received from one route server, forwarded to the other via iBGP, and then re-forwarded to the route server client that (presumably) has an eBGP session up with both route servers. This will basically cause unneeded headache (in the form of possible CPU and/or memory churn) for the route servers and all of their clients. With that said, yes, it is BCP to operate two route servers (without iBGP between them), and mandate that the route server clients peer with both of them if they're going to peer with them at all, as Leon Weber has already mentioned.

In addition, it should be noted that designing an IXP "with redundancy" does not simply mean "running two route servers." Using AMS-IX as an example again, I suggest taking a look at their design documentation (specifically for 10G customers) for a good example of redundancy in an IXP. Again, just to provide a single example - a discussion on designing for redundancy within an IXP is beyond the scope of the format of this site.

Answer 2

A common solution is to run two separate route servers and mandate all peers to peer with both of them simultaneously, announcing the same set of prefixes.

See the AMS-IX documentation for an example. They run two redundant route servers:

When peering with the route servers we mandate that routers are set up to connect to both route servers and advertise the same amount and length of prefixes for resilience.

Answer 3

It would be preferable to peer your two routers using ibgp. That way, your left hand is in sync with the right hand :-)

Your question doesn't specify if you have two different ISPs connected to your two routers or just one ISP that is connecting you. I would recommend using route filtering so as not to configure your routers to become a transit AS (unless you want to).

A common configuration would see your primary router advertising your AS normally and your backup router advertising your AS using prepending. That is to influence the return path towards packets that have left your network amongst other things.

Other configurations advertises your network to your primary ISP using the longest match while your backup router advertises your network as normal to the other ISP. For example RouterA is your primary connecting to ISP-A and your backup is Router-B connecting to ISP-B. Your network is 192.168.0.0/16. RouterA advertises 192.168.0.0/17 and 192.168.128.0/17 to ISP-A; RouterB advertises 192.168.0.0/16 to ISP-B

Truly there are many ways to manipulate how you advertise your networks and filter/manipulate the routes leaving/entering your network. Here is a good link towards prepending for your perusal: http://networklessons.com/bgp/how-to-configure-bgp-as-path-prepending/

Don't forget to test your configuration in a simulator like GNS3 first before implementing it on your production systems :-)