1

I have several Juniper EX4300 series switches that I'm trying to configure to meet the following specifications:

  • I'd like each switch to manage a /24 DHCP pool that corresponds to a rack
  • I'd like traffic to be able to route to any other rack all within a /16 subnet
  • I'd like the topology to be such that a single switch failure results in loss of access to only the rack that it is responsible for.

To accomplish this I have done the following:

  • Switches are arranged in a loop and peered with neighboring switches using eBGP
  • The peering interfaces use 10.33.N1.254/30
  • All other interfaces on a switch are part of a VLAN (different vlan-id per rack) with an IRB associated with it. So for rack2, it's IRB: 10.33.2.100/24
  • For the DHCP pool, I have it configured this way:
    address-assignment {
        pool dhcp-pool {
            family inet {
                network 10.33.2.0/16;
                range my-range {
                    low 10.33.2.1;
                    high 10.33.2.232;
                }
                dhcp-attributes {
                    server-identifier 10.33.2.100;
                    maximum-lease-time 300;
                    router {
                        10.33.2.100;
                    }
                }
            } 
        }
    }

When pinging a DHCP client from its own switch, I'm able to communicate with the client; however, when trying to ping the client from another switch I am unable to get a response. The packets arrive at the client device, but the route back to the requesting switch fails as the source IP of the incoming packet is the BGP interface IP address and the response packet doesn't go back through the 10.33.2.100 gateway. If I manually add a route to the client that specifies that the BGP IPs should go through the 10.33.2.100 gateway, I don't have any issues. This is happening for a number of clients connected to different racks, but here is the route table on a client

default via 10.1.10.1 dev eno8303 proto static
default via 10.33.4.100 dev eno8403 proto dhcp src 10.33.4.1 metric 100
10.1.10.0/24 dev eno8303 proto kernel scope link src 10.1.10.232
10.33.0.0/16 dev eno8403 proto kernel scope link src 10.33.4.1 metric 100
10.33.4.100 dev eno8403 proto dhcp scope link src 10.33.4.1 metric 100

Am I missing something in my DHCP configuration to ensure that traffic is routed through the gateway for anything in the /16 subnet?

For reference here is a minimal config

system {
    services {
        dhcp-local-server {
            group default {
                interface irb.20;
            }
        }
    }
}
access {
    address-assignment {
        pool dhcp-pool {
            family inet {
                network 10.33.2.0/16;
                range my-range {
                    low 10.33.2.1;
                    high 10.33.2.232;
                }
                dhcp-attributes {
                    server-identifier 10.33.2.100;
                    maximum-lease-time 300;
                    router {
                        10.33.2.100;
                    }
                }
            } 
        }
    }
}
interfaces {
    ge-0/0/0 {
        unit 0 {
            family ethernet-switching {
                storm-control default;
                vlan {
                    members mgmt;
                }
            }
        }
    }
    ge-0/0/46 {
        description "rack1";
        unit 0 {
            family inet {
                address 10.33.11.254/30;
            }
        }
    }
    ge-0/0/47 {
        description "rack3";
        unit 0 {
            family inet {
                address 10.33.21.253/30;
            }
        }
    }
    irb {
        unit 20 {
            family inet {
                address 10.33.2.100/24;
            }
        }
    }
}
forwarding-options {
    storm-control-profiles default {
        all;
    }
}
policy-options {
    policy-statement BGP-EXPORT {
        term 1 {
            from protocol bgp;
            then accept;
        }
        term 2 {
            from {
                protocol direct;
                route-filter 10.33.2.0/24 exact;
                route-filter 10.33.11.254/30 exact;
                route-filter 10.33.21.253/30 exact;
            }
            then accept;
        }

        then reject;
    }
}
protocols {
    igmp-snooping {
        vlan all;
    }
    lldp {
        interface all;
    }
    lldp-med {
        interface all;
    }
    rstp {
        interface all;
    }
    bgp {
        group public {
            type external;
            local-as 65332;
            export BGP-EXPORT;
            neighbor 10.33.11.253 {
                description rack1;
                peer-as 65331;
            }
            neighbor 10.33.21.254 {
                description rack3;
                peer-as 65333;
            }
        }
    }
}
routing-options {
    autonomous-system 65332;
}
vlans {
    mgmt {
        vlan-id 20;
        l3-interface irb.20;
    }
}

Any help or insight would be appreciated, thanks

3
  • 1
    Is your network a /16 or a /24? It can't be both at the same time. Trying to route and bridge at the same time is going to be a headache.
    – Ricky
    Sep 5, 2023 at 23:19
  • @Ricky hmmm well I guess I was hoping that I could have a /24 per rack but then a /16 across racks would help with routing but maybe that was a mistake. The goal was for each rack to manage its own DHCP pool but then be able to have any device in any rack talk to any other device. Is there a better way to accomplish that rather than mixing /16 and /24 subnets?
    – Suever
    Sep 6, 2023 at 0:17
  • From my understanding I should just be able to setup each rack as a /24 VLAN and then set its own IRB as the DHCP router which should be used as the default gateway by the clients. I think that will work on most machines where this is the only interface and we may just have to configure /16 routes on the machines with multiple interfaces
    – Suever
    Sep 6, 2023 at 0:32

1 Answer 1

1

The reason your ping is failing is because the DHCP server is giving a /16 mask to the client - when it looks up how to get back to the peering address of the remote switch, it will determine that the switch is in it's local subnet (it falls within the /16) and not attempt to use the gateway to reach it.

set access address-assignment pool dhcp-pool family inet network 10.33.2.0/24
commit

then renew the client's DHCP lease and everything should be working.

I'd like traffic to be able to route to any other rack all within a /16 subnet

BGP will take care of this for you - each time a new rack/switch comes online, all the other switches will learn the /24 for that rack, and be able to find the best path to it.

Also as an aside:

  • It is good practice to use the first available address in each subnet as your gateway, not a random number out of the middle of the range eg: change your IRB to 10.33.2.1/24
  • You don't need to manually set server-identifier in the DHCP server configuration, it will be automatically set when the packet is created, and it's one less thing you need to set/change when you copy this config to other switches
  • While the numbering scheme is "cute", it will top out at 10 racks, when your rack subnets clash with your peering addresses. Also, using 10.33.N1.254/30 won't work for every second rack - it's peering addresses will be borrowed from the adjacent rack subnets (eg: rack 3 will have 10.33.11.254/30 and 10.33.21.254/30, neither of which will identify it. I would recommend using a dedicated /24 for peering addresses and just allocate consecutive /30s (or /31s) for each of the links.
  • I notice your client machine has two default routes - this may be a quirk of your testing, but this is obviously not ideal for production
3
  • Thanks so much for the detailed response. I was able to use this to get it successfully working combined with specifying option 121 routes in the DHCP server config to advertise 10.33.0.0/16 to the clients. Thanks also for all the notes at the end, I appreciate the insight!
    – Suever
    Sep 11, 2023 at 11:39
  • Do you need option 121 routes? Wouldn't your clients just rely on a default gateway to reach 10.33/16? Or do they have multiple NICs with the DG on one of the others? Sep 12, 2023 at 21:16
  • 1
    Yea they have multiple NICs and one of the other ones is actually the "real" default gateway, this one is just responsible for a /16 subnet
    – Suever
    Sep 13, 2023 at 18:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.