I've got a very straight forward setup with the following devices:
- 1 x 10G Fibre uplink (internet)
- 1 x ASUS AC68U wifi router
- 1 x EX4200
- 20 x Servers (With IPMI and single ethernet)
The EX4200 is configured so the top row of ports are on vlan 0, and the bottom row are on vlan 5 (management vlan). This is all created from the initial setup wizard, I've change no other settings on the switch.
The ASUS AC68U router is on the management vlan, and provides DHCP and, private network IPs. All the vlan 5 ports connect with IPMI ports.
The 10G fibre uplink plugs into the SFP+ module, and provides vlan 0 with internet connectivity and external IPs via an external router. All the vlan 0 ports plug into the ethernet NICs.
Now, the issues I have had so far are as follows:
Initially I setup the switch without the fibre plugged in. Upon plugging it in, the link showed as "UP" and everything looked okay. However, we were getting no internet connectivity through the fibre. I performed a factory reset with the fibre plugged in, with identical settings and it suddenly started working.
A few days later I changed some network settings on the ASUS AC68U. I increased the private subnet from /23 to /22. Upon applying this change, we again lost internet connectivity to the fibre. They are on entirely different VLANs so I'm not sure how they affected eachother. The only connection is that the EX4200 IP is on that private subnet. However, I could connect to all of the private network still, it was just the fibre and internet connectivity that stopped working. A factory reset of the EX4200 (with identical settings) fixed this issue aswell.
It feels very fragile, and I really can't understand what is going on here. I am now too afraid to change any network settings on any connected device as it may cause another internet outage via the fibre.
Why is this setup so fragile, have I misunderstood something?