0

I set up our company network several months ago. Some weeks ago, we started having some issues, and I want to find the root cause. How do you debug networking problems in general?

My background is in Software Engineering, so I'm familiar with debugging techniques used there, you have simple console print statements, a live debugger to inspect variables in a running program, unit tests, and so on. However, I feel a bit lost in the network.

What are techniques/tools used by network professionals to find network issues?

Example 1: I have some VLANs on a router trunk port eth-r1, when connecting directly to it, I get a DHCP offer, when configuring a switch to bridge trunk on sw-eth1 and sw-eth2, connecting eth-r1 to sw-eth1 and my laptop to sw-eth2, I get no DHCP offer and I cannot ping other members.

Example 2: I have an internal WLAN, and clients are sometimes disconnected for no obvious reason, and don't connect automatically again.

What I know I can do:

  • Disconnect everything, then start connecting device by device (routers, switches, clients) to see if one of the devices causes the issue
  • In example 1, test intermediate devices directly to ensure they work correctly
  • Use Wireshark to analyse traffic – though I'm lacking knowledge here on what to look at and where (at which place in the network) to analyse

1 Answer 1

3

The basic approach is to start on the lowest layer = the physical layer. Of course, there are many scenarios where you can rule out any problems in the lower layers in the first place, but you should at least consider them.

Then, you work up the layers.

Your question is very broad, so troubleshooting requires a rather long list. Sorry, we can't provide a complete tutorial as that would easily fill a book.

  1. physical connectivity (L1): do all sides indicate successful links (LED, OS link state)
  2. physical connectivity (L1): check error counters on all links from each side
  3. MAC connectivity (L2): are both source & destination MACs visible in the expected VLANs (show mac-address on switch consoles)?
  4. IP-to-MAC (L3->L2): can hosts/gateways resolve the destination
  5. IP (L3): are the expected IP addresses configured/active?
  6. IP (L3): does the end-to-end routing work? (ping, traceroute, ...)
  7. TCP/UDP (L4): do the ports connect or datagrams reach their destination (netstat, packet capture, ...)
  8. SSL/TLS (L4/L7): does the encryption handshake work? (packet capture, indication in app)

The rest is rather application specific and off topic here (DNS, authentication, authorization, ...).

Of course, things can be much more complicated in a complex setup. E.g. 802.1X port security or WEP Enterprise requires you to check the entire authentication chain from supplicant over authenticator to the RADIUS server. Intermediate firewalls often require detailed tracing (packets and rule sets).

Example 1: I have some VLANs on a router trunk port eth-r1, when connecting directly to it, I get a DHCP offer, when configuring a switch to bridge trunk on sw-eth1 and sw-eth2, connecting eth-r1 to sw-eth1 and my laptop to sw-eth2, I get no DHCP offer and I cannot ping other members.

You'd need to check L2 connectivity between DHCP client⇔server (or between DHCP client⇔relay and relay⇔server).

Example 2: I have an internal WLAN, and clients are sometimes disconnected for no obvious reason, and don't connect automatically again.

That's an L1 problem - check logging on the WAP and on the client for which problems are indicated.

1
  • Thanks @Zac67, this bottom-up approach makes a lot of sense! Now I'll just have to learn about all the separate steps ;) Commented Jan 23, 2023 at 12:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.