0

I can barely put into words the problem I'm having, but I'll give it a go.

I have a pfsense router on which I've set up openvpn to act as a client to connect into another network.

This has worked flawlessly for months, but is now no longer working. I've made what I believe to be unrelated changes to the firewall and taken to few backups to be able to roll back and verify if they might be the source of the problem.

I've tried moving the openvpn connection from udp to tcp with the hope of that fixing if the cause was a flaky connection. No diff.

Utilizing the tunnel on the firewall itself seem to work flawlessly.

Setting up the same tunnel on my laptop seem to work flawlessly.

But when the tunnel is up and (different) units on my lan try to traverse it it does not work properly. The weirdest but most reliable example I have of the problem is pinging a remote unit. It works every second time. If ping works all packages in the same ping-run get proper replies. If I ctrl-c and try again every package fails instead. This can be repeated indefinitely.

50% ping works

How can this behaviour possibly be explained?

Something to describe the environment: LAN overview

On the firewall I can see the traffic on the LAN interface with tcpdump like so:

# echo "FAILED"; tcpdump -i igb0 -n -vvv host 172.16.66.2 and icmp
FAILED
tcpdump: listening on igb0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:35:04.521732 IP (tos 0x0, ttl 64, id 25001, offset 0, flags [none], proto ICMP (1), length 84)
    10.11.12.182 > 172.16.66.2: ICMP echo request, id 58627, seq 0, length 64
21:35:05.527179 IP (tos 0x0, ttl 64, id 16857, offset 0, flags [none], proto ICMP (1), length 84)
    10.11.12.182 > 172.16.66.2: ICMP echo request, id 58627, seq 1, length 64
21:35:06.532373 IP (tos 0x0, ttl 64, id 31585, offset 0, flags [none], proto ICMP (1), length 84)
    10.11.12.182 > 172.16.66.2: ICMP echo request, id 58627, seq 2, length 64
21:35:07.533037 IP (tos 0x0, ttl 64, id 46426, offset 0, flags [none], proto ICMP (1), length 84)
    10.11.12.182 > 172.16.66.2: ICMP echo request, id 58627, seq 3, length 64
21:35:08.538315 IP (tos 0x0, ttl 64, id 41339, offset 0, flags [none], proto ICMP (1), length 84)
    10.11.12.182 > 172.16.66.2: ICMP echo request, id 58627, seq 4, length 64

and the next attempt:

# echo "SUCCESS"; tcpdump -i igb0 -n -vvv host 172.16.66.2 and icmp
SUCCESS
tcpdump: listening on igb0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:35:23.228088 IP (tos 0x0, ttl 64, id 32178, offset 0, flags [none], proto ICMP (1), length 84)
    10.11.12.182 > 172.16.66.2: ICMP echo request, id 60163, seq 0, length 64
21:35:23.237680 IP (tos 0x0, ttl 63, id 54367, offset 0, flags [none], proto ICMP (1), length 84)
    172.16.66.2 > 10.11.12.182: ICMP echo reply, id 60163, seq 0, length 64
21:35:24.229373 IP (tos 0x0, ttl 64, id 38273, offset 0, flags [none], proto ICMP (1), length 84)
    10.11.12.182 > 172.16.66.2: ICMP echo request, id 60163, seq 1, length 64
21:35:24.238655 IP (tos 0x0, ttl 63, id 54368, offset 0, flags [none], proto ICMP (1), length 84)
    172.16.66.2 > 10.11.12.182: ICMP echo reply, id 60163, seq 1, length 64
21:35:25.230248 IP (tos 0x0, ttl 64, id 13988, offset 0, flags [none], proto ICMP (1), length 84)
    10.11.12.182 > 172.16.66.2: ICMP echo request, id 60163, seq 2, length 64
21:35:25.239532 IP (tos 0x0, ttl 63, id 54369, offset 0, flags [none], proto ICMP (1), length 84)
    172.16.66.2 > 10.11.12.182: ICMP echo reply, id 60163, seq 2, length 64
21:35:26.235280 IP (tos 0x0, ttl 64, id 62651, offset 0, flags [none], proto ICMP (1), length 84)
    10.11.12.182 > 172.16.66.2: ICMP echo request, id 60163, seq 3, length 64
21:35:26.244645 IP (tos 0x0, ttl 63, id 54370, offset 0, flags [none], proto ICMP (1), length 84)
    172.16.66.2 > 10.11.12.182: ICMP echo reply, id 60163, seq 3, length 64

I can't say I see anything differentiating the working packages from the ignored ones.

Using the same tcpdump (more or less, adjusted for openvpn NAT) I have been able to verify that the ping packets that don't get any replies never reach the remote end. The ones that do do of course arrive as expected.

Improve this question
6
  • You need to provide more information. We need the network topology (a diagram would be great), the network device models, and the network device configurations (obfuscate any passwords and public addresses). Also, please never use images for text. Instead, simply paste the text into your question, and use the Preformatted-Text feature ({}).
    – Ron Maupin ♦
    Commented Jan 28, 2018 at 19:05
  • I can and will supply the information you need later. The reasoning behind the gif is not to supply the text obfuscated, but to show what words cannot, the timing in how I ran the command - to that end I truly believe it to serve a good purpose.
    – azzid
    Commented Jan 28, 2018 at 20:01
  • You still need to give us the network device configurations. Also, remember that ping is bidirectional, and the destination must reply for it to succeed.
    – Ron Maupin ♦
    Commented Jan 28, 2018 at 20:40
  • 1
    Check your firewall logs. if it's dropping traffic, it will show up in the logs.
    – Ron Trunk
    Commented Jan 28, 2018 at 22:14
  • 1
    You need to tell us from where it's working and from where it isn't (laptop? computer A? computer B?). There are only two VLANs, LAN and WAN?
    – Zac67 ♦
    Commented Jan 29, 2018 at 12:13

2 Answers 2

Reset to default
0

When the ICMP echo requests all enter the tunnel (check on the pfSense) and no replies return, it's either the tunnel that's not working (unlikely, you'd see some symptoms on your side) or the far side isn't working.

On the far side, it can either be a problem with the ovpn gateway or with routing back into the tunnel. A problem with routing back might be visible using traceroute, depending on how far beyond the tunnel end the destination is and where the routing stops working.

Since you're having this problem only when you ping from a certain subnet(?) my bet is on a routing problem.

Improve this answer
0

Found the issue!

The tunnel I was troubleshooting was set up as a vpn client (from the firewall perspective). The was also a vpn server set up.

When dumping on the vpn-interface rather than the lan-interface the problem showed its face more clearly:

# tcpdump -i ovpnc1 -n host 172.16.66.2 and icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ovpnc1, link-type NULL (BSD loopback), capture size 262144 bytes
19:23:09.190440 IP 10.8.0.6 > 172.16.66.2: ICMP echo request, id 14789, seq 0, length 64
19:23:09.200916 IP 172.16.66.2 > 10.8.0.6: ICMP echo reply, id 14789, seq 0, length 64
19:23:11.214663 IP 10.11.13.1 > 172.16.66.2: ICMP echo request, id 3794, seq 0, length 64
19:23:14.230789 IP 10.8.0.6 > 172.16.66.2: ICMP echo request, id 27669, seq 0, length 64
19:23:14.241737 IP 172.16.66.2 > 10.8.0.6: ICMP echo reply, id 27669, seq 0, length 64
19:23:16.255924 IP 10.11.13.1 > 172.16.66.2: ICMP echo request, id 65327, seq 0, length 64
19:23:19.282051 IP 10.8.0.6 > 172.16.66.2: ICMP echo request, id 2529, seq 0, length 64
19:23:19.292422 IP 172.16.66.2 > 10.8.0.6: ICMP echo reply, id 2529, seq 0, length 64
19:23:21.309039 IP 10.11.13.1 > 172.16.66.2: ICMP echo request, id 41732, seq 0, length 64
19:23:24.324271 IP 10.8.0.6 > 172.16.66.2: ICMP echo request, id 42558, seq 0, length 64
19:23:24.335141 IP 172.16.66.2 > 10.8.0.6: ICMP echo reply, id 42558, seq 0, length 64
19:23:26.347318 IP 10.11.13.1 > 172.16.66.2: ICMP echo request, id 10840, seq 0, length 64
19:23:29.363609 IP 10.8.0.6 > 172.16.66.2: ICMP echo request, id 26327, seq 0, length 64
19:23:29.374302 IP 172.16.66.2 > 10.8.0.6: ICMP echo reply, id 26327, seq 0, length 64
19:23:31.385639 IP 10.11.13.1 > 172.16.66.2: ICMP echo request, id 37158, seq 0, length 64
19:23:34.399109 IP 10.8.0.6 > 172.16.66.2: ICMP echo request, id 22445, seq 0, length 64
19:23:34.410245 IP 172.16.66.2 > 10.8.0.6: ICMP echo reply, id 22445, seq 0, length 64

My local LAN is on 10.11.12.0/24, the remote network is 172.16.66.0/24. Openvpn has its own network in between those. That is supposed to be 10.8.0.0/24, but as you can see the corresponding network for the (presumably unrelated) vpn server, 10.11.13.0/29, shows up uninvited.

Disabling the vpn server makes the vpn client traffic work as it should.

A few more hours with the openvpn handbook is probably what I need. Thanks for all your input.

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.