I have two networks communicating with each other through a transparent firewall. My problem is that the software using these networks is reporting that it's losing packages. The number of lost packages varies, but it may be around 0.1%. This applies even with "any/any" rules in both directions on the firewall. I have done some capturing with Wireshark on one side of the firewall, and I see that there are some ICMP packets generated when the firewall is connected. If I replace the firewall with a patch cable, the number of ICMP packets significantly decreases.
In captures of 1.5M packets, both with and without the firewall:
With the firewall: About 100 ICMP packets saying
Time-to-live exceeded (Fragment reassembly time exceeded), and 40 to 50
Destination unreachable (Port unreachable)packets.
Without the firewall: About 40
Time-to-live exceeded (Fragment reassembly time exceeded)packets, and none of the
To me, it seems like some packets are delayed/damaged by the firewall, hence the ICMP messages, but I can't really imagine how that could be. Any ideas on why this is happening, or how to best continue troubleshooting? I'm thankful for any input!
The firewall is a Phoenix Contact mGuard GT/GT, and the all switches are Cisco 3650.
EDIT: Regarding network throughput, the mirrored interface on the switch I'm capturing traffic from shows only about 0.25% utilization, ie 2.5 Mbit. That should be well within the limitations on the firewall.
EDIT2: I have done some bandwidth testing through the firewall, and the results are quite bad. I'm using iperf3 on two machines connected to the switches on both sides of the firewall, and when forcing UDP traffic (the majority of the regular traffic is UDP) between them I can just send about 30Mbit/sec of data before it starts to lose packets. At 50Mbit/sec about 16% of the packets are lost and at 200Mbit/s there is a gigantic packet loss of over 80%.
The firewall can't be that bad, or am I missing something?