Hi @prussbw
There was an earlier request to see if the development kernel (5.10) made any difference.
With Core Update 157 installed you just need to download kerneltest-5.10.45-ipfire-x86_64.tar (presuming you are on x86_64) from the following link:-
https://people.ipfire.org/~arne_f/highly-experimental/kernel-5.10/
There is a readme file there as well telling you where to place the downloaded file and what to run.
Ok - I was able to download and install the replacement kernel without any issues, and have confirmed that itās running, but Iām still seeing the same fragmentation issue. (Iāll keep gathering PCAPs, but I wonāt send them unless someone thinks that it would add anything to the investigation.)
Iāll also try the c159 testing release and report back on that.
Update: Iāve also tried running c159 (as suggested by @ms in the Bugzilla issue thread). Unfortunately, I did not see a change in behavior - Iām still seeing UDP packets fragmenting at small sizes.
Iāll be rolling back to c152 shortly, but please let me know if thereās anything else I can look into.
I was looking at my routing tables using the āip routeā command. I got the following response with cleaned up IP addresses.
default via ##.##.##.## dev red0 proto dhcp src ##.##.##.## metric 202 mtu 576
##.##.##.0/20 dev red0 proto dhcp scope link src ##.##.##.## metric 202 mtu 576
192.168.7.0/24 dev green0 proto kernel scope link src 192.168.7.1
192.168.8.0/24 dev blue0 proto kernel scope link src 192.168.8.1
The mtu was a problem. I had been having issues with my phoneās VPN. After clearing that to the following settings I was able to get the VPN to work again. Tomorrow I will check out if this resolves for other services as well. I am not 100% of these settings. My hardware still says 1500 for the MTU.
default via ##.##.##.## dev red0 proto dhcp src ##.##.##.## metric 202
##.##.##.0/20 dev red0 proto dhcp scope link src ##.##.##.## metric 202
192.168.7.0/24 dev green0 proto kernel scope link src 192.168.7.1
192.168.8.0/24 dev blue0 proto kernel scope link src 192.168.8.1
I upgraded to 159 and the settings seem to still work.
A few new data points:
I have found that this issue can be replicated in a more simple manner by using ping packets with large payloads. To demonstrate the issue with ping, I can just use āping -s 1000 8.8.8.8ā (or āping -l 1000 8.8.8.8ā on Windows). On core 152, I get a normal response back (albeit truncated to a 68 byte payload). On later cores with the issue Iām seeing, the outgoing ICMP echo request packet gets fragmented (as confirmed with packet captures) and 8.8.8.8 ignores it.
While still running Core 152, I also checked the āip routeā command output and saw something very similar, with the MTU set to 576. I did a small experiment and saw some interesting results: If I send a large ping as described above from a host on the green interface subnet, it gets routed out the red interface without being fragmented. However, if I send the ping from the IPFire gateway itself, it gets fragmented and ignored. If I manually modify the routing table to remove the āmtu 576ā setting (as @bjaminn did), then large pings from inside IPFire work again.
So - the erroneous MTU setting appears to have been present prior to Core 153, when the issues started being seen. However, something else seems to have changed starting at 153, such that it also started affecting packets being routed out from internal subnets. That may be a red herring, though - the real question seems to be why the MTU setting is getting put into the routing table in the first place.
I did a quick search (grepping through /etc/ for things like āroute addā, āmtuā, and ā576ā), and while I can find the place in the init.d scripts where the routes are getting added, I canāt see anywhere where the MTU is set. If we can fix that, I think we may have this problem solved.
I have not yet tried upgrading to 159, but I have no reason to believe that I would see anything different from what @bjaminn saw.
Unfortunately, I jumped to 161 today and have gone back to mixed results. Some devices work some devices do not. It seems independent of the ip route mtu setting. I need to drop back to previous versions. Is there a quick and easy way to downgrade instead of wiping and starting over?
Can you add details of what worked and what did not work? Or any errors you might see in a log?
I have only done the wipe out and start over. Hopefully someone will respond with better newsā¦
There are two android phones that one works one does not. Both the same model and the same age. What does not work is the VPN connections that they use to an external service. MS teams sharing does not seem to work. Though i have not tried thoroughly. I will say that everything was working fine after the upgrade and power cycling my router until the second phone came into the network about 3 hours later and then the cable modem needed to be power cycled but the new phone never came back online.
I wouldnāt know where to start to getting anonomized logs that would help with this. As Brian Pruss mentioned i cannot find any reference to mtu 576 in anything even after grepping the whole file system. It does seem clear that setting was tied to the red port connecting and disconnecting. But with my testing today it was not clear that that setting did much.
Hi all,
first, I am sorry to bother you testing something again.
A while ago, @ms found a long-standing bug in Suricata (see this patch and this PR for technical details), which especially caused trouble for some TCP connections on machines having the IDS enabled and being connected to the internet via a slow line (mostly latency, but bandwidth seems to matter as well).
Core Update 162 (testing) comes with this fix. Since the troubles here are related to UDP, it might not actually help much, but please give it that update a test to ensure it at least does not make things worse. 
Thanks, and best regards,
Peter MĆ¼ller
I believe I may have a fix (or at least a workaround) for this problem, and it seems to be related to dhcpcd. After making a manual change to /var/ipfire/dhcpc/dhcpcd.conf , I was able to upgrade to Core 161 and use Google Meet successfully. I am no longer seeing fragmentation on large UDP or ICMP packets.
The change I made was to comment out the āoption interface_mtuā line in dhcpcd.conf under /var/ipfire/dhcpc/. After that, I rebooted, and verified that the āmtu 576ā option was no longer present in the output of āip routeā. I was able to perform large pings from both the Green subnet and IPFire itself.
At this point I upgraded from Core 152 to Core 159. After the upgrade and reboot, I saw that the dhcpcd.conf change had been reverted, so I once again commented out āoption interface_mtuā, rebooted, and tested both pings and Google Meet. All tests were successful. I then upgraded to Core 161. This time the dhcpcd.conf change was not clobbered, and I was able to once again run my tests successfully.
The odd thing about this is, after capturing a DHCP exchange between IPFire and my ISP, I donāt see the MTU option (26) being requested or sent at all. Itās obviously dhcpcd thatās passing the setting to the rest of the system, but I canāt see why itās doing that. It appears to have been doing this since at least Core 152 if not earlier, itās just that some other change in the packet flow appears to have made the route MTU setting affect Green<->Red packets starting in Core 153.
At this point Iām finally able to get current on my router software, so Iām satisfied for the moment. I donāt know what the best way would be to fix this problem in the IPFire core, but Iām willing to be a tester for any changes the developers may propose. Iāll copy the above to the bugzilla thread. Thanks again to everyone that helped.
@prussbw - I am curious: Were you able to try CU 162?
Blog: blog.ipfire.org - IPFire 2.27 - Core Update 162 released
I have not tried CU 162 yet. (Itās a long story and Iāll spare you the details, but I havenāt been able to get time for it.) Based on what Iāve seen and what @pmueller wrote, it doesnāt look like the Suricata update will affect the issue for me, although it may make for a general improvement in performance.
What Iāve found is that the problem goes away (for me at least) when the MTU configuration in the routing rules is removed. That appears to be getting set by dhcpcd for reasons that I have yet to understand, even though Iāve verified that the DHCP server isnāt sending MTU configuration. Manually changing the dhcpcd configuration to ignore the MTU clears up the problem for me, but Iād still like to find a way that the problem could be solved in mainline IPFire.
Wow, thank you for this, I just got my rpi CM4 setup and once again had issues with Google Meets and Microsoft Teams Meetings and this solved my issue.
Oddly enough my point-to-point calls worked fine on Teams but joining scheduled meetings I was unable to share a screen. Commenting out āoption interface_mtuā in /var/ipfire/dhcpc/dhcpcd.conf and a reboot fixed the issue.
Thank you!!!
Hi all,
just for the records, and to have everyone informed:
Upcoming Core Update 165 finally fixes this bug, which is sometimes caused by some cable modems or ISP equipment propagating a very low MTU. It would be great if those of you who are still experiencing trouble could test Core Update 165, and report if it fixes the issue for you as well.
Thanks in advance, and best regards,
Peter MĆ¼ller
I finally upgraded today after I saw the notification that 165 fixes this bug, but unfortunately it did not. 
The workaround suggested by @prussbw worked for me as well! See below.
The change I made was to comment out the āoption interface_mtuā line in dhcpcd.conf under /var/ipfire/dhcpc/. After that, I rebooted, and verified that the āmtu 576ā option was no longer present in the output of āip routeā. I was able to perform large pings from both the Green subnet and IPFire itself.
@pmueller and @ms: I could not get two Meet clients to fuly work with 166 (audio and video). The only difference I noticed was that Meet via Chrome could get audio this time around whereas back on 153 it could not and only Firefox could get audio. So either the client-side code in Chrome improved to handle what was happening or IPFire improved. However, video was still not workingā¦ I tried the workaround above and it worked. 
Please let me know if there is anything I can do to assist in cornering and squashing this bug.
Hello.
Today I did some quick tests.
My ISP provides me with 500/30 Mbps connection.
TEST 1
dhcpcd.conf
Setup->Networking->Address settings->Red
Below the test result from https://www.speedguide.net/analyzer.php
Below the result from the browser on speedtest.pl
Below the result of the test made with the āPRO Speed Testā program from https://pro.speedtest.pl/
Below the test of downloading 10GB file from ftp://ftp. task.gda.pl/test/10giga
Below a fragment of the ip route command result
default via 8x.xx.xx.xx dev red0 proto dhcp src 8x.xx.xx.xx metric 1003 mtu 1500
8x.xx.xx.xx/22 dev red0 proto dhcp scope link src 8x.xx.xx.xx metric 1003 mtu 1500
TEST 2
dhcpcd.conf
Setup->Networking->Address settings->Red
Below the test result from https://www.speedguide.net/analyzer.php
Below the result from the browser on speedtest.pl
Below the result of the test made with the āPRO Speed Testā program from https://pro.speedtest.pl/
Below the test of downloading 10GB file from ftp://ftp. task.gda.pl/test/10giga
Below a fragment of the ip route command result
default via 8x.xx.xx.xx dev red0 proto dhcp src 8x.xx.xx.xx metric 1002 mtu 576
8x.xx.xx.xx/22 dev red0 proto dhcp scope link src 8x.xx.xx.xx metric 1002 mtu 576
Best regards.
Tomas
edit:
Tested on version:
IPFire 2.27 (x86_64) - Core Update 166 Development Build: master/8f696f60
Could you please post some more details about this? The output of āip routeā for example?
Could you please post some more details about this? The output of āip routeā for example?
Hi @ms, sorry for the delay. Reverted the workaround, rebooted, and then ran ip route.
Here is the output of ip route:
default via 7x.xx.xx.xx dev red0 proto dhcp src 7x.xx.xx.xx metric 1003 mtu 576
7x.xx.xx.0/22 dev red0 proto dhcp scope link src 7x.xx.xx.xx metric 1003 mtu 576
192.168.xx.0/24 dev green0 proto kernel scope link src 192.168.xx.xx
192.168.xx.0/24 dev blue0 proto kernel scope link src 192.168.xx.xx
MTU size of 576 bytes strikes again.
And this happens when you force set the MTU to 1500? Could you send the output of āps auxā, too?
No, this happens just on plain installs and updates (post-152). As long as option interface_mtu is left on then this is what happens.
576 is not a random number either as 576 bytes is the minimum IPv4 datagram size that every host shall be capable of receiving. Although that is more of a historical footnote as most modern hosts can handle larger sizes.
I saw this GitHub issue for OPNsense (different as FreeBSD-based) where they were seeing similar issues suddenly on some of their userbase after an upgrade back in 2019: GitHub Issue Link.