Google Meet Stopped Working After Core Update 152->153

Thanks for your help!

The bad news is others are not seeing this issue so it makes it near impossible to fix.

The really good news is you and a few others are seeing it AND it is repeatable. AND that you are a good troubleshooter!

@jon: No, my wife could not connect with her students and other faculty over Google Meet which were not on the same local network. Her camera was functional on her side, but no video would go through to the other clients.

The company I work for uses Microsoft Teams and that was fine. My wife also successfully communicated with friends over Zoom. We also were able to video chat on the same network over Signal with no issue.

We found that clients outside the network were able to hear her audio on Firefox, but still no video - Chrome couldn’t even send audio. The same was true when we started troubleshooting together on Google Meet sessions together on the same network.

My initial kneejerk reaction that this might have something to do WebRTC was due to the fact that I read that Chrome bundles audio and video over the same channel (ICE Candidate) where Firefox uses separate channels (ICE Candidates) for audio and video.

After troubleshooting for a few hours I needed to get her connected to her students again - So I thought about what changed on the network since the Christmas Break and the only thing I had updated was IPFire from Core 152 to 153 and so I backed everything up via the Web Interface and dd-ed a USB Drive with Core 152. After I installed Core 152, reinstalled hostapd, and then restored the backups for both - I jumped in a Google Meet with her and we could see and hear each other again regardless of browser on the same local network. She could also communicate with faculty and students outside the local network without any problem.

Well, I just realised it can’t be tomorrow night because of curfew. Thirsday pm will be the next possible attempt.

I hear you! I do understand 152 works as expected and 153 does not. But for the Devs and people that tested it (like me) all works OK for Core 153. So right now things don’t make sense! I was hoping a LAN to LAN Zoom meeting was the key for the Zoom issue.

As I mentioned in an earlier post Zoom videoconferencing and MS Teams videoconferencing work perfectly for me.

As for Micheal (documented in bugzilla) all works OK with Zoom.

So we keep trying different things until we find the right combo that points us to a fix.

Core 153 started being updated near Oct 27, so there is a long list of changes.

Hello,

Some feedback of the tests I performed yesterday. Well, I was totally unable to reproduce the problem that was reported to me (two lan computer unable to see each other in a zoom session).

I decided to suspect a hardware intermittent failure as it is rather old (~12 years). Since we have for now, because of mandatory remote work, a number of unused, more recent and more powerful desktops, I decided to temporarily move Ipfire’s disk and nics in one of them. I guess that if nothing nasty happens within two weeks, we can conclude in hardware failure, at least in our case.

Note however that the bogus behavior was consistent for more than two weeks on the upgraded version, that we don’t have anymore… But maybe an upgraded install can be unclean, or the upgrade process not perfect.

Update from my side: I finally got some time to set up my VM test bed, but couldn’t reproduce the problem.

I used my backup ISO from Core 152 to create a VirtualBox VM with Green connected to an internal (simulated) network and Red connected to a normal NAT connection. I then connected a regular Ubuntu VM to the internal network so that my IPFire VM was in its path to the Internet, and attempted a Meet call (to my phone), which was successful. I then upgraded the IPFire VM to 153 and tried again, and the Meet call was successful again. I tried again after adding Guardian, but got the same results.

I’m still on Core 152 on my actual firewall machine right now, but I’ll try the upgrade again some time soon and see if I still have the issue. I’ll also keep the VMs around for a bit, so if you have a suggestion on something to try with them, I can give it a shot.

For the moment, here are the PCAP files of the DTLS failures I was seeing when I was running Core 153, and a success case after rolling back to Core 152. (These files have been scrubbed/anonymized with TraceWrangler. If you are an IPFire developer and need to see the originals, feel free to contact me directly through email.) If needed, I can attach them to the Bugzilla ticket as well - let me know if that would be helpful.

153_DTLS_Failure_PCAPs.zip (7.0 KB)

Thanks for everyone’s attention on this so far.

I have the same issue… in LAN we can use video but with people in other side of the red interface there is no video.

Apps that I tested, zoom MS Teams

Simple conf with 4 zones (Green, Red, Blue, Orange) and only outgoing rules for all that zones…
If I publish one service inbound some times it goes fine…

Im rolling back to 152 since I need the share screen to work!!!

This thread led me to think that my problem with the iracing connections dropping may be something similar. We also had zoom / facebook messenger video problems. I rolled back to 152 and my iracing is now working fine; still need to do the complete video call tests.

I have a few new data points, but unfortunately no progress in getting a more-specific characterization of this issue.

Over the weekend, I reattempted a 152->153 upgrade, but got the same results: I was unable to get audio or video from a Meet call. I’ve tried it with counterparts both on and off my LAN, by switching my phone between LTE and Wi-Fi, but didn’t see any difference.

I also tried doing a clean 153 install, and then restoring a backup, but the issue was still present.

If there are any further experiments you’d like me to try, please let me know. I’ve been watching the Bugzilla entry, and the UDP fragmentation issue seems like it could be plausible, so I could try to check for that.

Unfortunately, I don’t have a spare switch with port mirroring available right now, so I can’t sniff the Red port from the outside, but I could temporarily install tcpdump on my IPFire box directly and monitor from the inside. Also, finding a good window for network downtime is difficult right now, so it might take a few days.

How do you do the rollback? fresh install of the 152 or there is another way?

Thanks

I’m not sure this helps, my daughters who use the video call more than I, said point-to-point calls worked; multipoint calls failed video

According to the web you need to do a fresh install to rollback. Take note that configuration backups are different from the add-on backups. I did re-install and restored my backup…(forgot the add-on backup though). Good luck!

Thanks all systems up and running on 152!

1 Like

I was wondering… could it be something related to WPA3 that is the new update???

Still no signs of life or ideas from dev team?

1 Like

Investigations still ongoing. See bugzilla entry https://bugzilla.ipfire.org/show_bug.cgi?id=12563

Therefore i keep standing that update to 153 should not be advised. After 35 days maybe the update should be retired until solution is found and correctly applied.

2 Likes

Does anyone know if in version 154 the problem is fixed? I know version 154 is already available for testing

BUMP! Hoping that someone which updated to 154 tries to spin a fast Meet…

I hope one of the people experiencing the problem might test Core 154 and let everyone know for sure. I don’t think any of the Core Developers experienced the same problems and that makes this very hard to fix.

2 Likes