Proxy crash / Squid issue

Hey IP-Fire Community,

I had some problems with my Proxy but thanks to the very good documentation on the IP-Fire i had it under control.

After the Upgrade to 183/184 i had some Proxy/squid Problems.

Under command in shell Squid -k parse there seems all fine.

But no sites will open after a while after reboot.

After it reached the maximum Bytes of Cache, then no sites wanted to open anymore.

After I put the recommendation in the Documentation of IP-Fire it helps. →

Can someone explain why it is so? I think because when the cache gets full then the squid no longer accepts to open any sites anymore.

Is there a good Documentation that helps me to understand it better how it works and where i can find the Documentation?
How the data is processed?

And what this and more points mean? Some of this i understand but not all. Maybe there is a good link what explains all this. →

Screenshot 2024-04-17 at 14-03-26 CacheMgr@192.168.100.254 menu

Maybe it wasn’t this Problem. Maybe something of my fault?

Thanks Sergio

I’ve been having this issue as well.

I have just tried reducing cache size to 1024MB RAM and 1024MB Disk to see if it will crash less often, but my old settings were 1280 RAM and 2000 Disk and it would become unresponsive after 3-10 days with a similar spike in Number of Processes and that started after the 183 or 184 CU for me, too.

The rest of my settings are the same as OP’s but my max object size is 4096 kb.

Anything in the log files?

The logs are literally massive, so I really don’t know if there’s an easy way to look for what you want me to find, but here’s a snippet:

00:51:59 192.168.1.93 - http://192.168.1.1:81/redirect.cgi? 00:51:59 192.168.1.93 - http://192.168.1.1:81/redirect.cgi? 00:52:10 192.168.1.84 - http://connectivitycheck.gstatic.com/generate_204 00:52:24 192.168.1.80 - http://connectivitycheck.gstatic.com/generate_204 00:52:35 192.168.1.82 - http://connectivitycheck.gstatic.com/generate_204 00:52:35 192.168.1.82 - http://connectivitycheck.gstatic.com/generate_204 00:52:41 192.168.1.76 - http://connectivitycheck.gstatic.com/generate_204 00:52:48 192.168.1.81 - http://connectivitycheck.gstatic.com/generate_204 00:53:08 192.168.1.14 - http://firetvcaptiveportal.com/generate_204 00:53:10 192.168.1.84 - http://connectivitycheck.gstatic.com/generate_204 00:53:24 192.168.1.80 - http://connectivitycheck.gstatic.com/generate_204 00:53:34 192.168.1.87 - http://web.diagnostic.networking.aws.dev/4c16ec46-b9da-49b2-... 00:53:35 192.168.1.82 - http://connectivitycheck.gstatic.com/generate_204 00:53:36 192.168.1.82 - http://connectivitycheck.gstatic.com/generate_204

Although something I found that might be interesting is that when I loaded Epic Games, the number of processes spiked and I saw a lot of entries like this:

15:35:50 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:50 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:50 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:50 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:50 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:50 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:50 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:50 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:51 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:51 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:51 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:51 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...
15:35:51 	192.168.1.32 	- 	http://fastly-download.epicgames.com/Builds/Org/o-e2xslu4mlq...

Those disappear after a while but the process memory remains spiked afterwards.
getrrdimage

Try to turn it off the Cache-Manager in Version 185.
When I read it right, there is the new Squid Version 6.9.
In 183 and 184 I had many Proxy crashes. With the Cache-Manager I had it somewhat under control.
But with Version 185 you don’t need it anymore. Unless you really want it.

I tried unchecking the cache manager, since I wasn’t using it. It didn’t change anything as far as performance. At least with the cache size smaller it doesn’t seem to go unresponsive like before… so far (crosses fingers).

It went unresponsive again today. Cachemanager was disabled. When it gets like this I can’t even get the webproxy to stop in the WUI. I uncheck both transparent and green, then hit save and reload and the page never finishes loading, the services page still shows it running. I have to reboot the whole box to get it to shut down and come back up.

Do you have any messages about that in /var/log/messages and/or /var/log/squid/*.log

As a sideways thought, can you watch the packets with tcpdump? I’ve seen squid brought to its knees before with weird packet storms.

It doesn’t work with mine either.
Sry for my wrong hint.
Activate your Cache-Manager again and Try not to cache these domains which makes much traffic like epic games.

I make for you an example:

But I don’t know if it will work like this.
Maybe someone knows it better than me?

Question:
Must there be a star like *.epicgames.com or will it automatically include all subdomains?

Or should he write the hole domain like this?

fastly-download.epicgames.com

Maybe this will help.

I’m not really sure what I’m looking for in those. They are both massive with lots of errors and warnings (drops, dns lookup errors) that I’m pretty sure are common. I can at least parse through the squid one for stuff that stands out in the noise and found.

2024/05/14 15:47:24| WARNING: BCP 177 violation. Detected non-functional IPv6 l>
2024/05/14 15:47:24| aclIpParseIpData: IPv6 has not been enabled.
2024/05/14 15:47:24| aclIpParseIpData: IPv6 has not been enabled.
2024/05/14 15:47:24| aclIpParseIpData: IPv6 has not been enabled.
'

and

May 14 18:21:51 squid-asnbl-helper[9675] WARN: No ASNBL configured. This is acc>
May 14 18:21:51 squid-asnbl-helper[9683] WARN: No ASNBL configured. This is acc>
May 14 18:21:52 squid-asnbl-helper[9679] WARN: No ASNBL configured. This is acc>
May 14 18:21:52 squid-asnbl-helper[9684] WARN: No ASNBL configured. This is acc>
May 14 18:21:52 squid-asnbl-helper[9674] WARN: No ASNBL configured. This is acc>
May 14 18:21:52 squid-asnbl-helper[9678] WARN: No ASNBL configured. This is acc>
May 14 18:21:52 squid-asnbl-helper[9680] WARN: No ASNBL configured. This is acc

which look like they stand out and may have happened around when it went non-responsive.

Which parameters would you recommend for Wireshark?

TBH, I am not sure as it is years ago that I ran into the problem. For a start, just monitor a single PC’s source IP and then narrow it down:

tcpdump -nni green0 host 111.222.333.444

My grey cells seem to also remember the proxy log was helpful and contained a lot of escaped characters.

AFAIK, if you’re using the disk cache, your cache should be on a disk about 16x faster than your internet link. With modern internet links, this generally means SSD or it will slow your connection. A SATA3 connection has a maximum bandwidth of 6000Mb/s so if your SSD could run at that max speed, it would start impacting your connection if it is >= 375Mb/s. SATA2 is only 3000Mb/s and plain old SATA, 1500Mb/s. For line speeds beyond that, you’ll want the cache on an NVMe SSD.

it is often better to run without a disk cache.

2 Likes

I’m not really interested in the cache part so much as the URL filter, so I set both cache sizes to 0 and the min and max object size to 0.

Is there a better way to run a proxy without a cache for the URL Filter? I’m curious to see if zero cache means that Squid won’t crash again.

Is it possible to have a RAM cache without an HDD cache? I thought I set the RAM cache larger than the HDD cache earlier and it kicked out an error saying that I couldn’t do that, but it looks like I can set a RAM cache with a HDD cache of 0 now, so I’m confused.

When @nickh wrote that maybe the internal SSD is not fast enough to save the cache on the Disk, my thoughts were the same to put the Disk Cache to Zero. Im testing some Scenario. This will be one of them. Maybe it will work better.

And when it is really possible to work like this, then maybe it is possible to put the maximal Object size to zero? To not to Cache.

I don’t know if it’s recommended to work like this. For me I only need the Web Proxy for the URL-Filter.

The thing that I don’t understand, first with the new Version 183 the Problem started. :confused:

And @phane you can try not to use your System-Proxy. You can put the Proxy-Information into your Browser like Firefox native or Google Chrome, Brave, Edge with the Proxy Switcher Addon.

When I was using squid, it was with v3.5.20. Not having a fast-enough SSD just slowed the internet connection. It did not cause any crashing of squid itself.

I am trying to remember my tcpdump diagnosis. I seem to remember I was trying to see if there was a traffic storm so the tcpdump query was not too important. It becomes obvious if there is a storm.

Since setting the cache to zero, my memory usage has stayed flat at around 50 mb and I’ve seen no spikes. URL filtering still seems to be working, so it’s probably a cache issue that, like others mentioned, started with CU 183.