Need help with network issue - Maybe squid related?

I subscribe to a streaming service that is experiencing an issue. I disabled the URL filter, the Intrusion Prevention System, the QoS while trying to figure this out.

This has been happening since March 2022.

I’ve been going back and forth with the vendor and they are pointing the “blame” finger back to me (actually my network).

I’ve dug through the various logs and I don’t see anything noteworthy. But I did see LOTS of these errors in the squid access log starting on Mar 12:

1647144914.348    683 192.168.60.218  TCP_MISS_ABORTED/100 0 
 PUT http://name-changed.s3.amazonaws.com/SID_1234567890/123456/data.ts -
 ORIGINAL_DST/52.217.123.123 -

There are many TCP_MISS/200 to a different website but I think those are OK.

From decoding the message it looks like it tried to connect for 683 seconds but transferred zero bytes of data.

From the date stamp it looks like this started on March 12 which is a day after I upgraded to Core 164:

Mar 11 21:27:10 ipfire pakfire: CORE UPGR: Upgrading from release 163 to 164

My questions are:

  • Is the TCP_MISS_ABORTED/100 something that is caused by squid?

  • Is there something I enabled on IPFire that should be disabled?

  • Is there something squid related that might cause this?

If you allow your client to get directly to port 443 so it doesn’t use squid, does the stream work?

Everything should be open. And If it was closed I am hoping I’d see it in the firewall logs.

Screen Shot 2022-06-17 at 1.00.07 PM

I am talking about a rule to block the lan from going directly to the red interface over port 80 and 443. Do you have such rule? Is it also possible that the client that receive the stream is set to use your IPFire as a proxy? If yes, maybe you can remove that.

No port 80 or 443 rules.

(I’ll double check to make double sure)

And the proxy is in transparent mode. So I don’t think that is an issue.

This should not matter for streaming, as that traffic is encrypted and it will not go through the transparent thing. It should not involve Squid unless the client where you watch the streaming is set to use the proxy. I mean, If you watch the stream service using a browser and this is set to use the proxy, then it will engage squid.

EDIT: is it possible that part of the protocol to establish a streaming channel involves also the use of port 80 and somehow the transparent proxy, which is basically a man in the middle attack just not malicious, causes the dialog to fail?

Your EDIT is the big question!

I always assumed (I hate that word) that transparent mode I wouldn’t need to worry about the proxy. But for some odd reason I see TCP_MISS_ABORTED/100 messages and I am guessing that is a bad error.

I had thought about adding the IP address of my streaming device (the local network IP address) to the Unrestricted IP addresses (one per line) box. Maybe I’ll get lucky!

If it is the transparent proxy the problem I do not think the ACL will help. I would disable the transparent proxy temporarily just to test the hypothesis. If it is indeed squid due to the transparent proxy, maybe you can se the “do not cache these domains” field?

1 Like

good idea - I’ll try that second!

In the back of my mind I am thinking about re-installing Core 162 to see if that solves this odd issue. This will be my last ditch effort! I am wondering if a squid change may have happened before mid-March.

No joy!
:pensive:

No joy!
:pensive:

And still the same errors in squid access log (/var/log/squid/access.log)

I disable the transparent proxy and it all worked as expected. Now there are no TCP_MISS_ABORTED/100 messages! :smiley:


But why?!?

Because some applications don’t work well with a proxy.

I used to have a problem accessing my son’s Plex Media Server with my proxy enabled. Plex would come up with an error message when trying to show what was on the system.
The Plex FAQ help info said “don’t use a proxy on your system. Turn it off”

In the end my son was able to find out how to fix the issue but from searches it was clear that Plex doesn’t work well with a proxy.

I doubt thst it is the only app with a problem with proxies unfortunately.

3 Likes

Don’t know why, just what happens. Correct me if I misunderstand. Your client try to connect to the server to start a streaming session, it gets intercepted by squid on port 80 that takes over and initiate a new connection on behalf of the client. Squid waits for an answer to the TCP session but never gets one or it fails to get one and add it to the cache (the miss part of the message error). After 683 sec it gives up and abort the connection. Do you agree that this is what happens?

Now, the iana error 100, is a continue header that the client receives from the server. In this case the client is Squid. So, do I understand correctly that it is squid that does not continue with the connection?

quote:

When the request contains an Expect header field that includes a 100-continue expectation, the 100 response indicates that the server wishes to receive the request content, as described in Section 10.1.1. The client ought to continue sending the request and discard the 100 response

If all this is correct, why does squid fail to say “please continue”? I don’t see this happening. Rather is it possible that squid try to send this message, but it is the firewall that gets in the way and squid never manages to send successfully a continue header, or to receive back the answer from the server? Should this be visible in the kernel logs?

1 Like

Would your son be willing to share his fix?

This seems correct.

I think so - this is all above my understanding and skill level :exploding_head:

This is next to nothing in the kernel logs. My device is in the GREEN0 network and not the BLUE0 network.

I keep going back to this thought since all of my “issues” started on March 12 the day after the update to Core 164… Time to try a rebuild!


EDIT: Squid had a big update in core 164.
Core 163 - update to squid 4.16
Core 164 - update to squid 5.4.1

1 Like

Can you white list some of the streaming services ip addresses?

I think I did that here:

Is there another place to whitelist?

Not sure if this will help.

Note comment about using ip not url.?

1 Like

You have to read the wiki on this bit very carefully. I had originally thought the same as you that it would give unrestricted access without the proxy but in fact it means that it is unrestricted in terms of the time limits or transfer limits or the MIME type filter sections near the bottom of the Web Proxy WUI page.

That is no problem to share at all but it was specific to my plex proxy problem so not sure it will help you. We added port 32402 to the SSL ports section on the web proxy. My problem was access never being allowed to the specific web page while you have access for a period and then the access gets blocked.

What might help is the method my son used to find out that the port was the problem.
He opened the error/debug windows on my browser and then did the web access that gave a problem and then he read through the error/debug windows and was able to see that the communication was being stopped because the port was not accessible.
However I would say that my son could read through the browser error/debug windows very quickly and find the problem bit while I was still trying to read the first line. He does computing software work for his living, unlike me where it is just a hobby.
The challenge would be to find the appropriate error message(s) and then to figure out how to overcome that in the Web Proxy.

In Seamonkey the windows he used were accessed by the menu sequence Tools - Web Development and then the Error Console and Browser Console options.
In Firefox it is Tools - Browser Tools and then selecting Web Developer Tools and Browser Console.
Not sure about how to access the equivalent on Safari.

I when thru the squid access.log and found 1037 different IP address. So that may not be easy unless I can do multiple ranges like:

52.216.*.*
52.217.*.*
52.231.*.*

I’ll look into it!

I wish I had this skill! My skill level is more hobby level (similar to you but steps lower for me!).

The streaming device sends the http://name-changed.s3.amazonaws.com/SID_1234567890/123456/data.ts and I am not sure I can do the same with a browser. But I will give it a try!


Just to go in a different direction for a little bit…

Why is a transparent web proxy needed?

I am guessing it allows me to enable the URL Filter and the Update Accelerator for http (port 80) items. And it allows web page caching of http (port 80) items (e.g., images, etc.). But does it provide other functions?

What different (or bad) might happen if I don’t enable it?

Here is my current settings as of yesterday to get this streaming to work.