Unable to connect to the internet on HTTP/80

Networking
1

Hello,
A Solar Inverter Device in my home recently stopped reporting to it’s Cloud.

After some investigation I found that it tries an API request over HTTP (80). Ironically this API request just returns a redirect to 443, but without the first request working the device does not work.

While looking I found:

  • I can make requests to 80 from a shell on IPFire itself.
  • I cannot get to HTTP/80 at a variety of sites from any device inside my network.**
  • Connections are dropped instantly, like they are caused by a firewall denial not the proxy.
  • I haven’t put in any such firewall rules.
  • There are no firewall rules related to the events in the logs
  • There are no denials in the IPS log and even “whitelisting” the source or destination(s) doesn’t resolve the problem

** sites tested with HTTP from both curl and a browser include:

* reports.enphaseenergy.com
* api.publicapis.org/entries
* neverssl.com
* catfact.ninja/fact

I haven’t changed anything recently, but the problem only happened recently, perhaps since the last IPFire core update.

Has anyone seen this problem?
Could you please help me diagnose where this is being blocked by IPFire? (It isn’t obvious!)

Thank you!
dnl

2

Seems my Raspberry Pi’s have not been updating due to this problem too.

PS: I should mention that I’m using the IPFire proxy in what I understand is the default ‘transparent’ proxy configuration.

3

I am running Core Update 186. I am not seeing any problem as described.

The neverssl.com site is http but the other three are all https and when I tried http they automatically re-directed to https.

The only one that I could not connect to was the api.publicapis.org and that was due to that site not redirecting properly. This was with both http and https. There was a comment that this might be due to my cookie settings but I am not going to change those. I have other sites that do too many redirects and those are also blocked by my Firefox browser.

Just in case could you try to run location update
There has been an issue as per this post onwards.
https://community.ipfire.org/t/squid-was-not-started-at-rpi/11528/2

The squid issue was related to a problem with a specific location database.

I think it is worth just testing with the latest location database just in case that is what is causing your problem.

If that is your problem then you would be seeing messages about it in the
/var/log/squid/cache.log

If there really is nothing in any of the logs for the dropped traffic then the problem might not be with IPFire.

Has your ISP changed something recently.

1 Like
4

That makes me suspect that you are suffering from the problematic location database.

If it is then if you check on the WUI - Status - Services then you would find that the Web Proxy is Stopped.

You would also need to have the Fast Flux and/or the Selectively Announced Networks options check-boxes selected.

3 Likes
5

Thank you @bonnietwin the Web Proxy is in fact stopped!

I’ll look in to that location database issue you mentioned soon and reply back if it resolves the problem.

From memory I had both the Fast Flux and Selectively Announced Networks options check-boxes selected.

6

Then i am certain the location update command will fix your problem.

After running it you will need to press the web proxy Save and Restart button.

7

Thank you again @bonnietwin for the fast response with this one! You were totally right.

I never suspected the proxy was down because HTTPS traffic was unaffected! This has made me wonder how much the proxy does, especially as IPFire is NATing internal traffic (hiding which internal IP is making an external web connection).

I was just reading this thread and was surprised to learn that the proxy only carries HTTP connnections, when configured as a transparent proxy. I guess it makes sense as trying to transparently proxy a HTTPS connection would result in a MITM issue with certificates I think!

This would explain why I’ve found the URL Filter to be often ineffective.

So please correct me if I am wrong but the Proxy in IPFire adds the features of:

  • Proxying and caching of unencrypted (HTTP) traffic
  • URL Filter
  • Time Restrictions
  • Transfer Limits/Download Throttling
  • Anomaly Detections

So for me, if not for the URL Filter and Anomaly Detections, I don’t see why I’d need a Proxy in 2024.

I’m going to consider blocking all HTTP/HTTPS traffic and manually configuring the Proxy on my devices.

8

PPS: Thanks again @bonnietwin for the www.ipfire.org - Web Proxy Auto-Discovery Protocol (WPAD) / Proxy Auto-Config (PAC) wiki page. Very helpful!

1 Like
9

Glad you got things working again.

That is correct.

Even with https traffic you can still do the proxying and it ensures that any user agent info in the client machines is not sent out as the web proxy stops any of that info being provided.

I believe that the anomaly detections will still work with https traffic as I think those capabilities are at IP level and not down at URL level.

Currently I still have the Web Proxy running but it is much more limited on what it can do with https traffic.

Other people provided most of the content of that page. I just updated the DNS status for Firefox as I was able to confirm that it now can work with both DHCP and DNS.

1 Like