URL filter for HTTPS


I am running ipfire in a school. The underaged students should be allowed to work and use the web unsupervised, which makes a filter mandatory. The integrated URL filter of ipfire which updates the black lists by itself is great, but soon I realised that it does not filter HTTPS URLs.

So far I have understood that URL filtering for HTTPS is not possible because the proxy cannot see the URLs in encrypted HTTPS packages. I am wondering, would it not be possible to inspect and intercept the unencrypted DNS requests in which the URLs are still visible? Or are the DNS requests not going through the proxy which is doing the filtering? I found this old forum post, where @arne_f said, filtering HTTPS by domains would be possible. That would be fine with me! I would not even need an error page if students try to access forbidden pages. A log of such attempts would be nice but also not mandatory.

If the above is not feasible for some reason, I have read that putting the proxy into non-transparent mode would help? But why exactly is this? Would the URLs in HTTPS packages not be encrypted just the same?


Use non-transparent proxy! It doesn’t work with transparent proxy.

I’ll keep it easy:

transparent proxy: firewall acts just like a visitor
non-transparent proxy: firewall acts like a man in the middle

1 Like

Thanks for the quick reply! Could you direct me to an explanation why this is?

If you use a transparent proxy the firewall url filter just checks the target you want to get connected to. With HTTPS ist encrypted and it can’t check it with your blacklist/whitelist. It just doesn’t know the URL.

With non-transparent proxy you request a connection with the firewall being a part of the chain. So all information go through it and it knows the URL. It works like a NAT.

To understand how it’s working it’s essential that you understand the difference between forwarding and outgoing communication of the firewall.


All right, thanks a lot!

Has anybody ever thought about having a firewall that also blocks requests to the DNS, if the requested URL is on a black list? Or why would that be so absurd?

I think it does, because I use it and I get an “could not resolve address” error if some popup tries to open a browser window with blacklisted websites.

Sometimes I get a “custom blocked error” ipfire website. I don’t know the difference for this behaviour but the result is always the same: blocked. Maybe it’s because I use filter lists and custom blacklists and the ipfire error website just appears for custom blocked websites.

BUT: you may use online services to resolve the ip address of URLs and talk to the servers directly with this ip so your filter may become useless. This is actually a good point to request a upgrade for the URL filter: ipfire should resolve the server ip of any custom black/whitelisted website and block it as well so you can’t communivate with the server even you know its ip address.

1 Like

OK, but this should work regardless of whether it is HTTP or HTTPS, right?

This would in fact be great, because it would then also block HTTPS in transparent mode, as the IP address cannot be encrypted.

With the use of the non-transparent proxy, yes.

Actually I don’t think so. This is related to the difference of forward and outgoing communication/ttraffic.

For stationary devices I don’t see any reason for not using a non-transparent proxy. You need only a handfull ports open für regular internet activities: ICMP, NTP, DNS, HTTP, HTTPS.

So you set your firewall to block normally all communications and open that few ports for outgoing communication. Your firewall is now able to cummunicate with the internet again, but everything else is not. After that you set up your clients to use a global proxy (your ipfire webproxy) so the clients may talk http/https again, but not more. That’s it.

If you use some additional services you may open more ports, but in fact it’s always the same game and you can make sure that you only open doors you need to use.

Would I need to install a certificate on the clients such that the browser accepts my “man-in-the-middle” proxy for HTTPS?

No, there is no additional work needed on the clients.

I am wondering, would it not be possible to inspect and intercept the unencrypted DNS requests in which the URLs are still visible?

Basically, this is what Pi-Hole does.

There is a script that simulates the effect for IPFire: https://github.com/sfeakes/ipfire-scripts
For german readers, here’s a more detailed manual: https://www.kuketz-blog.de/dns-adblocker-skript-fuer-ipfire-ipfire-teil2/

Haven’t tried it out myself, since I use Pi-Hole.

1 Like

Hm, but even an HTTPS message must have an IP address attached to it, such that you know where to send it. If the firewall has a list of forbidden IP addresses it should be able to block the whole domain (like @arne_f said here).

Thanks a lot for the link. Maybe I will give this a try.

What about IPFire’s url filter setting that prevents accessing web sites via their IP address. Will this work for HTTPS in transparent mode?

Hm, but the browser will show a certificate error if IPFire instead of the actually requested web site tries to establish the HTTPS connection, right? If not, there is something wrong with the browser…

No, it won’t.

Maybe it will. I’m not sure and was thinking about it when I wrote my answer and had found a reason why it souldn’t work, but I lost the threat and can’t tell you right now.

No that’s something totally different. A webserver has nothing to do with a proxy server.

I tried it now:

  • Unticked the “Transparent on green” checkbox in the Web Proxy Settings.
  • Clicked on “Save and Restart”.

The only effect is that I now have to setup the proxy (same IP as IPFire with port 800) in the browser to access HTTP sites. But HTTPS can still be reached regardless of whether the proxy is setup or not. The web filter works as before for HTTP but not for HTTPS.

Did I miss anything? I also tried chaning “443 # https” to “#443 # https” in the allowed standard ports and allowed SSL ports. But to no avail…

I do not know what proxy service is used to filter traffic on IPFire but…
HTTPS/TLS Transparent proxy/filter can be implemented in couple levels.
The basic level is the by SNI only.
The more advanced is based on full layer 7 MITM which includes both HTTP and TLS.
It’s very simple to write a SNI proxy with destination domains black/white listing however it wouldn’t be effective as a full blown TLS MITM.

There RedWood proxy at: https://github.com/andybalholm/redwood
Does what most of the sysadmins would want from a HTTP/HTTPS filtering proxy.

I might be able to compile a simple sniproxy filter service in the future but I cannot package it.
If there is a packager then… let me know.

Ah, my bad. Apparently just removing 443 from the allowed proxy ports is not what you would do to actually block the port. I now added a firewall rule that blocks Green to HTTPS Red. The effect is that I can in fact only reach HTTPS pages via the proxy now and if the page is in the URL filter’s blacklist, I get a timeout. For HTTPS pages not on the blacklist there is also no certificate warning like @xperimental said. Thanks again!

That’s why it’s alway a good idea to open any communication ports to the internet, that are acutally only in use/needed. I also wrote something about it in here: Why URL filter is not able to block mobile network?

Oh god, oh god, oh god.

I can only urge you very very very strongly to never break a TLS connection. Never use DNS servers that you do not trust and certainly refrain from using any software like Pi-Hole where the authors do not even understand how badly they are breaking DNSSEC.

We are spending a lot of time here at IPFire to make things as secure as we can. Please do not break it with such bad decisions. At least be aware of what you are doing.

Use the proxy in non-transparent mode. That is all you will need. On top of that, you can enable Safe Search which will come with the next update.