URL filter for HTTPS

I am wondering, would it not be possible to inspect and intercept the unencrypted DNS requests in which the URLs are still visible?

Basically, this is what Pi-Hole does.

There is a script that simulates the effect for IPFire: https://github.com/sfeakes/ipfire-scripts
For german readers, here’s a more detailed manual: https://www.kuketz-blog.de/dns-adblocker-skript-fuer-ipfire-ipfire-teil2/

Haven’t tried it out myself, since I use Pi-Hole.

1 Like

Hm, but even an HTTPS message must have an IP address attached to it, such that you know where to send it. If the firewall has a list of forbidden IP addresses it should be able to block the whole domain (like @arne_f said here).

Thanks a lot for the link. Maybe I will give this a try.

What about IPFire’s url filter setting that prevents accessing web sites via their IP address. Will this work for HTTPS in transparent mode?

Hm, but the browser will show a certificate error if IPFire instead of the actually requested web site tries to establish the HTTPS connection, right? If not, there is something wrong with the browser…

No, it won’t.

Maybe it will. I’m not sure and was thinking about it when I wrote my answer and had found a reason why it souldn’t work, but I lost the threat and can’t tell you right now.

No that’s something totally different. A webserver has nothing to do with a proxy server.

I tried it now:

  • Unticked the “Transparent on green” checkbox in the Web Proxy Settings.
  • Clicked on “Save and Restart”.

The only effect is that I now have to setup the proxy (same IP as IPFire with port 800) in the browser to access HTTP sites. But HTTPS can still be reached regardless of whether the proxy is setup or not. The web filter works as before for HTTP but not for HTTPS.

Did I miss anything? I also tried chaning “443 # https” to “#443 # https” in the allowed standard ports and allowed SSL ports. But to no avail…

I do not know what proxy service is used to filter traffic on IPFire but…
HTTPS/TLS Transparent proxy/filter can be implemented in couple levels.
The basic level is the by SNI only.
The more advanced is based on full layer 7 MITM which includes both HTTP and TLS.
It’s very simple to write a SNI proxy with destination domains black/white listing however it wouldn’t be effective as a full blown TLS MITM.

There RedWood proxy at: https://github.com/andybalholm/redwood
Does what most of the sysadmins would want from a HTTP/HTTPS filtering proxy.

I might be able to compile a simple sniproxy filter service in the future but I cannot package it.
If there is a packager then… let me know.

Ah, my bad. Apparently just removing 443 from the allowed proxy ports is not what you would do to actually block the port. I now added a firewall rule that blocks Green to HTTPS Red. The effect is that I can in fact only reach HTTPS pages via the proxy now and if the page is in the URL filter’s blacklist, I get a timeout. For HTTPS pages not on the blacklist there is also no certificate warning like @xperimental said. Thanks again!

That’s why it’s alway a good idea to open any communication ports to the internet, that are acutally only in use/needed. I also wrote something about it in here: Why URL filter is not able to block mobile network?

Oh god, oh god, oh god.

I can only urge you very very very strongly to never break a TLS connection. Never use DNS servers that you do not trust and certainly refrain from using any software like Pi-Hole where the authors do not even understand how badly they are breaking DNSSEC.

We are spending a lot of time here at IPFire to make things as secure as we can. Please do not break it with such bad decisions. At least be aware of what you are doing.

Use the proxy in non-transparent mode. That is all you will need. On top of that, you can enable Safe Search which will come with the next update.

3 Likes

Hello Michael,

I did some trials and now I don’t understand why I actually need the non-transparent mode to make the web-filter work in HTTPS. It seems that setting 10.16.1.254:800 as the HTTPS-Proxy is enough to make it work regardless of whether in transparent or non-transparent mode.

Am I missing anything?

Micheal means only that you have to specify the proxy in the config and block the direct way for https domain filtering.

The “transparent” field is only valid for http. (It redirect all port80 connections to proxy:800)
There is no need to disable the “transparent mode” for http to catch the traffic of browsers without the config.

1 Like

…and this is done via an extra Firewall rule, right?
Thanks a lot!

One of the easier ways to do URL filtering via HTTPS, if you don’t like taking the proxy off transparency is to block the resolve.

You would do this by forcing all DNS traffic from the LAN to the firewall.

For example:
Edit the firewall.local and add the required entries below the line that reads ## add your ‘start’ rules here

vi /etc/sysconfig/firewall.local

	# Force DNS for green to query the firewall, and not an outside DNS server
	iptables -t nat -A CUSTOMPREROUTING ! -o green0 -p udp --destination-port 53 -j REDIRECT --to-ports 53
	iptables -t nat -A CUSTOMPREROUTING ! -o green0 -p tcp --destination-port 53 -j REDIRECT --to-ports 53

That ^ is my preferred option. It works, and that’s what counts for me.

Alternatively you can try and figure out how to do it in the Firewall GUI, but there you would be on your own….
I guess it could look something like this

Source = Firewall : GREEN
NAT = n/a
Destination = Standard networks: ANY
Protocol = TCP/53 & UDP/53
Accept/Drop/Reject = REJECT
Additional Setting = Remark: DNS BLOCK

Source = Standard networks : GREEN
NAT = n/a
Destination = Standard networks: RED
Protocol = TCP/53 & UDP/53
Accept/Drop/Reject = ACCEPT
Additional Setting = Remark: DNS

Source = Firewall : RED
NAT = n/a
Destination = Standard networks: ANY
Protocol = TCP/53 & UDP/53
Accept/Drop/Reject = ACCEPT
Additional Setting = Remark: DNS

… but as said, no clue on that.
I suppose the iptable entries would end up look something like

	# Maybe checkup, I might not be right :)
	iptables -I INPUT 1 -s <insert your LAN IP range> -d ANY -p udp --dport 53 -j REJECT
	iptables -I INPUT 1 -s <insert your LAN IP range> -d ANY -p tcp --dport 53 -j REJECT
	iptables -I INPUT 1 -i green0 -d red0 -p udp --dport 53 -j ACCEPT
	iptables -I INPUT 1 -i green0 -d red0 -p tcp --dport 53 -j ACCEPT
	iptables -I INPUT 1 -i red0 -d ANY -p udp --dport 53 -j ACCEPT
	iptables -I INPUT 1 -i red0 -d ANY -p tcp --dport 53 -j ACCEPT

And I guess you could even add that to /etc/sysconfig/rc.local …many roads lead to hours of frustration or how the saying goes.
I prefer the redirect option instead.

Any how… Once you now have forced all DNS queries via the firewall, irrespective of what DNS servers the LAN user wants to use, the firewall redirects that query to itself and responds.

Now you could add in either a DNS service that does the filtering for you, i.e. 185.228.168.168 for family friendly, or 185.228.168.10 for adult filter, as an example, I’m sure there are more.

Or alternatively you could go nuts like me and create a script (no longer works under core 142 correctly) that generates a blacklist for the unbound DNS from a source like http://dsi.ut-capitole.fr/blacklists/download/porn.tar.gz or if you already have the URL filter enabled you could just use the source /var/ipfire/urlfilter/blacklists/porn/domains instead. No point in downloading anything if you already have it available for abuse.

As said the script I created seems to break unbound.conf and until I have figured out which part of it is to blame, I cannot provide it.

The nice part about this approach, elegance aside and can be debated, is that you can add additional filtering as needed and unless the user goes nuts with vpn etc. there is no way around it. Unlike the url filter, this method prevents the resolve, no IP, no connection, problem solved.

I hope this helps you.
cheers

For this firewall.local solution you should also define the stop rules!

Oh …oops. Absolutely correct, or you would just keep adding the same custom NAT rule, until you restart the box, on reloads.

To fix that edit …
vi /etc/sysconfig/firewall.local

    ## add your 'stop' rules here
    /sbin/iptables -t nat -F CUSTOMPREROUTING

Leave the ;; in place

If memory services me correctly rc.local only gets loaded during boot time.
This means no stop rule needed, as all rules are flushed and reloaded during a reboot.

What do you mean by this? Is it the same problem as described in this thread : [issue] unbound restart / reboot need to long after update 139->141

I made a script which use the same blacklist, it worked with core 139 but needed modifications for core 142. But with core 142, unbound need half an hour to start…

Nope they seem to be unrelated. However to be honest, I have not actually checked how long unbound takes to come up. All I know is if I go TSL I get

Mar 28 19:25:53 unbound[14097:0] error: can’t bind socket: Address already in use for 127.0.0.1 port 8953
Mar 28 19:25:53 unbound[14097:0] error: cannot open control interface 127.0.0.1 8953
errors.

Which is odd, as nothing is running on port 8953, apart from TLS when enabled that is
UDP fails over time, as though it is degrading, swapped to TCP which resolved that problem.

Regarding the script, it had the same results, irrespective of the network protocol I chose to use.
It seems to hang during a specific process, when I kill it and try to do a /etc/init.d/unbound restart, it complains about unbound.config being in use. So unbound fails to load, and then only way to get out of this is to delete the unbound-blockporn.conf file that was created, and edit the /etc/unbound/unbound.conf file and remove the reference to it. Doing a /etc/init.d/unbound restart then actually works.
This tells me something with the way the script creates the blockporn file is borked.
As said, I have been editing the file since I discovered it (when I get time), but yet have to find a fix.

I would post the script, but it’s off topic. I also think there may still be a copy of it on the old forum page.

I had the same Problem: Long delay at startup, plus i do an unbound URL-Filtering too. I did not developed a custom script, but used the one Mike Kuketz recommended here.

My Problem was a wrong configuration in the DNS-Settings, you may read further here. After i switched the “Protocol for DNS queries” to “TCP”, all my Problems are gone especially the long delay at Unbound-Startup.

Stay healthy!

Regards,
Olaf

I’m an idiot by the way… I kept using the wrong port for DNS over TLS.
Not port 8953
It should be 853

yes it’s working now…duh! :shushing_face: