Content filtering options

Networking Web Proxy
1

The URL Filter feature within IPFire has been commented on here as “not very effective anymore” with the transition to HTTPS. I have not been able to get it to work reliably.

I am seeking a content filter option that does not require any client side configuration but that works effectively when using HTTPS. Other than using URL Filter, I have found references within this community message board detailing three other options.

  1. DNS server based filtering using Cloudflare, Quad9 etc, in concert with a firewall rule to block clients from using other DNS service providers.

  2. UnBound with a blocklist / blacklist. I don’t really understand how this is different from the URL Filter (Squid) built into IPFire.

  3. IPFblocklist Add-On. This add-on was created and formerly maintained by Tim FitzGeorge. Again, I don’t really understand how this is different from Squid.

This post is very open ended, I would very much appreciate any guidance including where to go to read the details of one of the options or details on the difference between using a web proxy (Squid) vice DNS Resolver (UnBound). Also would like to understand better why using a DNS service that provides content filtering is so strongly advised against by members on this message board.

Thanks, John

1 Like
2

Hey John!

This should help with item #1:


And this is being worked on (I am not sure of its current progress). I thought I read it will be part of the core system.

2 Likes
3

I’m wondering about the technical elements of achieving content filtering using blocklists. Of the options I listed above, each one uses a different process.

  1. Web Proxy (Squid)
  2. DNS Resolver (UnBound)
  3. IPFblocklist Add-On (iptables)

Is any one method theoretically “better” or “more effective” than another?

I think it works loosely like this, but would appreciate any corrections.

  1. The proxy is like a trusted man in the middle, but HTTPS prevents the proxy from “seeing” the URL.
  2. The DNS Resolver blocks identified URLs before they can even be sent to DNS.
  3. After a valid DNS request is returned, the firewall (iptables) blocks connection with the server at that URL.

Any help is greatly appreciated!

4

Ok, part of my understanding seems to be quite wrong. I don’t think IPFblocklist is a valid content filter.

I think IPFblocklist Add-On (iptables) results in something much closer to an IPS in that server connections to the network are blocked. How is this different than using the IPFire IPS (Suricata) with a good ruleset?

5

My education continues. Re-reading mutley’s GitHub page for <dns_blocklist.sh>, this seems like the way to go for me to achieve effective content filtering. It solves the difficulty of URLFilter not working reliably with HTTPS, even if I use WPAD in conventional (non-transparent) mode.

In fact, it seems like it could entirely replace URLFilter. Is there any reason why I would want to keep attempting to use URLFilter in addition to using this script via UnBound?

mut ley, thank you very much for all of your work. I have no idea how much more work is involved and as a new user, it is pretty arrogant for me to ask, but, is there any chance this could be made into an add-on?

6

Hi,

no offense intended, but I think this is a goal which becomes more and more unrealistic.

Ironically, the IT security scene itself has worked towards making this impossible, since technologies such as DNS over TLS/HTTPS, encrypted SNI, and in a way HTTPS are a double-edged sword: On the one hand, they are great for keeping things confidential and improving everyone’s privacy, on the other hand, they make it harder and harder for IT security departments to filter traffic in a centralised manner.

Any DNS-based security mechanism that tampers with DNS responses breaks DNSSEC on systems using it (since the DNSSEC signatures no longer match the actual DNS reply, which has been altered by the service). This is bad, especially because DNSSEC validation errors are usually security-relevant. If they are caused en masse due to a filtering DNS service, it makes spotting any actual attacks rather difficult.

Also, “blocking clients from using other DNS service providers” does not work well anymore: With DNS over HTTPS (DoH), there is a technology available that has been designed to make it hard to spot, and even harder to block. With popular operating systems (such as Windows 11) and applications (such as Firefox in some parts of the world) using DoH, it does not really come as a surprise that malware starts using it as well.

From a firewall perspective, all you see is a TLS connection being established to an IP address on port 443. There is no way of telling whether its HTTPS or DoH. - Bummer, but that’s the beast we (as the IT security scene) summoned. :frowning:

This means the filtering does not happen on a centralised entity, but locally on every IPFire system. While it still introduces lies to the DNS, it is a bit better since IPFire at least can validate DNSSEC beforehand - however, clients behind IPFire will experience DNSSEC validation errors, if they validate DNSSEC as well (which is desirable from a security perspective).

The differences between DNS-based filtering and a proxy are:

  • DNS is application-agnostic, hence not limited to certain protocols such as HTTP(S).
  • DNS filtering is transparent to the client, whereas when using a (non-transparent) web proxy the client knows it talks to a piece of software that can filter the traffic. The latter makes things more easy to debug and troubleshoot.
  • A web proxy has more detailed insight than a DNS resolver, at least on protocols it handles. Additional information such as the client’s user-agent are available as well as additional authentication methods.
  • From a puristic perspective, DNS-based filtering means tampering with a protocol. Web proxies do not have to do that, since they can operate fully RFC-/standard-conformant.

Indeed, it is not, and it was not intended to be one.

As you probably read between the lines, I am not really a fan of DNS-based filtering. But given our current situation, where no accurate blocklist source is available, it does not really matter which protocol we favour for filtering. First of all, we need to find blocklists that are accurate, free and usable for IPFire - the best filtering technique is useless if there is no information on what to block and what to permit.

Thanks, and best regards,
Peter Müller

3 Likes
7

Hello Peter,

Oh my goodness, you are so very incredibly generous with your time (as are many on this board.) I need to read your post a few times to ensure I understand it. I do have a few questions regarding breaking DNSSEC.

  1. How does the UnBound script break DNSSEC? Doesn’t this just drop the DNS query if on a list, otherwise, allows the browser to connect directly to the DNS server?

  2. How does using a DNS service that provides content filtering break DNSSEC? Doesn’t it just do the same thing? It returns a normal value if not blocked, otherwise, returns a null value?

I know that content filtering is more challenging now, which is why I am almost ready to give up on the Web Proxy with WPAD. I just can’t rely on it. I wish there was a more straight forward way to do it.

Thank you, John

8

If I may ask, is DoH hard to spot when using a well known DoH server like Google, or Quad9 or only when using a private DoH with unfamiliar IPs, domain names.?

I see both Windows 11 and the malware mentioned, are using the big DNS providers Google, Quad9, Cloudflare.

Somewhere on Github, I saw a list of DoH servers, I think I counted 300+ IP4s , and 200+ IP6
There was also another list pointing out that the "IP for dns.cloudflare.com is the IP also for cdnjs.cloudflare.com.

Now it is obvious that blocking IP for Cloudflare DNS is not a solution in this case.

9

Nope. Every DNS reply (also NXDOMAIN) must signed by the key of the authoritative nameserver of the zone, if the zone is DNSSec enabled. So you cannot fake a existing domain as not existent without breaking the DNS protocol. There is no “Blocked” returncode in the protocol.

2 Likes
10

Hi,

the REFUSED DNS reply code (not sure if that’s the correct term anyway) always appeared to be as vaguely in this direction. However, I never saw it in conjunction with DNS-based filtering, only if a DNS resolver denied clients by static ACLs.

Also, Unbound does not support it for their RPZ feature, and since its maintainers are quite competent and following RFCs closely, I guess that’s for a reason, but never found the time to ask. :expressionless:

Even further, some DNS errors (such as NXDOMAIN) can be cached, while some others (such as SERVFAIL) cannot. Regarding REFUSED, this would in the worst case cause clients to query the same FQDNs over and over, causing an unnecessary burden to the network infrastructure.

Thanks, and best regards,
Peter Müller

1 Like
11

Oh boy, I can only understand a small portion of the replies. I understand what a DNS service is, but I clearly don’t understand the actual networking involved.

This is my understanding of a client DNS lookup.

Client requests a website → UnBound DNS resolver in IPFire receives lookup request, checks cache, serves IP if there is a cache hit, otherwise, forwards to DNS server → DNS server receives lookup and returns IP to UnBound DNS resolver, which caches the data, and then forwards the IP to the client.

If I use mutley’s UnBound blocking script, how does that break DNSSEC? Is a block list within the DNS Resolver not equivalent to a cache hit? The difference being UnBound will return NXDOMAIN vice the IP address?

Thanks.