I created a test version of a RPZ add-on and I am looking for feedback

Another hack … if you want it to show disabled for the disabled :upside_down_face:

[root@ipfire sbin]# diff -u rpz-metrics.orig rpz-metrics
--- rpz-metrics.orig    2025-02-06 13:11:24.664758395 -0600
+++ rpz-metrics 2025-02-06 15:09:13.047136262 -0600
@@ -105,9 +105,15 @@

     #  is this RPZ list active?
     theActive="disabled"
-    if grep --quiet "^${theName}\.rpz" <<< "${ucListAuthZones}"
+    line=$(grep "^${theName}\.rpz" <<< "${ucListAuthZones}")
+    if ( test -n "$line" )
     then
-        theActive="enabled"
+        if [[ $line =~ "no serial" ]];
+        then
+            theActive="disabled"
+        else
+            theActive="enabled"
+        fi
     else
         [[ "${rpzActive}" == enabled ]] && continue
     fi

Probably not elegant or complete … just another hack. I’m sure you will find a proper solution by the time this makes into an official add-on.
Cheers, @jon and thanks for all the good work. :slightly_smiling_face:

3 Likes

I submitted the “official” version to the Developers Mailing list a few hours ago. Hopefully I’ve included what is needed for approval.

Keep those fingers crossed!
:crossed_fingers:

(I am still working on your find! - thank you for posting the above!!)

5 Likes

unfortunately this keeps the allow list as disabled in metrics when its enabled. the only true way to show the allow list and blacklist as enabled or disabled would be to read the /var/ipfire/dns/rpz/customlists.conf

below is the conf file with the whitelist disabled

2,off,on
0,*.icloud.com,dns.google,a2z.com,s3-ap-southeast-1-amazonaws.com,*.a2z.com,amazon.com,googleapis.com,sync.afraid.org,*.facebook.com,*.apple.com,*.icloud.com,*.amazon.com,*.microsoft.com,*.nvidia.com,*.roblox.com,*.gvt3.com,*.cloudflare-dns.com,googleapis.com,*.googleapis.com,wix.com,*.wix.com,*.sentry-cdn.com,sentry-cdn.com,*.nist.gov,nist.gov
1

this is with both lists disabled

2,off,off
0,*.icloud.com,dns.google,a2z.com,s3-ap-southeast-1-amazonaws.com,*.a2z.com,amazon.com,googleapis.com,sync.afraid.org,*.facebook.com,*.apple.com,*.icloud.com,*.amazon.com,*.microsoft.com,*.nvidia.com,*.roblox.com,*.gvt3.com,*.cloudflare-dns.com,googleapis.com,*.googleapis.com,wix.com,*.wix.com,*.sentry-cdn.com,sentry-cdn.com,*.nist.gov,nist.gov
1

and last but not least both lists on

2,on,on
0,*.icloud.com,dns.google,a2z.com,s3-ap-southeast-1-amazonaws.com,*.a2z.com,amazon.com,googleapis.com,sync.afraid.org,*.facebook.com,*.apple.com,*.icloud.com,*.amazon.com,*.microsoft.com,*.nvidia.com,*.roblox.com,*.gvt3.com,*.cloudflare-dns.com,googleapis.com,*.googleapis.com,wix.com,*.wix.com,*.sentry-cdn.com,sentry-cdn.com,*.nist.gov,nist.gov
1

with further research i find that the enabled/disabled does not matter for the zone files either because they are either there/not when enabled/disabled


technically if the zone file is enabled its shown in the metrics list. writing the code to show if the allow list/black list is enabled or not is more likely to be where this should go but if your looking at the WUI then that makes it null and void because the check boxes would let you know which state they are in…just my 2 cents

maybe take out the active column? (didnt fixed the spacing properly)

name               hits     lines     hits/line last download             
----------------------------------------
WindowsSpy         1306     359             363 %   2025-02-06
allow              40599    30           135330 %   2025-02-06
test               0        2840              0 %   2025-02-06
ultimate_mini      11431    171567            6 %   2025-02-06
swhocares_hosts    0        11402             0 %   2025-02-06
                   =======           ========                          
Totals -->         53336               186198   
3 Likes

And yet another option :upside_down_face:

--- rpz-metrics.orig    2025-02-08 08:07:44.941998520 -0600
+++ rpz-metrics 2025-02-08 08:46:38.989733190 -0600
@@ -107,7 +107,19 @@
     theActive="disabled"
     if grep --quiet "^${theName}\.rpz" <<< "${ucListAuthZones}"
     then
-        theActive="enabled"
+        case "$theName" in
+           "allow" | "block")
+               if [[ "$(</var/ipfire/dns/rpz/"$theName"list)" == *disabled* ]]
+                then
+                    theActive="disabled"
+                else
+                    theActive="enabled"
+                fi
+                ;;
+            *)
+                theActive="enabled"
+                ;;
+        esac
     else
         [[ "${rpzActive}" == enabled ]] && continue
     fi
3 Likes

this actually works as intended

2 Likes

Thanks @jon for this very interesting add-on leveraging the Unbound RPZ feature, I hope it is going to be included in the standard IPFire installation soon. I would like to ask a question on how it relates to the Squid proxy for content filtering.

From what I know relying on the Squid proxy is the most effective content filtering approach, first of all because it can block website access attempts through direct IP address entering (which would bypass DNS altogether), but also for the reasons explained at section “Take advantage of the web proxy” of the Firewall configuration recommendations for IPFire users guide. Also see section “Blocking sites by using proxy servers” of the OpenWrt Parental Controls guide.

So my question is if the RPZ add-on would provide any benefits when used on top of the Squid proxy, or would only be useful when the Squid proxy is not used for whatever reason.

Short classification of the filtering/blocking features:

  1. The proxy handles HTTP(S) traffic only. There content filtering ( Urlfilter ) can filter those connections only.
  2. IP Adress Blocklists demand that the access is to (fixed) IP. The blocklists must contain all possible IPs for a certain FQDN, and must be up to date.
  3. DNZ RPZ blocks on level of name resolution. Changing IPs are no problem, if the lists are updated frequently ( which is done automagically by unbound itself ).

All three types are based on external lists.
Cases 1 and 2 are well-known since years and the sources of lists are selected in an acknowledge manner by the core devs.
Case 3 is new to IPFire. There is much less experience with the quality of the various lists. First inspections show that unbound has some memory requirements, which are not negligible.

hello board :man_judge:

for everyone interestet on the topic here
i would like to give a recommendation for
an extraordinarily good read: :scroll:
https://lists.ipfire.org/development/02C0C954-80A6-4090-9056-EA114E8CB36B@ipfire.org/T/#t

and for the complete story:
https://marc.info/?t=173885958200004&r=1&w=2

it might help to get a better view :eyes:
have fun :piñata:

@jon
:mechanical_arm:

EDiT:
the original link is dead now.
a new link is replacing the old one.

3 Likes

I installed the test version using Jon’s instructions and restarted unbound without issues.
Versions?

1 Like

Wow, what a read!

Jon, many of us truly appreciate your hard work on RPZ. I can only hope that what, to me, sounded like a very harsh answer was simply due to time constraints, frustration with old projects, or perhaps even a case of talking past each other. I’m quite certain that, deep down, you all want the same thing—a well-functioning IPFire with features that users need and that work reliably.

And I truly believe your RPZ addon will contribute to that, but I’m worried that right now, you just feel like throwing everything in the trash. Please don’t—there are many of us who have been waiting for a solution like this.

Take a few deep breaths, and let’s try to look ahead.

Maybe you could ask as an example Bernhard if he can and would be willing to take the lead in discussing the way forward at the meeting on March 10? What features do they want to offer, and what do they not want to offer? Then we can get a clearer idea of the next steps. First, an entry for the Roadmap? What steps come after that, so we can move forward gradually, step by step?

No matter what Jon, please know that we truly value everything you do. Your work makes a real difference, and we appreciate it more than you know. Hang in there Jon! :heart:

8 Likes

I agree, the work you put in on this and what we tested is a valuable resource for quite a few people here and would be a waste to put it to the side.

6 Likes

@jon
good to see you are still alive and kicking :rocket:
did you notice the loss of information:

-- links below jump to the message on this page --
2025-02-06 16:35 [PATCH] RPZ: update code to include WEBGUI and additional languages Jon Murphy
2025-02-06 19:35 ` Bernhard Bitsch
2025-02-06 20:13 ` Michael Tremer
2025-02-08 18:41   ` jon
2025-02-08 19:27     ` Michael Tremer
     [not found]       ` <C28A0D7E-6C16-4F6F-9366-A8498F40631E@ipfire.org>
2025-02-14 12:07         ` Michael Tremer
2025-02-14 12:58           ` Bernhard Bitsch
2025-02-14 13:52             ` Michael Tremer
2025-02-14 14:16               ` Bernhard Bitsch
2025-03-01 10:18           ` Adolf Belka
     [not found]             ` <3BF29525-C9F4-4FD2-834D-FBE791E99E8C@ipfire.org>
2025-03-02 10:51               ` Adolf Belka
2025-03-10 17:47                 ` jon

source:
https://lists.ipfire.org/development/02C0C954-80A6-4090-9056-EA114E8CB36B@ipfire.org/T/#t

this one is complete:
https://marc.info/?t=173885958200004&r=1&w=2
:person_shrugging:

@board
there is new rpz-movement:

:popcorn:

1 Like

Sorry, I do not quite follow. What will this help with?

The snippet below will help with what @ms is asking for rpz not to do, which in the email chain he said he doesn’t want the plugin downloading the lists over and over again. The info above is not an example of what you can do to check if a remote file has changed since its for ssh only ( brain fart )

I haven’t tested this yet but I was thinking

curl -z -local.html http://remote.server.com/remote.html

Discussed here a bit

1 Like

The RPZ mechanism of unbound does this check internally.
The files contain a SOA record with update frequency and serial number. I suppose, unbound downloads changed files only.

Michael mentions ‘zone transfer’ functions ( AXFR, IXFR ), as are common use between DNS entities.
I haven’t found a solution to use that with the files of HaGeZi (yet). Further I haven’t found a description of the update process.

2 Likes

A bit OT.

How many systems use @jon 's RPZ addon?
Which lists are used?

It would seem to me that it would make sense to use either the ipblocklist list download code, which uses the perl-libwww module for the methods or the ids list download code, which uses the perl-HTTP-Message module for the methods.

Using one of the above approaches would mean we don’t end up with another method of downloading lists.

Both of the code approaches check if the list has been updated since it was last downloaded and will not do a download if no update has occurred.

Also no download is carried out if the time since the last download is less than the allowed download period.

If a download is carried out but fails to result in a completed successfully message then the list will not be installed and used.

1 Like

Hi @bbitsch, not sure if you’re asking us but just in case… I use @jon 's addon with the “Hagezi Multi Normal” list active. This addon replaced a dedicated RaspberryPi running PiHole.

I understand that there are concerns but I hope these can be resolved and @jon 's addon is approved for inclusion in the Pakfire collection.

2 Likes

The RPZ scripts that we created do not download anything. The Unbound RPZ code (that unbound implemented) does the downloading. And I don’t think I should be modifying/hacking Unbound code.

The Unbound RPZ code watches for a change in the SOA line of a RPZ file:

$TTL 300
@ SOA localhost. root.localhost. 1742298960 43200 3600 86400 300
  NS  localhost.
;
; Title: HaGeZi's Pop-Up Ads DNS Blocklist
; Description: Blocks annoying and malicious pop-up ads.

If the serial number changes, then Unbound RPZ code does its thing and downloads. Otherwise there is no download.

The Unbound RPZ code already accomplishes this.

Hope this helps, if this doesn’t make sense please ask.


TL;DR

SOA key

Example:

@ SOA LOCALHOST. ipfire.org. (	
	1			; serial number
	1d			; refresh period
	30m			; retry period
	30d			; expire time
	30			; minimum TTL
)

Can also be written as:

@ SOA LOCALHOST. ipfire.org. 1 1d 30m 30d 30
value key description
SOA TYPE SOA, abbreviation for start of authority
LOCALHOST. MNAME Master Name
ipfire.org. RNAME Responsible Name
1 [1] SERIAL Serial number for the zone (10 digits maximum length)
1d [1:1] REFRESH Query server every N seconds (or min, hours, days)
30m [1:2] RETRY Retry another request in N seconds (or min, hours, days)
30d [1:3] EXPIRE stop ansering after N seconds (or min, hours, days)
30 [1:4] MINIMUM time for negative caching (or min, hours, days)

Links


  1. The NUMBER is an integer. The default is seconds. The suffix is ‘s’ for seconds, ‘m’ for minutes, ‘h’ for hours or ‘d’ for days. ↩︎ ↩︎ ↩︎ ↩︎ ↩︎

2 Likes

Thx, @jon , explaining this further.
I assumed the unbound docs as known by the main devs. :wink:

(I apologise, if this statement was too harsh )