Customise DNS Firewall?

[praful]

I used to use this script to update Unbound blocklist. I used the blocklist on https://big.oisd.nl/. This list was designed to pass the “girlfriend test” (no complaints from girlfriend about websites/apps not working!). For over 3 years, it worked perfectly for me. No false positives. No broken websites or apps.

With the IPFire update 203 (Unbound replaced by Knot Resolver), I’ve had to switch to DNS Firewall and ticked no adverts. Already the NY Times app and website are not working properly eg NY Times app constantly reports “Something went wrong. Please try again later”. The website menu doesn’t work.

I could probably debug this and find the URL that needs to be whitelisted but I suspect this will be a recurring issue. I’d rather just go back to the OISD list.

I can see you can add URLs on the ipfire admin page for DNS Firewall. However, the OISD list changes daily. Is there a way of plugging in your own blocklist and have it updated periodically (hourly, daily, etc)?

[brackenhill_mob (Philip de Lisle)]

I second this, as I too was wondering how to add our own lists. I’ve just started to set up IPFire after 15 years using OpenWrt. I currently use Technitium with blocklists from Hagezi and Steve Black.

From my tests so far, DNS Firewall is far too restrictive for my needs - it certainly fails the “family test” for me. For example with advertising and malware turned on, my local fast food restaurant and several of my banking apps are blocked - in most cases the block is caused by their site having a link to a very well known CDN that is in the malware list which is way too strong IMHO.

How do we set our own lists? I want to continue using the ones that have worked for me for many years.

[praful]

Looking at this page, it seems as simple as

wget https://o0.pages.dev/Lite/rpz.txt -O 1hosts-lite.rpz

wget https://big.oisd.nl/rpz -O osid.rpz

sudo mv *.rpz /etc/knot-resolver/

Then setting Knot Resolver to update on RPZ file change (if it’s not already).

I read that Knot Resolver has been customised in ipfire. Will the above wget/move to /etc/knot_resolver/ work with the current version of ipfire?

[pscar13]

@praful

I read that Knot Resolver has been customised in ipfire. Will the above wget/move to /etc/knot_resolver/ work with the current version of ipfire?

No, that won’t work on IPFire that way.
The RPZ files are located in /var/lib/knot-resolver/zones/
And the zones are reloaded via a Lua script.
You can try and adapting this experimentally (not in production!).

wget https://big.oisd.nl/rpz -O osid.rpz
mv osid.rpz /var/lib/knot-resolver/zones/osid.rpz.zone
chown knot-resolver:knot-resolver /var/lib/knot-resolver/zones/osid.rpz.zone
echo osid.rpz,on,,,, >> /var/ipfire/dns/dnsbl
/usr/local/bin/dnsctrl reload

But it will be overwritten by a configuration change on the dnsbl.cgi page.

Edit :
It is not safe to use scripts found on the Internet (even on GitHub) if you do not know how to read the code and understand what you are doing!

[praful]

Many thanks @pscar13!

For others, in case it’s useful, here’s what I’ve done.

I’ve done some testing and this works. The script is:

#!/bin/bash

cd /tmp
wget https://big.oisd.nl/rpz -O oisd.rpz
mv oisd.rpz /var/lib/knot-resolver/zones/oisd.rpz.zone
chown knot-resolver:knot-resolver /var/lib/knot-resolver/zones/oisd.rpz.zone
echo oisd.rpz,on,,,, >> /var/ipfire/dns/dnsbl
/usr/local/bin/dnsctrl reload

The script is scheduled daily by being stored in /etc/fcron.daily/. Make it executable (chmod a+x ).

There was a typo on the website I linked (“oisd” became “osid”), which I’ve corrected for consistency.

As @pscar13 said, make sure you’re happy with the script before using it!

To test, I disabled all the ipfire blocklists on the DNS Firewall admin page. Then, from a client, I ran:

dig <domain-in-blocklist> <ipfire-ip-address>

For example, if you’re ipfire IP address is 192.168.1.1 and you want to check if zowunee.com is being blocked, run the following on a client:

dig``zowunee.com`` 192.168.1.1

At the top, you’ll see something like

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 23894

The status will be NXDOMAIN or REFUSED if the domain is being blocked.

You can check a working address, eg

dig ``ipfire.org`` 192.168.1.1

which will return something like:

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7694
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;ipfire.org.			IN	A

;; ANSWER SECTION:
ipfire.org.		7037	IN	A	81.3.27.38

Notice the status is NOERROR.

As @pscar13 said, if you change the blocklist via the DNS Firewall page, the settings in /var/ipfire/dns/dnsbl will be lost. You can wait until the next run (if you scheduled it via cron) or run the script manually. I’ve disabled all the ipfire blocklists and use just OISD, which has served me well for many years.

[brackenhill_mob (Philip de Lisle)]

@praful

echo oisd.rpz,on,,,, >> /var/ipfire/dns/dnsbl

I haven’t tried this script yet, so my question might be irrelevant but…

If this script is run every day, won’t there be multiple instances of this line in
/var/ipfire/dns/dnsbl? Or is there some other fcron script which cleans this file daily?

[praful]

@brackenhill_mob Thanks I realised this earlier today. The story is more complex. I hope someone with more knowledge can help. The story so far, which provides a working solution.

I updated the script to:

#!/bin/bash

# !!! DO NOT USE!!!

if wget -q -O /var/lib/knot-resolver/zones/oisd.rpz.zone https://big.oisd.nl/rpz; then
  chown knot-resolver:knot-resolver /var/lib/knot-resolver/zones/oisd.rpz.zone
  grep -qxF 'oisd.rpz,on,,,,' /var/ipfire/dns/dnsbl ||
      echo 'oisd.rpz,on,,,,' >> /var/ipfire/dns/dnsbl
  /usr/local/bin/dnsctrl reload
else
  echo "Failed to download OISD RPZ" >&2
  exit 1
fi

The script downloads to zones folder and if successful updates dnsbl if the line doesn’t exist.

However, after I wrote the script, I checked dnsbl this morning. I expected to see two “echo” lines but instead there were none! Something is overwriting dnsbl.

On further investigation, dnsbl.json seems to be the driver for what goes in dnsbl. I updated it to include:

    {
            "name"        : "OISD",
            "zone"        : "oisd.rpz",
            "primary"     : "localhost",
            "description" : "OISD blocklist",
            "license"     : "GNU General Public License v3.0"
    }

This results in the DNS Firewall page showing OISD as an option. I tried setting “primary” to “big.oisd.nl/rpz” and ran /usr/local/bin/update-rpzs but that didn’t work. update-rpzs uses zone-syncbut I couldn’t get it to use the OISD url. Setting to “localhost” also fails (obviously) but I’ve decided to use this modified script, which just downloads the rpz file but doesn’t update dnsbl:

#!/bin/bash

if wget -q -O /var/lib/knot-resolver/zones/oisd.rpz.zone https://big.oisd.nl/rpz; then
  chown knot-resolver:knot-resolver /var/lib/knot-resolver/zones/oisd.rpz.zone
  /usr/local/bin/dnsctrl reload
else
  echo "Failed to download OISD RPZ" >&2
  exit 1
fi

I couldn’t find what’s updating dnsbl.

Obviously this is a real hack because dnsbl.json will be overwritten – perhaps on upgrade, perhaps every day! I’ll see what happens tonight.

[pscar13]

The information I shared was intended for experimental purposes only.

Modifying IPFire’s behavior may lead to unexpected side effects.

The files /var/ipfire/dns/dnsbl and /var/ipfire/dns/dnsbl.json are used by dnsbl.cgi according to the selected configuration.

With the current implementation, adding another blocklist cannot be done without modifying that page (for example, by adding a form to manage additional lists).

This is more complex than just writing a simple Bash script.

[praful]

@pscar13

With the current implementation, adding another blocklist cannot be done without modifying that page (for example, by adding a form to manage additional lists).

Thanks @pscar13. I appreciate the help you’ve given so far.

dnsbl.cgi seems to generate the DNS Firewall page by using dnsbl.json (for the list of blocklists) and dnsbl (whether selected or not).

So just changing dnsbl.json, resulted in a different form:

But I couldn’t get the existing mechanism to download the OISD rpz file. The help forzone-syncused by /usr/local/bin/update-rpzs wasn't clear enough for me. So, I’m using a script to download the rpz file.

Having got used to no ads (prior to 203), I’d like to get this working. The designers of IPFire didn’t add an obvious extension mechanism. So what I’ve implemented is a hack. I’ve raised this as an issue on bugzilla.

[praful]

The good news is that the above approach created with @pscar13’s help works. I’ve had it running for a few days. My next step is to update the script so that if the json file is amended (eg after an update), the script continues to work.

The bad news is that after I raised this issue, one of the developers immediately closed it with “WONTFIX”, saying “We consider our approach much better and invite everyone to contribute to improve them further”.

My guess is that the developers believe that if they provide a custom blocklist feature, users will use that instead of trying to improve the IPFire blocklists. If the IPFire team think that maintaining blocklists is a good use of their time, that’s fine but why force users to do the same?

If they want to provide blocklists, they should be minimal, with very few false positives, like the OISD one. As it happens, the current ones are not fit for purpose. After enabling ads, malware and phishing, the New York Times website and android app stopped working. If the rules don’t work for probably the biggest newspaper in the world, what’s it going to be like for smaller sites/apps?

An irony is that the IPFire DBL lists are provided in various formats for others to consume, including https. But IPFire won’t consume https feeds itself! Also, the IPFire URL Filter feature, which is similar, has comprehensive custom blocklist support.

On the IPFire home page, the motto is “Your network. Your rules. No strings attached.” So someone in the team (the marketing people?) knows that the best software empowers users then gets out of the way. Maybe they need to change the motto to “Your network. Our rules. We know better.”

I thought the target audience for IPFire was more than hobbyists/home users. If so, as I replied in bugzilla, some organisations have internal RPZs, commercial threat feeds, or other specialised sources that aren’t appropriate for inclusion in the IPFire DBL. I say this as a former CTO of a finance company.

The current choices I can think of are:

1. Try to correct the IPFire blocklists: some people and organisations probably won’t do this because they’ve been fine-tuning their blocklists for many years.

2. Use other software, eg Pi-hole or another firewall that provides custom blocklists.

3. Hack IPFire. Fortunately, whoever designed the DNS Firewall feature made it extensible from the backend. A bit more work and custom blocklists could be a supported feature.

For now, I’ve chosen 3.

Hopefully, the IPFire team will eventually realise how user-unfriendly the current approach is.

[kiowacon1 (Kiowa Consultants)]

We agree fully, one size does not fit all, and also use special curated lists for each of our client’s particular “differing” business environments. Our clients utilize ipsets as well A similar concern was raised early on in testing core 203.

Our workflow differs slightly however as we bring down the current dbl lists from ipfire, compare them against the curated list and only export the differences to the new block list. This way we maintain the existing block lists ( ie advertising ) and also impart our additions only. Should our scripting break hopefully this leaves the existing base lists in play. I read somewhere though that if one list breaks due to an invalid domain or character then the whole dns firewall breaks. Might confirm if this is true.
* Caution to anyone attempting this - filter and parse first.

We have found that there are domains that ipfire considers invalid by “specs” however they are valid and do resolve elsewhere. These may break the dns filewall or at the least not allow you to use these in the custom input section. We attempted to call “kzonecheck” to validate the zone files but it could not be located on our dev and control system.

In recent testing this method works with stable core 203, however we are experiencing some inconsistencies with return results where most domains in the testing groups return NXDOMAIN but a few do not. Ugh! Have not modified the json file yet, but will try that next.

Hopefully the developers will expand this to include full custom list inserts. It would be nice if the IPS rule sets also had this option without work arounds. Perhaps feature requests are warranted.

[praful]

Thanks @kiowacon1 good to know I’m not a lone wolf! It’s a shame we’re having to jump through hoops for a basic feature.

[praful]

Since my last post, I’ve updated the script to download the OISD blocklist. It’s been running for a week without issues.

It now:

• updates dnsbl and dnsbl.json if they have changed (in case a process has changed it)
• downloads the blocklist
• replaces the existing blocklist if it has changed
• reloads Knot Resolver if required
• logs message to system log (/var/log/messages); to see the messages:

grep update-dns-blocklist /var/log/messages

The script runs daily by being placed in /etc/fcron.daily/.

I’ve created the script for my requirements. Don’t use it as is! I post it here because it may help someone make use of it for their needs. The script will probably break if the format of dnsblor dnsbl.jsonis changed in the future.

#!/bin/bash

# Allow custom blocklists to be used in IPFire. The script works with OISD, a low false positive
# blocklist. The script:
# - Updates the file that drives IPFire's blocklists (dnsbl.json); this will happen only if
#   dnsbl.json is updated by IPFire, possibly during an upgrade.
# - Enable the blocklist if it's not enabled; this will happen only if the options are changed in the GUI admin
#   DNS Firewall page or if the file is replaced during an upgrade.
# - Downloads the latest blocklist. 
# - Replaces the existing blocklist if it's older
# - Reloads Knot Resolver.
#
# I put this file in /etc/fcron.daily/ so that the script runs daily.

# enable errtrace for ERR traps (-E)
# exit on error (-e)
# treat unset variables as errors (-u)
# catch errror in pipelines such as cmd1 | cmd2  (-o pipeline)
set -Eeuo pipefail

# output info message to terminal (interactive mode) or system log (/var/log/messages)
log() {
    if [[ -t 1 ]]; then
        echo "$*"
    else
        logger -t update-dns-blocklist "$*"
    fi
}

# output error message to terminal (interactive mode) or system log (/var/log/messages)
error() {
    if [[ -t 2 ]]; then
        echo "ERROR: $*" >&2
    else
        logger -p daemon.err -t update-dns-blocklist "$*"
    fi
}

# delete temp files on exit
trap 'rm -f "$JSON_TMP" "$ZONE_TMP"' EXIT
# log exit error
trap 'rc=$?; error "Failed at line $LINENO (exit code $rc)"; exit $rc' ERR


# Configuration
ZONE_DIR="/var/lib/knot-resolver/zones"
ZONE_FILE="${ZONE_DIR}/oisd.rpz.zone"
ZONE_NAME="oisd.rpz"

DNSBL="/var/ipfire/dns/dnsbl"
DNSBL_JSON="/var/ipfire/dns/dnsbl.json"

# for debugging
# DNSBL2="/root/bin/dnsbl"
# DNSBL_JSON2="/root/bin/dnsbl.json"
# DNSBL="${DNSBL2}"
# DNSBL_JSON="${DNSBL_JSON2}"

DOWNLOAD_URL="https://big.oisd.nl/rpz"

JSON_TMP="$(mktemp)"
ZONE_TMP="$(mktemp)"


RELOAD=false

# Ensure OISD exists in dnsbl.json
if ! jq -e '.[] | select(.zone == "'"${ZONE_NAME}"'")' "${DNSBL_JSON}" >/dev/null; then
    jq '. += [{
        "name": "OISD",
        "zone": "'"${ZONE_NAME}"'",
        "primary": "localhost",
        "description": "OISD blocklist",
        "license": "GNU General Public License v3.0"
    }]' "${DNSBL_JSON}" > "${JSON_TMP}"

    log "Updated ${DNSBL_JSON}"
    mv "${JSON_TMP}" "${DNSBL_JSON}"
    RELOAD=true
fi

# Ensure OISD is enabled
if ! grep -qxF "${ZONE_NAME},on,,,," "${DNSBL}"; then
    echo "${ZONE_NAME},on,,,," >> "${DNSBL}"
    log "Updated ${DNSBL}"
    RELOAD=true
fi

# Download latest RPZ
wget -q -O "${ZONE_TMP}" "${DOWNLOAD_URL}"

# Install only if changed
if ! cmp -s "${ZONE_TMP}" "${ZONE_FILE}"; then
    log "New version of zone file. Replacing ${ZONE_FILE}"
    mv "${ZONE_TMP}" "${ZONE_FILE}"
    chown knot-resolver:knot-resolver "${ZONE_FILE}"
    RELOAD=true
fi

# Reload only if necessary
if $RELOAD; then
    /usr/local/bin/dnsctrl reload

    # The above works but is below the official way?
    # /etc/init.d/knot-resolver restart
    
    log "Knot Resolver reloaded."
else
    log "No changes: reload not required."
fi

[siosios]

Script works great, just had to chmod dnsbl & dnsbl.json so the wui could read it.

[praful]

That’s great! And thanks for letting me know about the chmod.

[fxneng (fang xn)]

Hello everyone,

Based on the community discussion, I used ChatGPT to create an unofficial patch for the IPFire DNS firewall that adds support for user-managed RPZ blocklists.

The patch was developed and tested on:

• IPFire 2.29 (x86_64) - Core-Update 203**

Main features

• Adds an Add button to the DNS Firewall list page.
• Supports adding custom RPZ lists from HTTPS URLs.
• Custom list fields include:
  • Name
  • Zone
  • Primary
  • RPZ URL
  • Description
  • License
  • Access control
• Custom lists can be enabled or disabled like official lists.
• Adds edit and delete icons for custom lists.
• Official list metadata remains read-only.
• Custom configuration is stored separately in:

/var/ipfire/dns/custom_dnsbl.json

This prevents official dnsbl.json updates from overwriting custom-list configuration.

Unified update mechanism

Custom and Official lists are updated through the existing IPFire updater:

/usr/local/bin/update-rpzs

The existing IPFire fcron entry runs this command once per hour at a randomized time:

%hourly,random * /usr/local/bin/update-rpzs

During each run:

1. Enabled official RPZ zones are synchronized using zone-sync.
2. Enabled custom HTTPS RPZ files are downloaded.
3. Downloaded files are validated.
4. Valid files are installed atomically.
5. DNS is reloaded once after the complete update.

Official and custom updates use the same lock file, preventing concurrent RPZ modifications.

Disabled custom lists are not downloaded.

RPZ validation and safety

Before replacing an existing RPZ file, the updater verifies that:

• The URL uses HTTPS.
• The downloaded content is not an HTML error page.
• The file contains an SOA record.
• The download completed successfully.

If downloading or validation fails, the existing working RPZ file is preserved.

Installation

Download and extract the package, then run it as root:

unzip ipfire-dns-firewall-custom-lists-1.1.0.zip
cd ipfire-dns-firewall-custom-lists-1.1.0
./install.sh

The installer:

• Checks the IPFire Core Update version.
• Validates all included scripts.
• Creates timestamped backups.
• Preserves the original official files for uninstallation.
• Installs English and Simplified Chinese translations.
• Preserves existing custom-list configuration.

Open the DNS Firewall page after installation:

https://<ipfire-address>:444/cgi-bin/dnsbl.cgi

Uninstallation

Run:

./uninstall.sh

The uninstaller:

• Restores the original official dnsbl.cgi.
• Restores the original official update-rpzs.
• Removes the added language entries.
• Removes custom RPZ entries from the active DNS configuration.
• Backs up custom configuration and RPZ files before removing them.
• Reloads DNS after restoration.

The uninstall backup is stored under:

/root/dns-firewall-custom-lists-uninstall-<timestamp>/

Example: OISD

The following custom list was used for testing:

Name: OISD
URL: https://big.oisd.nl/rpz
Description: OISD Big blocklist
License: https://github.com/sjhgvr/oisd/blob/main/LICENSE

Please note that OISD Big is approximately 14 MB and will be downloaded during every hourly RPZ update while enabled.

Testing performed

The patch has been tested for:

• Installation and uninstallation
• Reinstallation after uninstall
• Adding, editing, and deleting custom lists
• Enabling and disabling custom lists
• Real DNS blocking and unblocking
• Official and custom update serialization
• Successful HTTPS RPZ updates
• Failed-download preservation
• HTML and invalid-data rejection
• Atomic RPZ file replacement
• English and Simplified Chinese WebGUI
• Normal-domain DNS regression
• Package and SHA-256 verification

In the OISD test, the RPZ contained approximately 502,000 blocking records. A listed test domain returned NXDOMAIN when enabled and resolved normally when disabled.

Download

ipfire-dns-firewall-custom-lists-1.1.0.zip

Important notes

• This is an unofficial community patch，is intended solely for testing.
• It has currently been tested only on IPFire 2.29 Core Update 203.
• IPFire Core Updates may replace the modified files.
• Users are responsible for checking the content, reliability, and license of third-party RPZ feeds.
• Large HTTPS feeds are downloaded during every hourly update while enabled, so bandwidth and storage requirements should be considered.

Feedback, testing results, and suggestions are welcome.

[bbitsch (Bernhard Bitsch)]

Have you verfied the AI generated sources?
Which tests did you apply?
Why do you use hourly full downloads and not the AXFR/IXFR mechanism?

[fxneng (fang xn)]

Thanks for pointing this out. You are right; performing a full HTTPS RPZ download every hour results in unnecessary bandwidth and resource consumption.
I reviewed IPFire’s current implementation: the official lists continue to use AXFR/IXFR via zone-sync. However, the OISD source used in my tests provides only HTTPS RPZ files and lacks a Primary server capable of zone transfers, making direct AXFR/IXFR usage impossible.
In version 1.1.1, I switched to using HTTP conditional requests. The program stores the ETag and Last-Modified headers returned by the server, performing only a change check during subsequent hourly runs. If the list remains unchanged, the server returns a “304 Not Modified” status without transferring the full file; the file is re-downloaded, verified, and atomically replaced only when changes occur.
In actual testing, the initial download of “OISD Small” took 189 seconds, while the second check took just 3 seconds and returned HTTP 304; subsequent checks for “OISD Big” also returned 304, with no changes detected in the rule file’s hash or modification time.
The source code has undergone syntax, installation/uninstallation, DNS interception, failure-retention, mutual exclusion, HTTP 304, and regression testing on a live IPFire 2.29 Core Update 203 host. Thank you for your suggestions.

I am not a professional; I simply use AI to solve problems. This patch is intended for testing purposes only, and I have no plans to submit it as a new feature.

ipfire-dns-firewall-custom-lists-1.1.1.zip

[siosios]

The back button does not work on the “add” page because it wants that field filled out on version 1.1.1 of your patch. i am testing on Core Update 204.

[fxneng (fang xn)]

The issue has been fixed; feel free to test it.

ipfire-dns-firewall-custom-lists-1.1.2.zip