Blocking Malware payloads and Botnet C2 using Unbound and Threatfox database

Security
1

I want to credit @jon who came up with all the steps in this thread

I recently discovered Threatfox Suricata rules, a promising source of Indicators of Compromise. The only downsides is around 30 MB of rules which ends up being too memory intensive on a lot of security appliances.

Since Threatfox Suricata rules take up to 2GB of physical RAM when applied in IPFire IPS., I want to test the RPZ method that @Jon perfected for DoH blocking.

Quote from Threatfox: By using an DNS Reponse Policy Zone (RPZ), also known as DNS firewall, you can block the resolution of certain domain names ovserved in the past 6 month on your DNS resolver.

Header of the RPZ file

$TTL 30 @ SOA rpz.threatfox.abuse.ch. hostmaster.threatfox.abuse.ch. 2402061856 300 1800 604800 30 NS localhost. ; ; ThreatFox Response Policy Zones (RPZ) ; Last updated: 2024-02-06 18:56:40 (UTC) ; ; Terms Of Use: https://threatfox.abuse.ch/faq/#tos/ ; For questions please contact threatfox [at] abuse.ch

This database is being updated every 5 minutes (300 seconds)

Step 1)

A new directory is needed. Enter the command:

mkdir -v -p -m757 /etc/unbound/zonefiles

Now edit a new configuration file:

nano /etc/unbound/local.d/threatfox.conf

Step 2)

And add these lines:

server:
	module-config: "respip validator iterator"

rpz:
	name: threatfox
 	#zonefile: zonefiles/threatfox.zone
 	zonefile: /etc/unbound/zonefiles/block.threatfox.rpz.zone
 	url: https://threatfox.abuse.ch/downloads/threatfox.rpz
	rpz-action-override: nxdomain
	rpz-log: yes 
	rpz-log-name: threatfox

To check for issues in the config file, run the command on IPFire:

unbound-checkconf

Step 3)

Restart unbound to load config file:

/etc/init.d/unbound restart

Step 4)

Wait for 5-6 minutes for the zonefiles to be downloaded to

/etc/unbound/zonefiles/block.threatfox.rpz.zone
2 Likes
2

Iā€™m getting the same error in Unbound as with the DoH RPZ

error: could not open /etc/unbound/zonefiles/block.threatfox.rpz.zone.tmp23541: Permission denied
error: could not open /etc/unbound/zonefiles/block.threatfox.rpz.zone.tmp23541: Permission denied
error: could not open /etc/unbound/zonefiles/block.threatfox.rpz.zone.tmp23541: Permission denied
error: could not open /etc/unbound/zonefiles/block.threatfox.rpz.zone.tmp23541: Permission denied
error: could not open /etc/unbound/zonefiles/block.threatfox.rpz.zone.tmp23541: Permission denied
error: could not open /etc/unbound/zonefiles/block.threatfox.rpz.zone.tmp23541: Permission denied
error: could not open /etc/unbound/zonefiles/block.doh.rpz.zone.tmp27703: Permission denied
3

Looking at the memory usage per hour, there is a barely noticeable dip in memory usage

image
image1448Ɨ343 105 KB

4

The error: could not open is bad. Were you able to get the permission problem fixed?

If not, please enter this and post results:

 ls -al /etc/unbound/zonefiles

EDIT: I know Iā€™ve seen type of error.
Where did you see this error? In the message log or somewhere else? Did the message get shortened in anyway? (Iā€™m still looking!)

5

These results are no good.
Until it has the file permissions fixed.
It has nothing to do. With no files / info to use.

6

Iā€™ve set permissions to 757 ( as chown in mkdir command in post #1 ), definitively.
It works!

Safer dir generation commands are

mkdir -v -p /etc/unbound/zonefiles
chmod 757 /etc/unbound/zonefiles

These set the permissions even if the dir exists.

2 Likes
7

BTW.
Thx for pointing us to the RPZ functionality. Iā€™ve tried some list from GitHub - hagezi/dns-blocklists: DNS-Blocklists: For a better internet - keep the internet clean!. Interesting how many ā€˜popupā€™ requests are done eg.

3 Likes
8

Iā€™ve been reading/watching this one also. He has lots of interesting RPZ files available. I havenā€™t tried any of it (yet).

EDIT: Here are my results for ls -al:

[root@ipfire ~] # ls -al /etc/unbound/zonefiles
total 10992
drwxr-xrwx 2 root   root       4096 Feb  7 09:43 .
drwxr-xr-x 4 root   root       4096 Feb  7 07:32 ..
-rw-r--r-- 1 root   root        666 Jan 17 15:36 allow.doh.rpz.zone
-rw-r--r-- 1 nobody nobody    85626 Feb  6 15:24 block.doh.rpz.zone
-rw-r--r-- 1 root   root   11123883 Feb  5 10:47 block.sblack.rpz.zone
-rw-r--r-- 1 root   root        433 Feb  5 02:47 block.tld.rpz.zone
-rw-r--r-- 1 nobody nobody    27074 Feb  7 09:43 block.urlhaus.rpz.zone
[root@ipfire ~] #
9

I suppose the two files with owner nobody/nobody are the problem.
Deletion and restart of unbound should solve the problem.

10

No, those two work as expected. Donā€™t change them.

I think unbound runs as ā€œnobodyā€ but I donā€™t know how to verify.

EDIT: Found this:

[root@ipfire ~] # ps aux | grep unbound
nobody    2095  0.3  3.2 132956 130508 ?       Ss   Jan28  50:33 /usr/sbin/unbound
. . .
11

You are right.!
This means, the root/root files are the problem.

12

For the directory? Or the three root:root files? The three files work but are experimental and will eventually change to follow unbound

TMI - Too Much Info!

I cannot figure out why/how unbound runs as nobody! I can understand running as the user unbound but why (really how) user nobody??

Only a few things run as nobodyā€¦

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
[root@ipfire ~] # ps aux | grep nobody
nobody    2095  0.3  3.2 132956 130508 ?       Ss   Jan28  50:37 /usr/sbin/unbound
nobody    3502  0.0  0.1   9816  6928 ?        Ss   Jan28   0:17 /usr/sbin/openvpn --config /var/ipfire/ovpn/server.conf
nobody   23878  0.0  0.0  13560  3864 ?        S    Feb04   0:58 /usr/sbin/httpd -k start
nobody   23882  0.3  0.3 1275852 15964 ?       Sl   Feb04  18:16 /usr/sbin/httpd -k start
13

I could not get the ThreatFox to error like above. And I know Iā€™ve seen this error before, but I cannot locate what I did to fix.

error: could not open /etc/unbound/zonefiles/block.threatfox.rpz.zone.tmp23541: Permission denied

All works OK for me using the above ThreatFox steps. So I think something is different between the two setups. And I am guessing it is permission based but I need the @peppetech ls -al results to troubleshoot.

here is mine:

[root@ipfire ~] # lso /etc/unbound/zonefiles
total 14368
 757 drwxr-xrwx 2 root   root       4096 Feb  7 11:35 .
 755 drwxr-xr-x 4 root   root       4096 Feb  7 10:46 ..
 644 -rw-r--r-- 1 root   root        666 Jan 17 15:36 allow.doh.rpz.zone
 644 -rw-r--r-- 1 nobody nobody    85626 Feb  6 15:24 block.doh.rpz.zone
 644 -rw-r--r-- 1 root   root   11123883 Feb  5 10:47 block.sblack.rpz.zone
 644 -rw-r--r-- 1 nobody nobody  3453383 Feb  7 11:35 block.threatfox.rpz.zone
 644 -rw-r--r-- 1 root   root        433 Feb  5 02:47 block.tld.rpz.zone
 644 -rw-r--r-- 1 nobody nobody    27718 Feb  7 11:35 block.urlhaus.rpz.zone
[root@ipfire ~] #

-and-

[root@ipfire ~] # lso /etc/unbound/local.d
total 40
 755 drwxr-xr-x 2 root root 4096 Feb  7 10:59 .
 755 drwxr-xr-x 4 root root 4096 Feb  7 10:46 ..
 644 -rw-r--r-- 1 root root  229 Dec 16 14:45 10-log-options.conf
 644 -rw-r--r-- 1 root root  128 Dec 16 15:44 11-cache-ttl.conf
 644 -rw-r--r-- 1 root root   52 Oct  2 21:30 20-rpz.conf
 644 -rw-r--r-- 1 root root  407 Oct 17 22:31 doh.rpz.conf
 644 -rw-r--r-- 1 root root  198 Oct  9 21:34 sblack.rpz.conf
 644 -rw-r--r-- 1 root root  264 Feb  7 10:59 threatfox.conf
 644 -rw-r--r-- 1 root root  177 Oct  9 21:34 tld.rpz.conf
 644 -rw-r--r-- 1 root root  250 Oct  9 21:33 urlhaus.rpz.conf
[root@ipfire ~] #

EDIT: Here is the lso function if you are interested. Just cut & paste this into a Terminal/Console prompt:

lso() { ls -al "$@" --color | awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/)*2^(8-i));if(k)printf(" %0o ",k);print}'; }

Some other (not me!) very smart person created this!

14

Just to get the complete information.
What is the contents of the several .conf files?
Do they start all with

server:
	module-config: "respip validator iterator"

rpz:

Somewhere in the various docs about rpz with unbound I read that the ā€˜module-configā€™ shall occur only once.

1 Like
15

Was this to me or @peppetech? If me: Yes, this is correct for me.

Once I moved beyond one single config file to multiple config file I created a separate file and named it: /etc/unbound/local.d/rpz.conf

server:
	module-config: "respip validator iterator"

NOTE : I have not tested if the order of config file loading matters. This file may need to be named something like /etc/unbound/local.d/01-rpz.conf

1 Like
16

I didnā€™t realize there will be a lot of activity, :slight_smile:

The error is in System logs - DNS Unbound, and messages

I assumed the error happened because the zone file was an empty file and it didnā€™t get populated / downloaded even after the 300 seconds, not sure why

I get the same error with the DoH zone file. After I downloaded manually and replaced it, the error didnā€™t show up in the logs or messages in both DoH and Threatfox

 ls -al /etc/unbound/zonefiles
total 3384
drwxr-xrwx 2 root   root      4096 Feb  8 1:13 .
drwxr-xr-x 4 root   root      4096 Feb  8 1:07 ..
-rw-r--r-- 1 nobody nobody 3453516 Feb  8 1:13 block.threatfox.rpz.zone
17

Yes both conf files have the above line:

# cat "/etc/unbound/local.d/threatfox.conf"

server:
	module-config: "respip validator iterator"

rpz:
	name: threatfox
 	#zonefile: zonefiles/threatfox.zone
 	zonefile: /etc/unbound/zonefiles/block.threatfox.rpz.zone
 	url: https://threatfox.abuse.ch/downloads/threatfox.rpz
	rpz-action-override: nxdomain
	rpz-log: yes 
	rpz-log-name: threatfox
18

Make sure there is only one server: module-config: "respip validator iterator total.

You can do this if you wantā€¦

Concerning permissions

Did the permission error messages stop?

I know Iā€™ve seen this error!! Iā€™ll test and then temporarily add:

touch /etc/unbound/zonefiles/block.[name].rpz.zone
chown nobody:nobody /etc/unbound/zonefiles/block.[name].rpz.zone
chmod 644 /etc/unbound/zonefiles/block.[name].rpz.zone

ā€¦ to the instructions for now. But this seems too hacky to me!

1 Like
19

Yes the error stopped in both cases, once I manually downloaded the RPZ file .

ok I will make sure only 1 instance of

module-config: "respip validator iterator"
2 Likes
20

Correction,

I was looking at the wrong IPFire when I said Iā€™m not getting that RPZ error.

I started running a few IPfireā€™s to see how different HW reacts:
Now I am looking at the right IPFire with both RPZ files

Now I see it that ā€œpermission deniedā€ still occurs when running

grep rpz /var/log/messages

 [7543:0] error: could not open /etc/unbound/zonefiles/block.threatfox.rpz.zone.tmp7543: Permission denied
unbound: [7543:0] error: could not open /etc/unbound/zonefiles/block.threatfox.rpz.zone.tmp7543: Permission denied
unbound: [7543:0] error: could not open /etc/unbound/zonefiles/block.threatfox.rpz.zone.tmp7543: Permission denied
[7543:0] error: could not open /etc/unbound/zonefiles/block.threatfox.rpz.zone.tmp7543: Permission denied

ls -al /etc/unbound/zonefiles
total 6840
drwxr-xr-x 2 root root    4096 Feb  6 16:19 .
drwxr-xr-x 4 root root    4096 Feb  6 09:30 ..
-rw-r--r-- 1 root root       0 Feb  6 16:19 block.doh.rpz.zone
-rw-r--r-- 1 root root 3450385 Feb  6 17:36 block.threatfox.rpz.zone
-rw-r--r-- 1 root root   86535 Feb  8 01:25 doh.rpz
-rw-r--r-- 1 root root 3450385 Feb  6 15:55 threatfox.rpz

I did

# unbound-checkconf
unbound-checkconf: no errors in /etc/unbound/unbound.conf

chmod 757 /etc/unbound/zonefiles

/etc/init.d/unbound restart

I will be watching the log files for any changes