Add additional sources to IPFire blocklist feature

The new Firewall blocklist in Core Update 170 looks promising. :ok_hand:

Is there a way to include more Blocklists? From a reputable source like Github, Gitlab etc… and are well maintained?

2 Likes

Just went from zero to 16… Is it not enough?!?

:upside_down_face:

3 Likes

HAHAHA, no I am not complaining, just I like to use blocklists to block services like DoH …

1 Like

Is there something specific that you had in mind? (something reputable and well maintained?) I am just curious…

1 Like

The file containing the list of blocklists is /var/ipfire/ipblocklist/sources. The explanation of the fields in the file is in the file header, so it’s easy enough to add new entries if you want, however the file will be overwritten by any future updates.

If you find any lists which would be useful to other people you could submit a patch to update this file, but before doing this it’s important to check that you’ve got the maximum frequency of checking for updates right, as many lists will block anyone trying to update too often. If you’re going to submit a patch it’s also important to check that the licence for the lists allows the list to be included in the ipfire distribution.

Possibly in a future update we can allow the possibility of a sources.local file, but I think the feature needs to be left as it is for a while before adding new functionality.

.

6 Likes

Welcome back @Timf :slight_smile:
Many thanks for your detailed explanation, I will follow it and post some lists below for discussion before submitting it.

Interesting that Abuse.ch went or is going commercial very soon. so it makes sense to have option.
@jon There used to be plenty of open and well maintained lists, but they either went commercial or abandoned. But here are some I recently found and used in addition to the ones already in the addon:

1- This list contains Common DoH servers and I think it is useful in cases when clients are circumventing IPFire’s DNS server.
It doesn’t mention a Licence. In terms of update checks I think Github is pretty forgiving for update checks if done in a “random” pattern or intervals.

https://raw.githubusercontent.com/jpgpi250/piholemanual/master/DOHipv4.txt

2- 3CoreSec Blacklist - they share threats from their honeypot projects
https://blacklist.3coresec.net/lists/all.txt

3coresec lists are under GNU Affero General Public License
https://www.gnu.org/licenses/agpl-3.0.en.html
3coresec doesn’t mention update check terms.

The "ALL list "above is combined from these specific lists

3- Hosts involved in SSH brute-force
https://blacklist.3coresec.net/lists/ssh.txt

4-Hosts involved in mass scanning and/or exploitation attempts
https://blacklist.3coresec.net/lists/misc.txt

5-Hosts involved in HTTP brute-force and/or enumeration
https://blacklist.3coresec.net/lists/http.txt

6 Likes

Took the 4 lists once and hits are coming in. I also hope that later you can add your own lists via web interface.

Paul

Honey = All.
It makes no sense, to include the other ones.

1 Like

Looks like @Pablo78 found my suggestions useful:

Here is another one:
a Phishing Database of Active IP’s: by MitcheKrogza

https://raw.githubusercontent.com/mitchellkrogza/Phishing.Database/master/phishing-IPs-ACTIVE.txt

Here is the readme and other lists

License: MIT License
Updated: hourly

Would it be useful to post a FULL DB in tar.gz
I think it contains Domains or Links.

would this still work with tar.gz

‘parser’ => ‘ip-or-net-list’,

let me know if you think it is useful.and I can submit a patch

2 Likes

Hello, old topic, but usefull
=> yes please add to the list (and perhaps allow us to have a section maintained by us to put a lot of more ?)
=> also perhaps you can put more (and at the end we choose through a list as you have done)

anyway thanks it is a great idea

Perhaps, Mitchell stopped supporting the IP-Active lists. about 6 months ago…?

He is still updating the Domains and Links lists so I think I will switch to a DNS based blocked for Phishing protection.

I also wonder if Gerd’s IP list is useful for IPFire IP blocklist:

Threat-intelligence-feeds:
https://raw.githubusercontent.com/hagezi/dns-blocklists/main/ips/tif.txt

Gerd Hagezi’s lists seem to be well maintained in a lot of formats and became popular in last few months

including an an Unbound list which would work for IP Fire, I believe.

Any thoughts?

1 Like

:thinking: Might be worth a look here

Thinking about using additional lists

1 Like

Is that a regional list?

I noticed a lot of .pl domain names

The idea to add own download URLs is a nice thing.
Same for URL Filter…

I manage my lists in “many” ~ 107 categories on DMZ at NAS;
for domains and IPs of different maintainers, either of who are no further active or reachable as ex. badips, shalla, fake domains, etc.
Other custom and reported … received spams and the phishing IPs URL domains.

Thanks Tim for that great job! Many positive hits were blocked and reported over the years.

By the way
Who think about adding blocklists, should either think about own managed.
Also it makes big sense to have (own) managed whitelist … for the case of false positive.

BR
Trash

Old screenshot of modification to Tims code:

1 Like

I’m still clicking in boxes of the customize rulesets IPS :joy: :triumph: :see_no_evil: :hear_no_evil: but bookmarking this topic as peer-block lists sounds great to add in.
Regards
G70P
https://www.iblocklist.com/lists

What is the screenshot from?

How is the URL filter working for URL’s that are using Https?

All phishing links now use https so I am not sure how to block these

I am including another source for IP Blocklist with all information @timf requested

  1. **SSLBL Botnet C2 IP Blacklist (IPs only) **

https://sslbl.abuse.ch/blacklist/sslipblacklist.txt

The Botnet C2 IP Blacklist gets generated every 5 minutes. Please do not fetch it more often than every 5 minutes.

Abuse.ch appointed Spamhaus as their Licensee - not sure what it means in practical terms but here are TOS:

   - By using the website of SSLBL, or any of the services / datasets referenced above, you agree that:
   - All datasets offered by SSLBL can be used for both, commercial and non-commercial purpose without any limitations (CC0)
   - Any data offered by SSLBL is served as it is on best effort
   - SSLBL can not be held liable for any false positive or damage caused by the use of the website or the datasets offered above

Let me know if this is useful, x there are 37 IP’s on the list as of this writing.

The Suricata rules are much larger

I am including another source for IP Blocklist with all information @timf requested

This is an unofficial Threatfox IP blockist list from Github:
Machine-readable .txt IP blocklist from [ThreatFox](https://threatfox.abuse.ch) by [Abuse.ch](https://abuse.ch), updated every hour.

  1. THREATFOX

https://raw.githubusercontent.com/elliotwutingfeng/ThreatFox-IOC-IPs/main/ips.txt

The Botnet C2 IP Blacklist gets generated every 1 hours.

Since this is an unofficial list of IP’s generated, I am including all the licensing information I could find
The IPs in this blocklist are compiled by **Abuse.ch** under the [Creative Commons Zero v1.0 Universal](https://threatfox.abuse.ch/faq) license.

**Disclaimer:** *This project is not sponsored, endorsed, or otherwise affiliated with Abuse.ch.*

License information from Github page:

Abuse.ch appointed Spamhaus as their Licensee - not sure what it means in practical terms but here are official TOS:

   By using the website of ThreatFox or any of it's services / datasets, you agree that:

    All datasets offered by ThreatFox can be used for both, commercial and non-commercial purpose for free without any limitations (CC0)
    Any data offered by ThreatFox is served as it is on best effort with no warranty
    ThreatFox can not be held liable for any false positives or damage caused by the use of the website or the datasets provided
    Any submission to ThreatFox will be treated and shared under TLP:WHITE and under Creative Commons No Rights Reserved (CC0)

Let me know if this is useful, x there are 88.000 IP’s on the 1,2 MB list as of this writing.

The Suricata rules files are much larger 30 MB

I just made an Interesting observation.

While looking for overlapping blocklists, I noticed that these 2 lists overlap by 87.964 lines of IP addresses, as of this writing

  1. THREATFOX

https://raw.githubusercontent.com/elliotwutingfeng/ThreatFox-IOC-IPs/main/ips.txt

contains 88.048 lines or IP addresses

  1. Threat-intelligence-feeds:

https://raw.githubusercontent.com/hagezi/dns-blocklists/main/ips/tif.txt

contains 176.920 lines or IP addresses

1 Like

Good find! Except for 66 lines it is almost a perfect sub-overlap.

These are the THREATFOX lines missing from the Threat-intelligence-feeds:

$  diff --side-by-side --minimal --suppress-common-line <(sort ips.txt) <(sort tif.txt) | grep "<" 

0.0.0.1							      <
0.0.1.0							      <
0.0.1.1							      <
0.0.1.4							      <
0.0.4.0							      <
0.0.5.0							      <
0.0.5.4							      <
0.1.0.0							      <
0.1.1.0							      <
0.1.1.4							      <
0.1.4.4							      <
0.1.5.0							      <
0.1.5.4							      <
0.151.228.146					      <
0.2.0.0							      <
0.2.0.1							      <
0.208.210.72					      <
0.32.0.1						      <
0.64.0.0						      <
0.64.0.1						      <
0.64.0.65						      <
0.64.16.1						      <
0.64.16.65						      <
0.64.2.1						      <
0.64.2.9						      <
0.66.0.0						      <
0.66.0.1						      <
0.68.0.1						      <
0.68.2.9						      <
0.96.0.1						      <
0.96.0.65						      <
0.96.16.1						      <
0.96.16.65						      <
103.86.130.79						      <
123.249.86.77						      <
124.223.56.72						      <
127.200.198.38						      <
139.59.238.68						      <
149.248.21.89						      <
154.12.85.223						      <
154.53.160.71						      <
154.8.138.27						      <
172.94.32.33						      <
172.96.14.67						      <
185.49.70.105						      <
187.135.122.173						      <
187.135.130.228						      <
188.119.112.49						      <
189.152.202.202						      <
193.233.254.10						      <
193.233.254.106						      <
20.170.42.196						      <
211.97.157.183						      <
43.128.203.170						      <
43.138.110.8						      <
44.219.14.139						      <
46.4.80.247						      <
47.109.136.12						      <
47.93.98.77						      <
47.99.93.124						      <
5.188.86.214						      <
51.195.83.136						      <
78.46.135.92						      <
8.212.183.173						      <
81.28.6.17						      <
89.148.24.117						      <
1 Like