Add additional sources to IPFire blocklist feature

[peppetech]

@jon

Good find! Except for 66 lines it is almost a perfect sub-overlap.

These are the THREATFOX lines missing from the Threat-intelligence-feeds:

$  diff --side-by-side --minimal --suppress-common-line <(sort ips.txt) <(sort tif.txt) | grep "<" 

0.0.0.1							      <
0.0.1.0							      <
0.0.1.1							      <
0.0.1.4							      <
0.0.4.0							      <
0.0.5.0		

I wonder if some of those not overlapping are covered by BOGON FULL that @timf included already.

The others could be just out sync because there is a long delay in processing ?

I noticed that the Github Threatfox IP blocklist is released every 1-3 hours.
Threat-intelligence-feeds are only released 1 times a day at 4 AM

The official Threatfox at Abuse.ch gets generated every 5 minutes.

Open source is the king but the one fact that is bothering me about the Threatfox Github IP blocklist, is there a reliable way to verify the source of this IPs
I read through the code but I can’t figure out how the IP’s are extracted.

[jon]

@peppetech

I read through the code but I can’t figure out how the IP’s are extracted.

You may just have to ask Gerd (hagezi). He has a proton mail email address.

[peppetech]

@adamgibbo

Do you happen to know if they use ThreatFox as a source, or if it’s just a coincidence to have such an overlap?

I couldn’t verify the source

@jon

You may just have to ask Gerd (hagezi). He has a proton mail email address.

I was wondering about the other gihub account not Gerd’s, because Gerd is known around the Blocklist community.

[jon]

@peppetech

I was wondering about the other gihub account not Gerd’s

The elliotwutingfeng list at elliotwutingfeng (Wu Tingfeng) · GitHub ?

Is there nothing good in the ThreatFox-IOC-IPs/update.py at main · elliotwutingfeng/ThreatFox-IOC-IPs · GitHub code?

(I am afraid I dont speak Python very well)

[peppetech]

same here

[peppetech]

I have seen this list referenced in few places. what are your thoughts about

FireHOL IP Lists | IP Blacklists | IP Reputation Feeds

does it look useful for an blocklist? It is only listing subnets so it is relatively short
is there any advantage to use subnets only?

# A firewall blacklist composed from IP lists, providing 
# maximum protection with minimum false positives. Suitable 
# for basic protection on all internet facing servers, 
# routers and firewalls. (includes: bambenek_c2 dshield feodo 
# fullbogons spamhaus_drop spamhaus_edrop sslbl ransomware_rw)
#
# Maintainer      : FireHOL
# Maintainer URL  : http://iplists.firehol.org/
# List source URL : 
# Source File Date: Sat Feb 10 08:55:02 UTC 2024
# This File Date  : Sat Feb 10 09:21:26 UTC 2024
# Update Frequency: 1 min 
# Aggregation     : none
# Entries         : 1966 subnets, 612332544 unique IPs
#
# Full list analysis, including geolocation map, history,
# retention policy, overlaps with other lists, etc.
# available at:
#
#  http://iplists.firehol.org/?ipset=firehol_level1
#
# Generated by FireHOL's update-ipsets.sh
# Processed with FireHOL's iprange
#
0.0.0.0/8
1.10.16.0/20
1.19.0.0/16
1.32.128.0/18
2.56.58.0/24
2.56.192.0/22
2.56.247.0/24
2.57.122.0/24
2.57.232.0/22
2.58.95.0/24
5.42.64.0/22
5.42.92.0/24
5.42.199.0/24
5.105.62.0/24

[peppetech]

I keep adding sources whenever I found something interesting.

I have been using this list for domain filter and now I see they have even an IP list.
I checked 3 IP addresses and none of them overlapped with any existing IP Blocklists at least the moment I checked…

I am including another source for IP Blocklist with all information @timf requested

Phishing IPs Blocklist by MALWARE-FILTER

https://malware-filter.gitlab.io/malware-filter/phishing-filter-dnscrypt-blocked-ips.txt

A blocklist of phishing websites, curated from PhishTank, OpenPhish, phishunt.io. Blocklist is updated twice a day.

License
src/: Creative Commons Zero v1.0 Universal and MIT License
filters: CC BY-SA 4.0
PhishTank: Available free of charge by Cisco for commercial and non-commercial use.
PhishTank is either trademark or registered trademark of Cisco Systems, Inc.
OpenPhish: Available free of charge by OpenPhish
Tranco List: MIT License
Umbrella Popularity List: Available free of charge by Cisco Umbrella
csvquote: MIT License
phishunt.io: All rights reserved by Daniel López
Cloudflare Radar: Available to free Cloudflare account
This repository is not endorsed by PhishTank/OpenDNS and OpenPhish.

[peppetech]

@peppetech

Phishing IPs Blocklist by MALWARE-FILTER

https://malware-filter.gitlab.io/malware-filter/phishing-filter-dnscrypt-blocked-ips.txt

I am getting an error on the WUI page for IPFBL

Could not download blocklist - A download error occured.

link works I even switched to a different mirror, so no DNS issues

EDIT: I tried a few other mirrors and IPBlocklist was accepted by IPFire

[jon]

@peppetech

I keep adding sources whenever I found something interesting.
. . .
Phishing IPs Blocklist by MALWARE-FILTER

I looked through the Malware-Filter web site. Lots of interesting info! I did not spend enough time to know if the Malware-Filter is good or bad. I did find it had an RPZ list so sometime in the future I will take a closer look

Thanks!

[peppetech]

@jon

I looked through the Malware-Filter web site. Lots of interesting info! I did not spend enough time to know if the Malware-Filter is good or bad. I did find it had an RPZ list so sometime in the future I will take a closer look

Thanks!

Yes, it has RPZ, Suricata, a lot of formats.
I figured using it for IP blocklist would be the most efficient. but let me know when you get to it

[jon]

@peppetech

I figured using it for IP blocklist would be the most efficient

That is one “test” I am struggling with. I am hoping to determine if IP blocklist is more efficient or less efficient than RPZ…

[peppetech]

Each of these have a downside,

With RPZ , it would only filter outgoing requests and only if it receives a DNS query.
There is also the rate limiting. with Unbound which I couldn’t figure out how reset the rate limiting after a domain gets blocked, rejected or gets NXdomain.

Here is some interesting facts about IP Blocklist"

Order of defence line features/options

In general the rationale for putting the IP Blocklist early in the chain is that it can search through a large list of blocked addresses without using very much processing power. It doesn’t have to analyse the packet; it just extracts the IP address and looks it up a a series of hash tables. By putting this early IPFire can avoid processing these packets in later stages, for example in the IPS which can do the same job, but has to analyse the packet in more detail.

[gw-ipfire (ipfire-test ipfire-test)]

hi

That is one “test” I am struggling with. I am hoping to determine if IP blocklist is more efficient or less efficient than RPZ

rzp is more flexible in the filter rules it seems to me to performce it does not have to be used as a more list if not this use a lot of memory and a resource but if we use rpz in white list very little use of the memory
ty

[g70p]

Snort also have some IPadressblocklist. Here:

https://www.snort.org/downloads/ip-block-list/

[crazyfire (D R)]

Peppe Tech, I tested the Common DoH server list and it blocked everything from getting to the internet. I couldn’t resolve to DNS when using it. The 3CoreSec is working great so far, but the common DoH is a no go, it blocks regular dns traffic. I had to disabled the DoH one for my systems to get online.

[peppetech]

@crazyfire

I tested the Common DoH server list and it blocked everything from getting to the internet. I couldn’t resolve to DNS

Yes, that is unfortunate and that happened to me as well, I posted about it a while ago.

[peppetech]

@g70p

It was blocking Let’s encrypt SSL requests

How would that happen? Was LE on that list?

[peppetech]

@crazyfire

the common DoH is a no go, it blocks regular dns traffic. I had to disabled the DoH one for my systems to get online.

So I am not the only one, when this happens, Unbound DNS supposed to switch to recursive mode, but it just did nothing…

Looks like a DoH IP blocklist doesn’t work for DoH, because they share IP with DoT

but it happens with websites hosted on some of the CDN’s as well.

[g70p]

It was blocking Let’s encrypt SSL requests

[g70p]

@peppetech sorry for the late reply. I was configuring SSL certificates for nginx and accordigly to the ipfire help manual for the add-on. The help manual suggest to ask Let’s encrypt certificate on Example 3:

Example 3: Managing ssl-certificates for all your sites by acme.sh and Let’s Encrypt.
Your nginx is working as a reverse proxy for a couple of websites with different domains behind. User who surf to your sites by ssl see the nginx delivered ssl-certificate . In most cases this is self-signed and would be marked by browsers as unsecured. You need for every of your hosted domains a secure ssl-certificate and nginx should deliver it. The solution is a little script acme.sh and Let’s Encrypt as CertAuthority!
www.ipfire.org - Nginx

at the time I couldn’t

curl https://get.acme.sh | sh

because doh was blocking. it, I didn’t try again once I ended up doing a selfsigned cert for personal use.