Today I stumbled across ScanThe.Net while checking some IP addresses on AbuseIPDB.
They are a new but very active contributor to AbuseIPDB, with over 300,000 contributions since November 2024, and their standing there is marked as “good”.
They offer three IP blocklists: a daily, weekly, and monthly, with the daily list being updated every 5 minutes. They also have a live feed, which looks interesting.
I have tried to add their blocklists to IPFire; however, they appear to be using Cloudflare’s bot detection, and the download is failing, which is unfortunate.
If you are using CU192 then that still has the default user agent string that libwww-perl uses, which is libwww-perl. As this perl package is routinely used by script kiddies and scammers for scraping for information, then Cloudflare, but also many other providers, do just block any access where the user agent string starts with libwww-perl.
Bear in mind that Cloudflare provides a box of these type of tools and they are usually turned on by default but the user of the site can tailor their Cloudflare rules.
A change has been made in CU193 for the User Agent string for libwww-perl when it is used for the IP Blocklist Downloading.
It is now IPFire/…
When tested out with Threatview.io that User Agent string was accepted by Cloudflare, so you might want to test out if the download works with CU193 Testing.
If it doesn’t then you would need to contact the blocklist provider and ask them to bypass the browser checks for a User Agent string starting with IPFire.
Same here — I think they’re fairly new. Possibly only active since around November 2024.
I actually tried it on CU194 next/ab7e955f, which I believe includes the recent user agent change. It looks like both their website and the blocklists are protected by Cloudflare’s CAPTCHA, which is what’s stopping the download.
I’ll try reaching out to them — first to see if they’re happy for us to use their lists, and also to ask whether they might be willing to whitelist our user agent string.
I just checked the download url and it is not a problem with the User Agent String, you can’t even download it via the url without checking a checkbox to prove you are human.
So that blocklist provider is not allowing automated downloads. They require you a human to manually download the lists and presumabbly then copy the downloaded material into whatever system you are using.
On that basis, their blocklists are 100% unusable by IPFire.
Yes, that was what I just found out. They would need to allow the IPFire User Agent string to bypass the CAPTCHA requirement, otherwise it will never work.
I am not so sure how long term they are likely to be. It looks like this was something that they managed to “get some free time so they created another project”.
Based on the above this looks like something that got fitted in when some time became available. What happens when some time is no longer available is unclear. Also not clear how these IP’s are being created to fill the list.
There are 20,921 IP’s in the current daily list. Those are not being reviewed in any way on a daily basis so I would suspect a high false positive response. They are probably being extracted from some other lists elsewhere on the web.
EDITED:
They are definitely not being reviewed or checked as I just searched and found that there are 13 private IP addresses in the daily list. 9 from 192.168.x.x and 2 from 10.x.x.x and 2 from 172.16.0.0/12
This part doesn’t worry me too much. Plenty of projects start as side projects, and their long-term success depends on community interest and adoption. That said, if users can’t easily access their blocklists due to the CAPTCHA requirement, it could limit uptake.
This is more concerning. Including private IPs in a public blocklist could block legitimate traffic, which would be a real issue for us. Quality over quantity is key for any blocklist we should include.
Thanks for raising these points, Adolf. I really appreciate your take on this.