URL Filter and self updating blacklists?

Hi,
is there a possibility to set some self updating lists for google, facebook and co. tracking? Not by uploading a txt file, but by seting the url with such file.

best regards
fstarter

Hi @fstarter.

You can create your own blacklist using the “Blacklist editor”. It creates a category for you that you can enable/disable and put the URLs you want.

Too easy. You can Google countless lists to block the URLs you want. There are a lot of them and if they are not in the copy/paste format, you modify them with “notepad++” and you create your own filter.

Hope this idea helps you.

You will tell us.

Greetings.

1 Like

Hi @roberto,
thanks. I know that I can create and / or upload a blacklist. The thing is, that it’s better to have blacklists, which are maintained / updated by somebody. The world of tracker and co. is very dynamic. So there is maby a possibility to insert a url to the blacklist of some organisations. That would be nice.

Best regards
fstarter

Hi,

This can be done:

HTH,
Matthias

The problem is these external blacklist databases are formatted in a variety of ways. They generally have to be reformated and “cleaned up” before they can work with the URL filter. We wrote shell scripts that take care of downloading, reformating, merging into the URL Filter, and then restarting Squid to activate the updated blacklists on a scheduled basis. It was not a trivial task and they to be maintained on a fairly regular basis.

The IPFire developers have more than enough to keep themselves busy. Something like this would be a low-priority item.

Best regards,
Fred

is there any list-url out there just with the right format?

I don’t believe we ever found a useful list in “native” IPFire format. When everyone moved from HTTP to HTTPS the usefulness of the URLFilter part of squid decreased significantly. We switched to doing the filtering in UnBound, which performs better.

How is your curl, awk, sed, grep and bash scripting? It’s not hard to put together an automated script to do this. It just isn’t going to be a pretty web page in IPFire unless you are willing to invest a lot of time.

hmm… my skills in programming is on the html level :smiley: so just nothing.
what do you mean with unbound filtering? Just filter via DNS-Server?

Unbound drives the DNS service in IPFire. Adding a block list in the /etc/unbound/local.d directory will do the job.

Does it need a special formating of the list?

I haven’t looked into Squidguard in detail yet. Only cursorily a few months ago I used it once in IPFire. My impression was that the result was significantly worse compared to Pi-Hole or AdGuard Home. That was only a short trial. However, I’m afraid that my impression would be confirmed if I used it more intensively.

The pre-built blacklists for Squidguard are probably not nearly as extensive as all the lists you can use together in Pi-Hole, etc.

Trackers are the biggest problem, for which I would additionally like to use something like Pi-Hole.
I know that Pi-Hole and AdGuard can be bypassed. Nevertheless, you can clearly notice the difference.

Apparently there are ways to convert the lists. Seems to involve effort though. There are some converters, but I don’t know how good they work (and if they still work).
Would be nice to have such functionality in IPFfire.

Hi all,
not sure if this interesting but we have had a little similar discussion in → https://community.ipfire.org/t/protect-users-who-get-spammy-phishing-links-in-emails in here → Protect users who get spammy phishing links in emails - #40 by ummeegge (more specific) with an older idea of how to get lists updated in SquidGuard with a little bashing around but more important are IMHO some good lists.

Best,

Erik

1 Like