Protect users who get spammy phishing links in emails

Networking DNS
40

@anon65703081
I think the core developers are waiting for a actual push from Tim as you can read in the topic. IPset is now available since 2016 but it is not that often used at it seems (may i am wrong with this). I think i won´t use Tim´s development even it is really great what he have done but i simply use own lists and wanted to have a more liberal choice.

@anon45931963
To go one step further with your request, have modified now the script and integrated only your list.

Script looks like this:

#!/bin/bash -

set -x

#
# Test script for individual automation for custom blacklists and their updates
# ummeegge[at]ipfire.org, $date Wed Oct 01 17:35:42 CEST 2015
# Modified for community request --> 
# https://community.ipfire.org/t/protect-users-who-get-spammy-phishing-links-in-emails/4111
# for a phising list. $date Sat Dez 19 10:42:23 CEST 2020
###########################################################################################
#

# Locations
INSTDIR="/tmp/customlists";
TMPLIST="/tmp/customlists/list";
SET="/var/ipfire/urlfilter/settings";
LIST="/var/ipfire/urlfilter/blacklists/custom/blocked/domains";

# Check for installation dir
if [ -d "${INSTDIR}" ]; then
	rm -rf ${INSTDIR};
    mkdir ${INSTDIR};
else
    mkdir ${INSTDIR};
fi

# Set hock for 'Customblacklist' in URL-Filter CGI and activate 'Custom blacklist'
HOCK=$(grep -Fx 'ENABLE_CUSTOM_BLACKLIST=on' ${SET})
if [[ -z ${HOCK} ]]; then
    sed -i -e 's/ENABLE_CUSTOM_BLACKLIST=.*/ENABLE_CUSTOM_BLACKLIST=on/' ${SET}
    squidGuard -c /etc/squidGuard/squidGuard.conf &
    sleep 2;
fi

#---------------------------------------------------------------------------------

# Change dir
cd ${INSTDIR};
## Download URLs and files for external Domain lists
URL1="https://phishing.army/download/phishing_army_blocklist_extended.txt";
FILE1="phishing_army_blocklist_extended.txt";
####################################################################################
## If you want to integrate your own local blacklist, delete the '#' in the USERLIST
## line and set the path to your list and the name.
#USERLIST="/path/to/list/nameoflist";
####################################################################################

## Download and process and/or integrate it
# Get all lists
wget ${URL1} >/dev/null 2>&1;
## Processing domains
#1
sed -e '/^#/d' -e '/^?/d' ${FILE1} > ${TMPLIST};

############################################################################################################
## if you want to integrate your own above defined local blacklist, delete the '#' in the '${USERLIST}' line
#cat ${USERLIST} >> ${TMPLIST};
############################################################################################################

# remove unwanted Carriage Returns
sed -i -e 's/^M//g' ${TMPLIST}

# sort and delete double entries and paste it to custom blacklist
sort ${TMPLIST} | uniq > ${LIST};

# Updating squidGuard database and restart and log it to syslog
squidGuard -C "${LIST}";
squidGuard -c /etc/squidGuard/squidGuard.conf &
sleep 2;
/etc/init.d/squid flush;
logger -t ipfire "URL filter: Custom blacklist updated";

# CleanUP
#rm -rf ${INSTDIR};

# EOF

URL-Filter and Squid are running including logging with a modified block page settings.
Your above posted list have:

$ wc -l customlists/phishing_army_blocklist_extended.txt 
19814 customlists/phishing_army_blocklist_extended.txt

entries.

Have executed it in debugging mode and checked the needed time:

$ time ./domain-phising-list 
+ INSTDIR=/tmp/customlists
+ TMPLIST=/tmp/customlists/list
+ SET=/var/ipfire/urlfilter/settings
+ LIST=/var/ipfire/urlfilter/blacklists/custom/blocked/domains
+ '[' -d /tmp/customlists ']'
+ rm -rf /tmp/customlists
+ mkdir /tmp/customlists
++ grep -Fx ENABLE_CUSTOM_BLACKLIST=on /var/ipfire/urlfilter/settings
+ HOCK=ENABLE_CUSTOM_BLACKLIST=on
+ [[ -z ENABLE_CUSTOM_BLACKLIST=on ]]
+ cd /tmp/customlists
+ URL1=https://phishing.army/download/phishing_army_blocklist_extended.txt
+ FILE1=phishing_army_blocklist_extended.txt
+ wget https://phishing.army/download/phishing_army_blocklist_extended.txt
+ sed -e '/^#/d' -e '/^?/d' phishing_army_blocklist_extended.txt
+ sed -i -e 's/^M//g' /tmp/customlists/list
+ sort /tmp/customlists/list
+ uniq
+ squidGuard -C /var/ipfire/urlfilter/blacklists/custom/blocked/domains
+ sleep 2
+ squidGuard -c /etc/squidGuard/squidGuard.conf
+ /etc/init.d/squid flush
Stopping Squid Proxy Server (this may take up to a few minutes)....                                    [  OK  ]
Creating Squid swap directories...                                                                     [  OK  ]
Starting Squid Proxy Server...
2020/12/19 10:37:18| WARNING: BCP 177 violation. Detected non-functional IPv6 loopback.                [  OK  ]
+ logger -t ipfire 'URL filter: Custom blacklist updated'
./domain-phising-list  1.80s user 1.54s system 25% cpu 13.315 total

The warning can be ignored --> Squid web proxy started with 2 errors .

URL-Filter custom-blacklist:
url-filter_after_exporting

Just tested it with hxxx://102312.*.com/ which looks like this:

block_page
block_page1601×347 30.7 KB

URL-Filter logging:

url-filter_logging
url-filter_logging1369×46 17.9 KB

Needed resources with 5 users:

$ ./ps_mem.py | grep -E 'squid|redirect_|squidGuard'
  5.0 MiB +   1.8 MiB =   6.8 MiB	squidGuard (4)
 21.7 MiB +   3.7 MiB =  25.4 MiB	redirect_wrappe (4)
 24.7 MiB +   3.0 MiB =  27.7 MiB	squid (2)

To get a little more butter to the fish as we say it here.

Best,

Erik

EDIT: Wiki for “Web Proxy Auto-Discovery Protocol (WPAD) / Proxy Auto-Config (PAC)” --> https://wiki.ipfire.org/configuration/network/proxy/extend/wpad .

3 Likes