Can I help progress version2 of ipblacklist addon

I have been using the extremely useful version 1 of the ipblacklist addon to drop connections from a botnet attack on my email server but I am getting problems because some of the blocklists which I am trying to use have too many entries for default ipset maxelem value of 65536 .

I can see that Tim FitzGeorge has a new version (2) which among numerous other improvements would increase the maxelem value to suit the blocklist size. (For instance I have tried to use IPSUM and FIREHOL_LEVEL_4 blocklists both of which exceed the default maxelem in version 1).

As Version 2 seems to have been dormant since last May is there anything I can do to help this version becoming available. I am more than willing to help if I can.

Rob

1 Like

hi @helix,

so let’s hope that @timf will answer you…

1 Like

I am rather disappointed that there hasn’t been any further response to this topic. I believe from the IPFire development lists that the code was sufficiently advanced for IPBlacklist V2 to be merged into IPFire in the middle of last year but then seems to have fallen into a black hole. I hope that this is still the intended plan but the lack of any visibility on any issues that could have held it back is worrying me that it might have been dropped from IPFire.
Perhaps someone can enlighten me on what are the plans for this delightful addon as it would be a great shame if it was lost to the community.

Rob.

1 Like

The response needs to come from @timf

A patch had been submitted in May 2020. There were some questions back and forth on that and in Dec 2020 the source was changed because of changes in iptables but an updated patch was not supplied.

In Feb 2021 an updated patch set was requested but no response has been forthcoming to that, neither on the forum or the development mailing list.

Since the patch to implement the addon was submitted there have been changes to the source code and to IPFire itself.

As the IPFire core devs team is quite small they require that if someone wants to have an addon implemented into IPFire that the originator of that code needs to be willing to support that addon on an ongoing basis to provide fixes, updates etc.

If you are able to pick up the addon from timf and progress it and support it’s inclusion into IPFire then that would be welcomed. The wiki has sections about how to access the git source to make any required additions and changes and then to submit patches for those.

1 Like

Thank you so much for your reply Adolf

Sorry I incorrectly thought this was last years date, I now see as you stated it was 2020.

Yes I understand and I hope @timf is OK. I don’t think he has posted here since Dec 2020 and it looks like you haven’t heard from Tim since then?

Yes, I understand.

I have looked through the V2 patches Tim supplied in May 2020 and they don’t seem to apply to the V1 code Tim made available on his Github pages. For one thing the addon seems to have had an identity crisis and I’m not sure if it should be called IPBlocklist (V1) or IPBlacklist (V2). I’ll try and contact @timf and get some clarification. If I cannot contact him are the sources for V2 available on IPFire repositories ?

I do hope that @timf can submit V2 to yourselves and as I stated at the beginning of this topic that V1 has some issues particularly with the default maxelem value.
I am more than happy to help the development team to get this version available but I don’t want to tread on anyone’s toes in the process.

Rob

Hi Rob,

Effectively the last patch series that ended up in the IPFire patchwork is the source code of Tim’s addon into IPFire.
https://patchwork.ipfire.org/project/ipfire/list/?series=1215
IPFire doesn’t have anything additional to that. Those patches fitted into Core Update 153 or 154. I think the major differences are the files that are patched in the infrastructure patch v2, 8/8 as several of these files will be very different now compared to back then. If this is different from what is in Tim’s github pages then it would be good if you are able to make contact with him. As far as I am aware no one in the dev team has heard from Tim since Dec 2020.

I don’t think you will be treading on any toes from the IPFire dev team. They have more than enough other things to work on. If you can’t get hold of Tim and/or we don’t get any response from him to this thread then you are welcome to pick it up if you feel willing and able to do that. Let us know if you will be doing this and we will look to support you with how to submit things into IPFire.

Good luck Rob.

Hi Adolf,

I too have been unable to get a reply from Tim. The last message I can find was in August 2020 where he said " been busy with COVID-19 related stuff". I hope he is OK but if I am pessimistic, I have to consider that I might never get a reply.

I downloaded the source code from patchwork, and from code inspection it looks like the huge long patch that Michael was having problems with ( [PATCH v2 0/8] ipblacklist: IP Address Blacklists ) has been incorporated into the source code on patchwork.

Was there a version of IPFire where Tims software was incorporated and tested? Other than not being able to get in contact of the maintainer, do we know if there are any other problems with the code?

I will copy the V2 code to my development IPFire system and see if I can get it to run OK on there and if OK, I could try it on my on-line firewall, however the IPblocklist V1 is very much in use protecting my mail server ATM and wouldn’t want to compromise the on-line system.

I’ll let you know the outcome.

Regards,
Rob

Hi Rob,

No it has not been merged into any IPFire version.

If you join the Development mailing list you can ask the question there.
https://wiki.ipfire.org/devel/mailing-lists

My own view is that it might be better for you to work through the code yourself and modify and bring up to date the patches based on the next branch in the IPFire git repository. Then you could submit this patch set and then see what feedback you get.

I think it would be good to find a way to split that patch you referenced into smaller groups of files. 29 different files were changed and that makes it much harder to review and figure out what the changes mean for IPFire.
I would definitively at least split the patch into separate ones dealing with the backup changes, the menu changes, the cgi page code and the ipblacklist source changes.

Good luck with your testing on your development IPFire system.

Both of my test IPFire boxes are still at 32 bit and I have been struggling to compile a 32bit binary for the 2 helper programs required for this addon. (I’ve discovered my IPFire 32bit source code has been corrupted and now no longer available)
Can you help in compiling these 2 ipblacklist c programs?

getipsetstat.c

t/* IPFire helper program - GetIPSetStat
 *
 * Get the list from IPSET LIST
 *
 */

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/types.h>
#include <fcntl.h>
#include "setuid.h"


int main(void)
{
        if (!(initsetuid()))
                exit(1);

        safe_system("/usr/sbin/ipset list -t -f /var/tmp/ipsets.txt");
        safe_system("chown nobody:nobody /var/tmp/ipsets.txt");

        return 0;
}

ipblacklistctrl.c

t/* This file is part of the IPFire Firewall.
 *
 * This program is distributed under the terms of the GNU General Public
 * Licence.  See the file COPYING for details.
 *
 */

#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <fcntl.h>
#include "setuid.h"

int main(int argc, char *argv[]) {

    if (!(initsetuid()))
        exit(1);

    if (argc < 2) {
        fprintf(stderr, "\nNo argument given.\n"
                "ipblacklistctrl (update|restore|log-on|log-off|"
                "enable|disable)\n\n");
        exit(1);
    }

    if (strcmp(argv[1], "update") == 0) {
        safe_system("/usr/local/bin/ipblacklist update >/dev/null 2>&1 &");
    } else if (strcmp(argv[1], "restore") == 0) {
        safe_system("/usr/local/bin/ipblacklist restore >/dev/null 2>&1 &");
    } else if (strcmp(argv[1], "log-on") == 0) {
        safe_system("/usr/local/bin/ipblacklist log-on >/dev/null 2>&1 &");
    } else if (strcmp(argv[1], "log-off") == 0) {
        safe_system("/usr/local/bin/ipblacklist log-off >/dev/null 2>&1 &");
    } else if (strcmp(argv[1], "enable") == 0) {
        safe_system("/usr/local/bin/ipblacklist enable >/dev/null 2>&1 &");
    } else if (strcmp(argv[1], "disable") == 0) {
        safe_system("/usr/local/bin/ipblacklist disable >/dev/null 2>&1 &");
    } else {
        fprintf(stderr, "\nBad argument given.\n"
                "ipblacklistctrl (update|restore|log-on|log-off|"
                "enable|disable)\n\n");
        exit(1);
    }

    return 0;
}

Rob

Hi Rob,

I think you are going to have problems then. 32 bit IPFire has been announced as deprecated for a year or more and end of 2021 was its EOL.
The last Core Update where it was still available was 162. Your IPFire systems will still work but you will not be able to do any updates as Core Update 163 onwards will not have 32 bit i586 code available.

Any git clone will no longer have 32 bit capability except for armv6l.

All my builds are done with an x86_64 system.

Hi Adolf,
Unfortunately one of the my development boxes runs an alix board which won’t run 64 bit code and the other has some modifications that would take some time to re-apply if I changed to 64 bit. My main firewall is 64 bit however.

I think I have all the other parts of ipblacklist available to install it’s just these 2 binaries I need to complete.

I am trying to recover my corrupted source code and get the following error when I run
sudo ./make.sh build
mkdir -pv /usr/share/doc/licenses/
cp -vf /usr/src/doc/COPYING /usr/share/doc/licenses/GPLv3
cp: cannot stat ‘/usr/src/doc/COPYING’: No such file or directory
make: *** [stage2:116: /usr/src/log/stage2] Error 1

Any thoughts on where I am going wrong.

Thanks for your help

Rob

This is indicating that you are missing a file in your source tree.

In your ipfire2.x/doc/ directory there should be a file called COPYING which is the GPL license.

I suspect that you are missing that file resulting in the cannot stat message when IPFire tries to copy that file to the /usr/share/doc/licenses/GPLv3

You should be able to download it from the IPFire git repository.

I had restored from backups to recover the corruption and it looks like I had missed that one. Odd the language files in that directory were intact. I restored the missing files from an earlier backup.

build is running OK now, many thanks for the pointer.

Rob

1 Like

I managed to set up a i586 build for v2.25 core 158 and compiled the 2 C programs and installed then on my alix test system and copied the remaining files over.
Results look ok but this box is behind my production firewall so doesn’t see any firewall hits. Looking OK but I can’t get fcron to update (maybe a permission problem) and looking into it.
I now need to decide on making this available as a stand alone add-on or patch it back into IPFire for which I would need a hand hold !
A couple of screen shots:


2 Likes

Congratulations on the progress you have made.

I would be willing to provide whatever helping hand that would be needed for creating the patch and how to submit it. Also the other people on the dev mailing list are very helpful too if there are any questions.

However to be able to submit it as a patch you are going to have to be able to build it with either an x86_64 or armv6l or aarch64 architecture.

i586 is no longer in IPFire so I believe that any patch submission can not use that architecture. To confirm you could always ask that question on the dev mailing list.

1 Like

Thank you for the offer of help. I have been subscribed to the dev mailing list for some time so I am aware of the helpfulness there.
As this add-on was originally intended to be incorporated into the main IPFire I am a bit concerned about who will maintain the program as I am not as knowledgable as @timf so I think it would require additional support for the code from others.

There isn’t any architecture specific code in ipblacklist. The 2 binary programs are compiled at build time and the remaining code is perl and some bash so should be independent of architecture.
I do intend to run it on my main x86_64 system when I have finished testing, so should confirm it runs OK.

I think I have sorted out the fcron problem and the blocklists now seem to be updating correctly.
More testing to-morrow.

Best
Rob

2 Likes

Hi Rob,

Sorry for the delay in any reply.

This is likely to be the problem aspect. If you are not able to support the ongoing maintenance of this tool then you are going to need to find someone who is capable of doing that and willing to take it up.

With no maintainer for it the I suspect that the IPFire devs would not be willing to accept it as the result would be that when problems occurred they would then be expected to figure out what the code is doing and fix it.

You could always raise the question on the dev list to see what their response would be.

I have now had a good look at @timf 's code and is well written and has comprehensive comments so that it is easy to understand. It shouldn’t be difficult to maintain.

I’ll pop along there shortly.

I now have this code working on my main firewall (IPFire 2.27 (x86_64) - Core Update 161) and I must say I’m quite impressed with the new features. The “IP Address Blacklist Logs” page is a very useful addition to see which blacklist is contributing to the most effective blocking and I have added an additional Local Blacklist which is generated by parsing my mail server logs for abusive ip addresses and adding the list to my web server.

I found a few problems:
There is a conflict between ipblacklist and ipfblocklist if it is installed. The ipsets generated by ipfblocklist are not removed when the addon is disabled via the WUI and therefore this addon should be uninstalled before ipblacklist is loaded

There are few missing files on Patchwork:
There is only the english lang file available.
There isn’t a init script included. I used a modified ipfblocklist one.
The logwatch file ipblacklist.conf isn’t included and needs to be written.
ipblacklists.dat needs a couple of patches:

  1. remove - require “${General::swroot}/geoip-functions.pl”; - This isn’t included in core 161 and not used in this program.

  2. change
    &General::readhash( “/srv/web/ipfire/html/themes/”.$mainsettings{‘THEME’}."/include/colors.txt", %color );
    to
    &General::readhash( “/srv/web/ipfire/html/themes/ipfire/include/colors.txt”, %color );

Finally the maxelem value I had problems with at the start of this topic is now solved as it is set at a value to accommodate the size of each blacklist .

Rob

1 Like

Hi @helix
Thanks for looking into this. I’ve been using V1 for years.
It is a far more preferable way to drop bad ip addrs as opposed to loading up Suricata processing with tons of rules for bad ip addrs.
I hope you can manage to get this incorporated into IPFire – either as part of core or as an addon

Thanks again,
@cbrown

2 Likes

Thank you for your support. I am hopeful it can be incorporated into IPFire, but I could turn it into an addon if the devs don’t support incorporation. From Adolf’s comments here I am hopeful it will become part of the code.

Rob

1 Like