IPS rules haven't updated since updating core to 168

Hi -

Since updating to core 168, I see that my IPS rules have stopped updating. I observed them previously updating daily.

In trying to diagnose what’s happening, I went looking for a cron entry, which I can longer find on core 168. Did the mechanism for IPS updates change? Is there something I need to do to recreate the cron entry?

I tried updating the rules manually, which worked, but didn’t cause it to start updating automatically/regularly. I also tried disabling and re-enabling “automatic updates” but no change. What else can I try?

Thanks
-Dan

Per the blog
Release notes.

Intrusion Prevention System improvements

Stefan contributed a patch series for notably improving the IPS, particularly when it comes to handling of ruleset providers. While many of the changes are done under the hood, the following are visible to the web interface:

  • Monitoring mode can now be enabled for each ruleset provider individually. This makes baselining and testing much less of a hassle, since newly introduced IPS ruleset providers can now first be used for logging only, without risking disruptions or unintended side-effects.
  • Parsing and restructuring changed or updated rulesets has been improved and is now faster by orders of magnitude.
  • The downloader will now automatically check whether a ruleset has been updated on its providers’ server by checking the ETag HTTP header. This allows us to drop the update interval selection; every IPS ruleset will now updated automatically on the appropriate interval.
1 Like

OK. But it shows that the rules have not been updated since 2022-06-16 08:54:36, and yet I know that the rule provider is updating them.

So again - how to debug?

Check that the addition into the fcrontab occurred. The entry should be at lines 65 and 66.

The entry is

65 # Perform a surciata rules update every 12 hours.
66 @ 12h [ -f "/var/ipfire/red/active" ] && /usr/local/bin/update-ids-ruleset >/dev/null 2>&1

If it is not there then something went wrong when you carried out the CU168 upgrade.

I have those lines in my fcrontab and the rules are being updated regularly.

Do you have Automatic Updates selected for each of the Providers in the Ruleset Settings table?

Thanks! It’s there. Let me try to execute it manually and see what happens…

I was able to execute /usr/local/bin/update-ids-ruleset manually, but it did not output anything, nor as far as I can tell did it send anything to the log. It just paused then exited. :frowning:

OK, I take that back. I grep’d for oinkmaster and I see this:

Jun 21 15:17:29 ipfire oinkmaster[24844]: Performing update for subscripted.
Jun 21 15:17:30 ipfire oinkmaster[24844]: Skipping subscripted - The ruleset is up-to-date

So it apparently thinks there is no update. I will see if I can dig any deeper…

My two ruleset providers are EmergingThreats.net and Abuse.ch

I don’t have any subscribed rulesets so I can’t test that.

This is strange

And… digging more deeply, it looks like the Talos rules haven’t actually been modified since 6/16. So this is my mistake :man_facepalming: Thanks for the help debugging, and apologies for the false alarm. I will revisit once the rule provider updates the rules and hopefully everything updates fine… :slight_smile:

1 Like

And… to close the loop, the rules were updated by the provider since I last posted and they have also been picked up by my ipfire automatically. So it seems like everything is working as intended, at least for me. Thanks again.

2 Likes

Seems oinkmaster process generates some error messages and I have some trouble understanding them: does this squence means it used OLD zip files from TEMP?

Logs:

Jul 25 16:38:05 grey-x86-64 oinkmaster[7671]: <ERROR> Downloading ruleset for provider: emerging.
Jul 25 16:38:07 grey-x86-64 oinkmaster[7671]: <ERROR> Downloading ruleset for provider: sslbl_blacklist.
Jul 25 16:38:07 grey-x86-64 oinkmaster[7671]: <ERROR> Downloading ruleset for provider: subscripted.
Jul 25 16:38:19 grey-x86-64 oinkmaster[7671]: Loading /var/ipfire/suricata/oinkmaster.conf
Jul 25 16:38:19 grey-x86-64 oinkmaster[7671]: Loading /var/ipfire/suricata/oinkmaster-provider-includes.conf
Jul 25 16:38:19 grey-x86-64 oinkmaster[7671]: Loading /var/ipfire/suricata/oinkmaster-emerging-modified-sids.conf
Jul 25 16:38:19 grey-x86-64 oinkmaster[7671]: Loading /var/ipfire/suricata/oinkmaster-modify-sids.conf
Jul 25 16:38:20 grey-x86-64 oinkmaster[7671]: Copying rules from /tmp/ids_tmp/rules... 169 files copied.
Jul 25 16:38:22 grey-x86-64 oinkmaster[7671]: Setting up rules structures... done.
Jul 25 16:38:45 grey-x86-64 oinkmaster[7671]: Processing downloaded rules... disabled 1, enabled 0, modified 85059, total=82942
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]: Setting up rules structures... done.
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]: Comparing new files to the old ones... done.
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]: Updating local rules files... done.
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]:
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]: [***] Results from Oinkmaster started 20220725 16:38:47 [***]
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]:
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]: [*] Rules modifications: [*]
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]:     None.
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]:
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]: [*] Non-rule line modifications: [*]
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]:     None.
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]:
Jul 25 16:38:47 grey-x86-64 oinkmaster[7671]: [+] Added files (consider updating your snort.conf to include them if needed): [+]

Late edit - logs are misleading - inside /var/tmp/ I see the files being downloaded (I erased old ones)

ls -lth /var/tmp/
total 141M
-rw------- 1 nobody nobody 3.3M Jul 25 15:43 idsrules-emerging.tar.gz
-rw------- 1 nobody nobody  93M Jul 25 15:43 idsrules-subscripted.tar.gz
-rw------- 1 nobody nobody 1.5M Jul 25 15:43 idsrules-sslbl_blacklist.rules
-rw-r--r-- 1 root   root    11M Jul 25 01:26 hosts.block.experemental
-rw-r--r-- 1 root   root    32M Jul 25 01:25 hosts.block.reverse
-rw-r--r-- 1 root   root   1.1K Jul 24 16:27 apcupsd.email

The WUI is correct: it shows that all rules were updated despite the error messages logged at the beginning of the process…