IPFire 2.27 (Core 168) - suricata not working

xperimental · 14 June 2022 18:48

I installed core 168 on a new system and therefore checked the functionality.

|20:37:20|suricata: |This is Suricata version 5.0.9 RELEASE running in SYSTEM mode|
|---|---|---|
|20:37:20|suricata: |CPUs/cores online: 4|
|20:37:20|suricata: |HTTP memcap: 268435456|
|20:37:20|suricata: |Enabling fail-open on queue|
|20:37:20|suricata: |NFQ running in REPEAT mode with mark 2147483648/2147483648|
|20:37:20|suricata: |dropped the caps for main thread|
|20:37:20|suricata: |fast output device (regular) initialized: fast.log|
|20:37:20|suricata: |Packets will start being processed before signatures are active.|
|20:37:20|suricata: |binding this thread 0 to queue '0'|
|20:37:20|suricata: |setting queue length to 4096|
|20:37:20|suricata: |setting nfnl bufsize to 6144000|
|20:37:20|suricata: |NFQ running in 'workers' runmode, will not use mutex.|
|20:37:20|suricata: |fail-open mode should be set on queue|
|20:37:20|suricata: |binding this thread 1 to queue '1'|
|20:37:20|suricata: |setting queue length to 4096|
|20:37:20|suricata: |setting nfnl bufsize to 6144000|
|20:37:20|suricata: |NFQ running in 'workers' runmode, will not use mutex.|
|20:37:20|suricata: |fail-open mode should be set on queue|
|20:37:20|suricata: |binding this thread 2 to queue '2'|
|20:37:20|suricata: |setting queue length to 4096|
|20:37:20|suricata: |setting nfnl bufsize to 6144000|
|20:37:20|suricata: |NFQ running in 'workers' runmode, will not use mutex.|
|20:37:20|suricata: |fail-open mode should be set on queue|
|20:37:20|suricata: |binding this thread 3 to queue '3'|
|20:37:20|suricata: |setting queue length to 4096|
|20:37:20|suricata: |setting nfnl bufsize to 6144000|
|20:37:20|suricata: |NFQ running in 'workers' runmode, will not use mutex.|
|20:37:20|suricata: |fail-open mode should be set on queue|
|20:37:20|suricata: |all 4 packet processing threads, 2 management threads initialized, engine starte d.|
|20:37:20|suricata: |rule reload starting|
|20:37:20|suricata: |Including configuration file /var/ipfire/suricata/suricata-homenet.yaml.|
|20:37:20|suricata: |Including configuration file /var/ipfire/suricata/suricata-dns-servers.yaml.|
|20:37:20|suricata: |Including configuration file /var/ipfire/suricata/suricata-http-ports.yaml.|
|20:37:20|suricata: |Including configuration file /var/ipfire/suricata/suricata-used-rulesfiles.yaml.|
|20:37:20|suricata: |[ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - Invalid rule-files configuration sectio n: expected a list of filenames.|
|20:37:20|suricata: |No signatures supplied.|
|20:37:21|suricata: |cleaning up signature grouping structure... complete|
|20:37:21|suricata: |rule reload complete|
|20:37:21|suricata: |Signature(s) loaded, Detect thread(s) activated.|
|20:39:01|suricata: |rule reload starting|
|20:39:01|suricata: |Including configuration file /var/ipfire/suricata/suricata-homenet.yaml.|
|20:39:01|suricata: |Including configuration file /var/ipfire/suricata/suricata-dns-servers.yaml.|
|20:39:01|suricata: |Including configuration file /var/ipfire/suricata/suricata-http-ports.yaml.|
|20:39:01|suricata: |Including configuration file /var/ipfire/suricata/suricata-used-rulesfiles.yaml.|
|20:39:01|suricata: |[ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - Invalid rule-files configuration sectio n: expected a list of filenames.|
|20:39:01|suricata: |No signatures supplied.|
|20:39:01|suricata: |cleaning up signature grouping structure... complete|
|20:39:01|suricata: |rule reload complete|

I tried two different rule-sets and it always fails to load them and therefore doesn’t work at all.

pmueller · 15 June 2022 19:55

Hi,

thank you for reporting back.

Hm. Suricata is working fine with the ET Community ruleset here, but the logs you posted indicate something broken indeed.

@stevee: May I ask you to have a look at @xperimental’s case?

Thanks, and best regards,
Peter Müller

xperimental · 15 June 2022 20:46

I don’t have that problem with build 167 (but other problems there).

Also whenever I delete a ruleset the webui doesn’t refresh automatically.

xperimental · 16 June 2022 10:14

I’ve been browsing the web looking for similar issues: Do not download the rules of the IPS Suricata - Support - NethServer Community

So I tried to ping rules.emergingthreats.net
100% packet loss

But I can ping emergingthreats.net and any other IP and Domain it tried.

However any client can access rules.emergingthreats.net via webbrowser, but can’t ping, too.

I really think it’s an download problem, because I don’t have any rulesets .

xperimental · 16 June 2022 11:01

Don’t ask me why I haven’t tried and checked it before.

I unchecked/disabled the rule and checked/enabled it again. I checked the ruleset and this time it was not empty anymore (I actually didn’t know that it was empty before).

Now it’s working .

stevee · 16 June 2022 11:05

Hello Terry,

thanks for reporting your issue.

Sadly your posted suricata startup log, only tells us, that the generated used-rulefiles.yaml is not correct/valid.

Please post it’s content here, may this will bring some light into the dark.

How did you configure your IDS/IPS, which providers do you use ?

Where any rule categories displayed, when using the “Customize Ruleset” button?

-Stefan

PS: I also cannot ping the “rules.emergingthreads.net” host but access via browser, so this behavior seems to be normal for me.

xperimental · 16 June 2022 11:43

I don’t know out of the box the location of this file. However the problem is solved and that’s why I don’t think the file is still not correct.

I only use the “Emergingthreats.net Community-Regelsatz”

That was the point. I’ve created a standard configuration for ipfire that’s used to be restored (as backup) on new installations. So did I with this new setup. IPS is enabled on red with this ruleset. When I restored my default config, the ruleset was there and checked as active. But suricata came up with this error. So I removed and added the ruleset multiple times, but never checked the rules within this ruleset. However even checked as active, the ruleset was empty. So I checked and unchecked the ruleset. After that the rules appeared, but were not checked as supposed to be (in the backup). I checked some rules and restarted suricata.

Tataa! Now it works again. So something went wrong with the restore of the backup for suricata or there’s something wrong with the restoration of suricata in principal. That checkbox issue is not new to me. I’ve encountered that problem in the past with the webproxy. However in the meanwhile this has been fixed .

I’m getting still warnings as in previous builds, so nothing new, but I still wonder and since it looks like I don’t speak the programmers language I have no idea what’s that supposed to mean and if it’s important and should be checked/solved.

12:59:07	suricata:	rule reload starting
12:59:07	suricata:	Including configuration file /var/ipfire/suricata/suricata-homenet.yaml.
12:59:07	suricata:	Including configuration file /var/ipfire/suricata/suricata-dns-servers.yaml.
12:59:07	suricata:	Including configuration file /var/ipfire/suricata/suricata-http-ports.yaml.
12:59:07	suricata:	Including configuration file /var/ipfire/suricata/suricata-used-rulesfiles.yaml.
12:59:14	suricata:	27 rule files processed. 11958 rules successfully loaded, 0 rules failed
12:59:14	suricata:	Threshold config parsed: 0 rule(s) found
12:59:14	suricata:	11958 signatures processed. 1 are IP-only rules, 2041 are inspecting packet payl oad, 9720 inspect application layer, 105 are decoder event only
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘is_proto_irc’ is checked but not set. Checked in 2002029 and 4 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.http.javaclient.vulnerable’ is che cked but not set. Checked in 2013036 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.ELFDownload’ is checked but not se t. Checked in 2019896 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘et.DocVBAProject’ is checked but not set. Checked in 2020170 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.MSSQL’ is checked but not set. Che cked in 2020569 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.wininet.UA’ is checked but not set . Checked in 2021312 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘et.MS.XMLHTTP.ip.request’ is checked but not set. Checked in 2022050 and 1 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘et.MS.XMLHTTP.no.exe.request’ is chec ked but not set. Checked in 2022053 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘et.MCOFF’ is checked but not set. Che cked in 2022303 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘et.MS.WinHttpRequest.no.exe.request’ is checked but not set. Checked in 2022653 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.http.binary’ is checked but not se t. Checked in 2023741 and 2 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.armwget’ is checked but not set. C hecked in 2024242 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘et.IE7.NoRef.NoCookie’ is checked but not set. Checked in 2023671 and 7 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.smb.binary’ is checked but not set . Checked in 2027402 and 4 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.Socks5.OnionReq’ is checked but no t set. Checked in 2027704 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.http.javaclient’ is checked but no t set. Checked in 2015657 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.autoit.ua’ is checked but not set. Checked in 2019165 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘min.gethttp’ is checked but not set. Checked in 2023711 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.tcpraw.png’ is checked but not set . Checked in 2035477 and 0 other sigs
12:59:14	suricata:	[ERRCODE: SC_WARN_FLOWBIT(306)] - flowbit ‘ET.telnet.busybox’ is checked but not set. Checked in 2023019 and 2 other sigs
13:00:13	suricata:	cleaning up signature grouping structure… complete
13:00:13	suricata:	rule reload complete
13:00:13	suricata:	Signature(s) loaded, Detect thread(s) activated.

bonnietwin · 16 June 2022 11:58

Do you update this backup after every core update. IPS was modified significantly recently with multiple providers etc and maybe there is a problem if a backup from an older version of IPS is being restored.

stevee · 16 June 2022 12:01

Great that this is working now.

The error lines about the FLOWBITS are from some broken rules from the ruleset provider. Suricate skips those rules, so they do not harm the system and shows those notifications instead.

Sadly this issue/messages only can be solved by fixing the rules upstream on the provider side.

Best regards,

-Stefan

xperimental · 16 June 2022 12:39

Yes I have a separate ipfire installation (uSD card) for my Raspberry Pi 4B that is only for testing of new core updates (actually there was not much testing in the past because the update cycles are too short for me). After not having found any major problems I create a backup of this installation and use it for new installations + www.ipfire.org - Migrate to new hardware

OK thanks for the info. As long as it’s not significant and can’t be fixed by myself, there’s no work to do.