What core version are you on?
I see this about the IPS.
Hi there,
dzenc beat me to it. I too am wondering how to disable the default suricata rule sets. Normally I run Emergingthreats.net Community Rules, but for debugging purposes, I am currently running no rule sets, and the WUI complains about this. However, the IPS logs are filling up fast. All logged events look like this:
I have also verified that the change in behavior of suricata occurred in the upgrade from CU 167 to 168. No SURICATA IKEv2 (and the like) events prior to CU168 update. The update from 167 to 168 seems to have awaken the suricata baked in rule-sets. If one looks at some of the documentation, such as https://tools.emergingthreats.net/docs/ETPro%20Rule%20Categories.pdf which I found in the IPfire IPS wiki, one can find that a number of ET rules sets are by default included in suricata 5.0 or later. Some of these are Adware-PUP, Coinmining, Current Events and Malware to name a few.
I have not had time to fully decipher where everything is but here are a few places to begin looking:
/usr/share/suricata/
/usr/share/suricata/rules/
/etc/suricata/
/var/ipfire/suricata
Based on a examination of these directories, it would appear that suricata has more rule set built in than just the ET rule sets. Hopefully we can figure out how to turn off some of these. My plan is to dig into this on the weekend. When my android phone is on the net work, if fills up the IPS log to the point of making the log useless.
PZ
3 Likes
The IPFire ids-functions.pl has a code section that searches through the application layer protocols being used in your IPFire and looks for the related rulesets for that protocol in /usr/share/suricata/rules/
These are then included into the /var/ipfire/suricata/suricata-used-rulesfiles.yaml file. In my case two of the default rules are not included, presumably because those application layer protocols are not being used.
The IPFire code deciding which of these application layer protocol default rules should be included is in lines 1457 to 1483 in /var/ipfire/ids-functions.pl or you can find the file in the IPFire git repository in the tree under config/cfgroot/ids-functions.pl
My view would be to be careful about deciding to remove some of the Application Layer Protocol checks that have been put in place by the IPFire developers. If you have a lot of errors in place then maybe you have some applications that are doing incorrect things and that need to be fixed. Or there could be false positives, which then should be fixed.
If you have data showing that these are incorrect errors then you could raise a bug about them.
5 Likes
Thanks for the detailed info. I was able to review the code and decided that the simplest answer was to remove/rename the offending rules files in /usr/share/suricata/rules and force the yaml to be rebuilt. But there really needs to be a GUI method to do this, because this is now broken in 168.
To respond to your comment about leaving them alone: I don’t know if they are valid or invalid rules but, they are all trivial things like “SURICATA HTTP Request unrecognized authorization method” spewing out constantly and filling the logs. There’s nothing actionable from this message in terms of intrusion prevention (it’s more likely an imperfect use of the HTTP protocol than an attempted intrusion) and, worse, it’s filling up my IDS logs to the point that actually useful information is not visible.
Before core 168, my IDS logs would indicate an occasional real attempted attack - maybe once or twice a day - with an actual attack signature triggered. This was useful information and I could see the IDS was working properly. Now I can not locate these attacks because my logs are filled with trivial protocol errors, obscuring the real and useful information. It’s fine if this is the default behavior (no, really it’s not, but that’s for others to decide), but IMHO, there NEEDS to be a way to disable these rules individually or collectively so that the important events are clearly visible.
Again, this is new to 168, so something broke that used to work.
1 Like
I don’t think CU 168 broke something in IPS. IPS just flags more illegal proceedings of the smartphone OS.
With filling up the logs, you are right. But this should be discussed more deeply. As you found the rules are not false, but are trigged by faulty devices.
To manage this per GUI, it would be nice to group these annoying cases. These group(s) could be enabled or disabled. Not all users go such deep in analysis like you, but should be able to manage these messages also.
This needs to be documented in wiki also. This possible for each member of the IPFire community with knowledge about the topic.
BTW: these changes were introduced by core developpers. These can be reached for discussion mainly in the development mailing list.
4 Likes
I have the same problem as dzenc. The log is flooded with legitimate connections, all with priority 3. Mail-traffic, kerberos, dns-queries, printer-queries, etc.
I’m not really familiar with suricata now, but is it possible, that priority 3 means just monitoring, because this connections are not blocked. All real attack-rules (Emergingthreatst Community Rules) are logged with priority 2. So wouldn’t it be enough, if we could set which priority should be logged?
2 Likes
Hi all,
thanks for raising this issue and the lively discussion.
To be honest, the value of Suricata’s own IPS rules is rather limited to me as well. Even very small networks are already way too noisy to investigate on IPS hits generated by those.
@stevee: What is your opinion on that front? Do you see a better balance between making use of Suricata’s own rules and keeping the IPS logs relevant to NOC/SOC/IT security folks?
Thanks, and best regards,
Peter Müller
3 Likes
Out of curiosity, how are Suricata’s priorities used in IPFire. The Suricata documentation indicates priorities can range from 1 to 255 (priorities 1 to 4 being the most common). The priority is used to dictate the order for the processing the rule sets. A priority 1 rule is processed before a priority 2 and so on.
Looking back through the logs, I see many of the ET rules triggered with priority 1 or 2 and Suricata’s built in set being logged with a priority of 3. Are all IPS rule set blocked regardless of priority or are Suricata’s IPS actions based on priority, that is priority 1 is blocked with other priorities being some other action?
Thanks for the help, PZ
The standard rules from my attachment with priority 3 are definitely not blocked, only logged. ET rules with priority 1-2 are blocked.
1 Like
Hello,
Here are the files that tells suricata to use Default rules
/etc/suricatasuricata.yaml
Contains this line:
# Include default rules.
include: /var/ipfire/suricata/suricata-default-rules.yaml
and /var/ipfire/suricata/suricata-default-rules.yaml contains
cat /var/ipfire/suricata/suricata-default-rules.yaml
%YAML 1.1
---
#Autogenerated file. Any custom changes will be overwritten!
- /var/lib/suricata/whitelist.rules
#Default rules for used application layer protocols.
- /usr/share/suricata/rules/app-layer-events.rules
- /usr/share/suricata/rules/decoder-events.rules
- /usr/share/suricata/rules/dhcp-events.rules
- /usr/share/suricata/rules/dns-events.rules
- /usr/share/suricata/rules/files.rules
- /usr/share/suricata/rules/http-events.rules
- /usr/share/suricata/rules/ipsec-events.rules
- /usr/share/suricata/rules/kerberos-events.rules
- /usr/share/suricata/rules/nfs-events.rules
- /usr/share/suricata/rules/ntp-events.rules
- /usr/share/suricata/rules/smb-events.rules
- /usr/share/suricata/rules/smtp-events.rules
- /usr/share/suricata/rules/stream-events.rules
- /usr/share/suricata/rules/tls-events.rules
How suricata is started: with config file /etc/suricata/suricata.yaml
/usr/bin/suricata -c /etc/suricata/suricata.yaml -D -q 0:3
Hope it helps
Late edit: same /etc/suricata/suricata.yaml tells suricata to use threshold.config from /usr/share/suricata/rules/
And then oinkmaster.conf tells suricata to skip it
And same oinkmaster.conf tells suricata to also skip local.rules the user manually created
# have put some local rules in our own local.rules and we don't want it
# to get overwritten by the empty one from the archive after each
# update.
skipfile local.rules
So on and so forth…
These 2 files /etc/suricata/suricata.yaml and /var/ipfire/suricata/oinkmaster.conf controlls suricata behavior…
2 Likes
Is this an old setup? On my system the following files do not exist:
/var/ipfire/suricata/suricata-default-rules.yaml
/var/ipfire/suricata/oinkmaster.conf
But /var/ipfire/suricata/suricata-used-rules.yaml is included by suricata.yaml and in this file are the standard-rules. After commenting them out, it works for me. But they’re back, when the rules are updated.
I don’t believe oinkmaster.conf configures suricata, does it? It configures the automatic rules updater. If so, it’s not telling suricata to skip those files. It’s telling the updater not to change them.
At least, that’s my understanding (but I haven’t verified it.)
-Dan
It does during rules upgrade - this files controll the update process to add, or bypass rules or setting from various files (threshold.config, local.rules, etc)
Not that old: core 167
Suricata changed significantly from CU167 to CU168 with multiple providers and modified update process etc.
What interfaces have you enabled ISP on?
I Have just updated to CU169
ISP on GREEN
Will monitor logs.
I’m sure it makes more sense to enable on RED.
I also have problems with a lot of logs, see How to stop IPS "SURICATA ICMPv4 invalid checksum" logs?
Hi!
I have fireinfo.ipfire.org - Profile 72a721ca22ee04f59b30ada2f17f4a85fb1ebd07 and I am using Emergingthreats.net Community Rules 2022-08-10 00:08:52.It seems like my HarmonyHub ( Harmony Hub = remote control system for TV etc = 192.168.0.57) is spamming the IPS logs every 30 seconds with:
Date: 08/11 10:08:02 Name: SURICATA ICMPv4 invalid checksum Priority: 3 Type: Generic Protocol Command Decode IP info: 192.168.0.254:0 → 192.168.0.57:0 References: none found SID: 2200076 It started like a month ago, don’t know if it was after update of HarmonyHub or Ipfire.
I have searched in WUI - Firewall - Intrusion Prevention - Customize Ruleset, but can’t find what to turn off to stop the flood.
While it is annoying, it is also making it hard to see if there are other important stuff in the logs that I should know about.
Any tips for fixing this, that do not include threats and/or violence against the stupid HarmonyHub?
So maybe I can temporary edit /usr/share/suricata/rules/decoder-events.rules, eg under # checksum rules comment out this:
alert icmp any any -> any any (msg:"SURICATA ICMPv4 invalid checksum"; icmpv4-csum:invalid; classtype:protocol-command-decode; sid:2200076; rev:2;)
Maybe with:
sed -i 's/alert icmp any any -> any any (msg:\"SURICATA ICMPv4/#alert icmp any any -> any any (msg:\"SURICATA ICMPv4/g' /usr/share/suricata/rules/decoder-events.rules
/etc/init.d/suricata restart
Is it possible to run the above automatically every time the rules have been updated? Naturally temporary until something better is figured out. Maybe put above commands in tempskipicmp.sh, then do fcrontab -e and edit
@ 12h [ -f "/var/ipfire/red/active" ] && /usr/local/bin/update-ids-ruleset >/dev/null 2>&1
to
@ 12h [ -f "/var/ipfire/red/active" ] && /usr/local/bin/update-ids-ruleset >/dev/null 2>&1 && /root/tempskipicmp.sh
This may not help, but on my home system I need to disable a few of the Suricata rules due to a high volume of false-positives.
As a temporary work-around, I’ve found a configuration which works. (Note: I made a few attempts but I believe these are the steps which succeeded after a normal ruleset update)
The emerging-threats ruleset is enabled on my system. There is a /var/ipfire/suricata/oinkmaster-emerging-modified-sids.conf file. It warns that it can’t be modified as it will be overwritten, but has an include reference to a file which doesn’t exist. So, I created that missing file and disabled the specific rules which were causing problems (using oinkmaster configuration, not the 2210007=disabled syntax).
For example in a /var/ipfire/suricata/oinkmaster-emerging-modified-sids.conf file
# Oinkmaster.conf included file created by me but originally referenced in default IPFire configuration
# Disable Suricata built in - "SURICATA STREAM ESTABLISHED SYNAC resend with different seq"
disablesid 2210024
# Disable Suricata built in - "SURICATA STREAM 3way handshake SYNACK with wrong ack"
disablesid 2210007
I forced a rule reload which didn’t do anything as no update was required, however by the next day it seems to have worked.
1 Like
What is this file called
Where did you put it.
I guess the path would help.
Great work to find a solution.
2 Likes
@hvacguy oops, I accidentally named the wrong file the first time.
On my system oinkmaster-provider-includes.conf already existed. It referenced a oinkmaster-emerging-modified-sids.conf file which didn’t. After creating the second and adding the appropriate entries (explained in my last post) I found that it had worked when I checked the next day.
We’ll see if it still works with the latest IPFire update. I’ll test and let you know once that update has hit my local mirrors.
1 Like