Threatfox IPS Rules

Security Intrusion Prevention
1

I added new source for Suricata IPS rules from Abuse.ch

ThreatFox is a free platform from abuse.ch with the goal of sharing indicators of compromise (IOCs) associated with malware with the infosec community, AV vendors and threat intelligence providers.

Rules are available in different formats. i.e. DNS Response Policy Zone (RPZ), JSON etc…

If anyone is interested adding this to Suricata,
ruleset-sources.zip (1.5 KB)

I attached ruleset-sources, you can just upload it to your ipfire using SSH/

I commented out the changes, but as always, good idea is to backup your old file:

/var/ipfire/suricata/ruleset-sources

4 Likes
2

If anyone is interested here is

  1. DNS Response Policy Zone (RPZ) that can be used with Unbound

https://threatfox.abuse.ch/downloads/threatfox.rpz

  1. Simple domain list that can be used with pi-hole ad-guard etc.

https://threatfox.abuse.ch/downloads/hostfile/

1 Like
3

I had never heard of threat fox before.

It might help others to post links about threatfox. And other info (e.g., reviews).

1 Like
4

edit

A quick search:
E.g. the following pages mention threatfox, in 2022.

Regards

1 Like
5

I also just heard about TF few weeks ago but I see Threathfox’s Git hub account is about 4 years old,

Threatfox is a relatively new security platform or database but I see that few other vendors picked it up for reference. One of the benefits of TF is that their platform is Open and not closed source

ThreatFox

makes it easy to search , share and export indicators of compromise associated with malware.

You can export these as MISP events, Suricata IDS Ruleset, Domain Host files, DNS Response Policy Zone, JSON files and CSV files.

6

I got really excited when I discovered the new Suricata IDS rules called ThreatFox.

Boy, I totally underestimated how much resources 30 MB of rules can take from my appliance. It has been constantly using 2GB of precious memory and keeping my memory utilization at around 75%…

The Gzipped version of the same rules is only 1,5 MB currently and there are about 83.000 rules just for illustration.

7

Since there are currently 83,000 rules it makes no sense to go through them and see which ones are unnecessary,

I tried to see if I could find 2 random IPs (a CnC server) from threatfox overlapping in IP blocklists or other ruleset.

I checked Spamhaus DROP list, ET Community rules (52 lists currently), Feodo Aggressive and I couldn’t find these 2 threafox IPs overlapping on other blocklists.

Nothing to conclude, just a basic observation for starters

Below is an example of threatfox rules

alert tcp $HOME_NET any -> [61.19.254.6] 2024  (msg:"ThreatFox Cobalt Strike botnet C2 traffic (ip:port - confidence level: 100%)"; threshold: type limit, track by_src, seconds 60, count 1; reference:url, threatfox.abuse.ch/ioc/1235384/; target:src_ip; metadata: confidence_level 100, first_seen 2024_01_30; classtype:trojan-activity; sid:91235384; rev:1;)
alert tcp $HOME_NET any -> [39.105.51.11] 28101  (msg:"ThreatFox Cobalt Strike botnet C2 traffic (ip:port - confidence level: 100%)"; threshold: type limit, track by_src, seconds 60, count 1; reference:url, threatfox.abuse.ch/ioc/1235385/; target:src_ip; metadata: confidence_level 100, first_seen 2024_01_30; classtype:trojan-activity; sid:91235385; rev:1;)
alert tcp $HOME_NET any -> [39.105.51.11] 28104  (msg:"ThreatFox Cobalt Strike botnet C2 traffic (ip:port - confidence level: 100%)"; threshold: type limit, track by_src, seconds 60, count 1; reference:url, threatfox.abuse.ch/ioc/1235386/; target:src_ip; metadata: confidence_level 100, first_seen 2024_01_30; classtype:trojan-activity; sid:91235386; rev:1;)
alert tcp $HOME_NET any -> [186.169.37.61] 5552  (msg:"ThreatFox NjRAT botnet C2 traffic (ip:port - confidence level: 75%)"; threshold: type limit, track by_src, seconds 60, count 1; reference:url, threatfox.abuse.ch/ioc/1235391/; target:src_ip; metadata: confidence_level 75, first_seen 2024_01_30; classtype:trojan-activity; sid:91235391; rev:1;)
alert tcp $HOME_NET any -> [18.197.239.5] 14272  (msg:"ThreatFox NjRAT botnet C2 traffic (ip:port - confidence level: 75%)"; threshold: type limit, track by_src, seconds 60, count 1; reference:url, threatfox.abuse.ch/ioc/1235399/; target:src_ip; metadata: confidence_level 75, first_seen 2024_01_30; classtype:trojan-activity; sid:91235399; rev:1;)
alert tcp $HOME_NET any -> [195.144.21.204] 1312  (msg:"ThreatFox Mirai botnet C2 traffic (ip:port - confidence level: 75%)"; threshold: type limit, track by_src, seconds 60, count 1; reference:url, threatfox.abuse.ch/ioc/1235406/; target:src_ip; metadata: confidence_level 75, first_seen 2024_01_30; classtype:trojan-activity; sid:91235406; rev:1;)
8

Hi Everyone,

I’ve spent a few weeks testing the ThreatFox IPS ruleset on IPFire and wanted to share my observations and a potential strategy for optimisation.

Observations:

  • Resource Usage: The ruleset is notably resource-intensive. On my system, it consumes approximately 2 GB of RAM.
  • CPU Utilization: When the ruleset updates, there’s a noticeable spike in CPU usage, with a single core hitting 100% utilization for 1–3 minutes. This is observed even on a relatively new i3-N305 processor.

Data Analysis:

  • Upon examining the ruleset, I found it comprises 36,961 IP address-based rules and 49,047 other types of rules.

Proposed Optimisation Strategy:

  • Disabling IP Address-Based Rules: Considering the high number of IP-based rules, we could disable them in bulk to reduce the load.
  • Migrating to IPFire’s IP Address Blocklists: Replacing these rules with IPFire’s more resource-friendly IP Address Blocklists could be a viable solution. Notably, ThreatFox offers these blocklists in .json and .csv formats, which could be integrated into IPFire.

I’m keen to hear your thoughts on this approach, or any experiences you might have had with similar configurations.

1 Like
10

A G, I agree with your observations,

Regarding optimization, how would you disable IP based rules?
Unfortunately Threatfox.abuse ch do not provide an IP blocklist.

If you scroll up, they provide domain based blocklists:

11

After searching some more I found that this Github user offers a list of IP addresses compiled from Threatfox.

Just a word of warning that this github account is in no way affiliated with the official Threatfox Abuse project. so I don’t know how useful or practical this IP blocklist is.

Machine-readable .txt IP blocklist from ThreatFox by Abuse.ch, updated every hour.

Direct link to the IP blocklist above:
https://raw.githubusercontent.com/elliotwutingfeng/ThreatFox-IOC-IPs/main/ips.txt

1 Like
12

I concur that, as far as I’m aware, there isn’t a straightforward method to disable all IP-based rules directly within Suricata, especially not through the WUI without it being a time-consuming process. However, integrating the JSON or CSV file provided by ThreatFox could be a viable alternative, depending on the compatible formats for IPFire’s IP Blocklist filter.

In terms of system resource utilization, I would label this ruleset as moderately heavy in the Wiki. While an additional 2 GB of RAM usage might be negligible for some systems, it could be significant for older hardware or smaller VM setups. Therefore, it’s important to consider the specific capabilities and limitations of your system before implementing this ruleset. For those with sufficient resources, the added memory usage might be an acceptable trade-off for the security benefits provided.

13

Or simply… post the ram increase before and after adoption, with date of information?
Negligeble, moderately heavy or unsostainable should be a sysadmin task to declare.

1 Like
14

@pike_it

Great idea about sharing the RAM usage data for the ThreatFox ruleset. This will help everyone see exactly how much memory it uses.

I agree that terms like ‘moderately heavy’ can mean different things to different people. Clear RAM usage figures will make it easier for sysadmins to decide if this ruleset is right for their system.

I’ll work on getting this info, and I’ll update the Wiki once Core Update 183 has been released. Thanks for the suggestion!

2 Likes
15

RAM usage seems to vary, at least in my cases, here is my data of increased RAM usage after customizing IPS rules, and checking “Threatfox IPS Rules” and hitting APPLY,

APU2 - 4GB RAM -AMD- Jaguar - RAM usage increased 2GB
old desktop #1 - 4GB RAM - Intel i3 - RAM usage increased 1.3 GB
old desktop #2 - 8GB RAM - Intel i5 - RAM usage increased 0.7GB

Looks like available the RAM usage increase is negatively correlated to available RAM

There might be other factors than hardware but I thought it is interesting to list.
I wonder if Kernel versions have an effect besides the CPU and total RAM available

1 Like