Bandwidth usage per IP / host

Hi,

I love IPIFire especially after the feature addition of “IP Address Blocklists” in latest update.

The only thing that’s bugging me is lack of an easy way to monitor and keep record of per IP bandwidth usage.

Can someone guide me with main steps for a DYI approach they are using to achieve what I am after – that is, to visually monitor and keep historical record of per IP data usage?

The command line tools like iptraf-ng and pmacct are not for me.

I tried to pull data from snmp to Zabbix (server and agent set-up) and while I got some data being pulled from IPFire, I could not get per host data because of my lack of experience with these tools, on first time use. Also, Zabbix server seems heavy on my computer.

I tried the telegraf+net-snmp+InfluxDB+Grafana route, but with little success.

The tools listed here Ecosystem · pmacct/pmacct Wiki · GitHub are decade old in some cases, and / or not updated in a very very long time.


I am willing to get my hands dirty; but I need a clean solution that does not take days to set-up.

Even if some of you reading this do not have set-up something like this, but can share a solution that might work, I’ll take it.

Please help.

I tried a few different packages to do the same and none really worked except for pmacct.

2 Likes

Thank you Jon. What would be a decent way to pull pmacct data and show it visually on a separate computer? Aside from the way one of our friend here (I read that) doing it in Excel sheet + Pivot tables.

And for the devs, just wondering, I’ve known IPfire from IPCop days; is this feature a really tough nut to crack? Programming/effort wise.

pmacct can be set-up to store in a database called sqlite3. And with sqlite3 it is fairly easy to export to a csv file.

Keep in mind if you store pmacct in a database once per minute (or even once every 5 minutes) the csv export will be way too big for Excel. You may need to store once per hour to keep things manageable.

The slightly harder part would be for you to create a script to automate…

tough: “maybe”. impossible: “no” You don’t need any heavy duty skills to do the above. If you are comfortable woking in the console / Terminal and know some programming you should be fine.

2 Likes

Hi all,
for the first @jon, great to see your “Biggest User project” guide and your effort in this project :+1: .
According to the CSV formatting, you can do this also without sqlite but with the memory table plugin → pmacct/QUICKSTART at master · pmacct/pmacct · GitHub which i think is the cheapest solution according to the storage.

For a webinterface look like, there should be different examples already available may in the logs section, one might be also the iptables.cgi with not that much modification you can sort the Pmacct output also by bytes e.g.:

to bring on some possibilities into this, there needs to be for sure more effort pmacctctrl.c etc. but this should be nevertheless achievable.

Best,

Erik

3 Likes

Whoa! Jon thank you for your guide. New to forum, I had actually scrolled over it thinking this was your signature :man_facepalming:
263--20-Sep-2022--21-39-32

Erik’s mention of your “guide” made me look again.

Thank you, again, this is definitely a great start. And I’ll try to set it up and get back.

2 Likes

Hello @jon

I hope you are well.

I am on Step 2 of your guide and I am encountering an error when I run this:

./addr_byMonth.sh /var/spool/pmacct/pmacct_sqlitev1a.db
Parse error near line 2: near ".": syntax error
  .timer ON      CREATE VIEW         addr_byMonth (ip_addr, mac_addr, totalBytes
  ^--- error here

How would I fix that.

Ah, never mind. Found the solution. The dot commands can’t have any white space in beginning, the doc says:

A dot-command has a more restrictive structure:

- It must begin with its "." at the left margin with no preceding whitespace.
- It must be entirely contained on a single input line.
- It cannot occur in the middle of an ordinary SQL statement. In other words, it cannot occur at a continuation prompt.
- There is no comment syntax for dot-commands.
1 Like

Hello @jon

If I were to change totalBytes to totalMBytes in Step 1, how would I switch to it?

Open Step two:

and look for this line:

So totalMBytes would be similar to this by removing one of the /1000. The highlighted actually adds one column.

Thank you, this is great.