# Data usage monitoring

**URL:** <https://community.ipfire.org/t/data-usage-monitoring/707>\
**Category:** Community\
**Created:** [20 December 2019 15:28 UTC](https://community.ipfire.org/t/data-usage-monitoring/707 "2019-12-20T15:28:19Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![iotapi322](https://community.ipfire.org/user_avatar/community.ipfire.org/iotapi322/32/2958_2.png) [@iotapi322](https://community.ipfire.org/u/iotapi322)\
**Post date:** [20 December 2019 15:28 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/1 "2019-12-20T15:28:19Z")

</div>

I’ve done some searching and I see some discussion on this in the past. I’m about to switch my Comcast cable account over to a program that has 1TB a month data cap.

I want to on a monthly basis track current data usage per device and a monthly grand total.

Now I’ve seen some discussion of ntopng and iptraf.

I could use ntopng and do a port mirroring on my switch to another machine… However I would like to know if anyone else has any other solutions.

thanks,

---

<div class="post-metadata">

**Author:** ![ms](https://community.ipfire.org/user_avatar/community.ipfire.org/ms/32/16615_2.png) [@ms](https://community.ipfire.org/u/ms)\
**Post date:** [20 December 2019 16:19 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/2 "2019-12-20T16:19:34Z")

</div>

There is general collection of the data that is being passed on each interface.

However, there is no automatic deactivation of the Internet connection after a quota has been reached.

---

<div class="post-metadata">

**Author:** ![iotapi322](https://community.ipfire.org/user_avatar/community.ipfire.org/iotapi322/32/2958_2.png) [@iotapi322](https://community.ipfire.org/u/iotapi322)\
**Post date:** [20 December 2019 17:08 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/3 "2019-12-20T17:08:37Z")

</div>

I just want to monitor,  
Is there an internal database that shows per device data usage that I can sum up? Basically where is the general collection kept at ?

---

<div class="post-metadata">

**Author:** ![jon](https://community.ipfire.org/user_avatar/community.ipfire.org/jon/32/15363_2.png) [@jon](https://community.ipfire.org/u/jon)\
**Post date:** [20 December 2019 22:00 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/4 "2019-12-20T22:00:12Z")

</div>

I use Comcast and I am looking for the same thing:

[IPFire (old) Forum - Help Monitoring Data Caps](https://forum.ipfire.org/viewtopic.php?f=27&t=23475)

[IPFire (old) Forum - Traffic accounting](https://forum.ipfire.org/viewtopic.php?f=27&t=23497#p128193)

&nbsp;

The closest I came to monitoring each device was using pmacct (and with a ton of help from Erik @ummeegge!)

But I had to post process everything in Excel using pivot table to get the info I wanted. And I could not determine how to make that happen via a script.

[IPFire (new) Community - Pmacct - A lightweight passive network monitoring tool](https://community.ipfire.org/t/pmacct-a-lightweight-passive-network-monitoring-tool/47)

---

<div class="post-metadata">

**Author:** ![xeonium](https://community.ipfire.org/user_avatar/community.ipfire.org/xeonium/32/7860_2.png) [@xeonium](https://community.ipfire.org/u/xeonium)\
**Post date:** [21 December 2019 11:45 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/5 "2019-12-21T11:45:12Z")

</div>

Did you try sarg addon? Shows hourly, daily, weekly and monthly data usage per device so in my mind.

---

<div class="post-metadata">

**Author:** ![iotapi322](https://community.ipfire.org/user_avatar/community.ipfire.org/iotapi322/32/2958_2.png) [@iotapi322](https://community.ipfire.org/u/iotapi322)\
**Post date:** [21 December 2019 13:03 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/6 "2019-12-21T13:03:02Z")

</div>

Doesn’t sarg require everything to go through the squid proxy? Meaning all the bittoreent / https traffic wouldn’t get counted?

Also I don’t see sarg in the firepak list of available packages.

---

<div class="post-metadata">

**Author:** ![iotapi322](https://community.ipfire.org/user_avatar/community.ipfire.org/iotapi322/32/2958_2.png) [@iotapi322](https://community.ipfire.org/u/iotapi322)\
**Post date:** [21 December 2019 13:12 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/7 "2019-12-21T13:12:12Z")

</div>

This is very interesting!!!  
I work for Cisco specifically in the in the Cloud Ops department so doing queries in sql is no problem. This seems like a great starting point.

Thanks I’m going to start working on this and see where I end up.

---

<div class="post-metadata">

**Author:** ![jon](https://community.ipfire.org/user_avatar/community.ipfire.org/jon/32/15363_2.png) [@jon](https://community.ipfire.org/u/jon)\
**Post date:** [21 December 2019 16:59 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/8 "2019-12-21T16:59:49Z")

</div>

> [@xeonium](#):
>
> Did you try sarg addon? Shows hourly, daily, weekly and monthly data usage per device so in my mind.

For me Sarg did not give me the correct info. For all of November 2019:

- Comcast = 411 GB
  - from the Comcast website

- IPFire = 413.36 GiB
  - from IPFire WUI menu **Status** \> **Net-Traffic**

- SARG = 110.44 G Bytes
  - from the Squid Analysis Report Generator

Maybe SARG is only looking at non-SSL type data? (e.g., HTTP and not HTTPS). I am guessing!

---

<div class="post-metadata">

**Author:** ![jon](https://community.ipfire.org/user_avatar/community.ipfire.org/jon/32/15363_2.png) [@jon](https://community.ipfire.org/u/jon)\
**Post date:** [21 December 2019 17:50 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/9 "2019-12-21T17:50:15Z")

</div>

> [@iotapi322](#):
>
> I work for Cisco specifically in the in the Cloud Ops department so doing queries in sql is no problem.

Having someone that does SQL queries is worth gold! Please share what you come up with. I’d be happy to help test.

---

<div class="post-metadata">

**Author:** ![xeonium](https://community.ipfire.org/user_avatar/community.ipfire.org/xeonium/32/7860_2.png) [@xeonium](https://community.ipfire.org/u/xeonium)\
**Post date:** [21 December 2019 18:14 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/10 "2019-12-21T18:14:20Z")

</div>

> [@jon](#):
>
> Maybe SARG is only looking at non-SSL type data? (e.g., HTTP and not HTTPS). I am guessing!

You are right. Squid is non-TLS/SSL only.  
Without breaking the certificate chain of Internet data stream squid is not able to analyze all traffic.  
But that’s another topic/thread.

---

<div class="post-metadata">

**Author:** ![iotapi322](https://community.ipfire.org/user_avatar/community.ipfire.org/iotapi322/32/2958_2.png) [@iotapi322](https://community.ipfire.org/u/iotapi322)\
**Post date:** [21 December 2019 19:22 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/11 "2019-12-21T19:22:14Z")

</div>

If you want you can send me via a link a dump of your sql database and I can write some queries for you with sql and /or make sql queries in python. I do that all the time for work.

---

<div class="post-metadata">

**Author:** ![iotapi322](https://community.ipfire.org/user_avatar/community.ipfire.org/iotapi322/32/2958_2.png) [@iotapi322](https://community.ipfire.org/u/iotapi322)\
**Post date:** [21 December 2019 20:01 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/12 "2019-12-21T20:01:54Z")

</div>

What instructions did you use to compile pmacct for ipfire?

---

<div class="post-metadata">

**Author:** ![ummeegge](https://community.ipfire.org/user_avatar/community.ipfire.org/ummeegge/32/16602_2.png) [@ummeegge](https://community.ipfire.org/u/ummeegge)\
**Post date:** [21 December 2019 21:10 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/13 "2019-12-21T21:10:22Z")

</div>

Hi all,

> [@iotapi322](#):
>
> What instructions did you use to compile pmacct for ipfire?

we used here different versions with Sqlite only but also with libmaxminddb and rabbitmq → [https://people.ipfire.org/~ummeegge/pmacct/](https://people.ipfire.org/~ummeegge/pmacct/) where some versions needed additional packages. In the old forum beneath a lot more tests, versions with support for MySQL and PostgreSQL has also been build.  
Some potential build instructions can be found e.g. in here → [git.ipfire.org Git - people/ummeegge/ipfire-2.x.git/blob - lfs/pmacct](https://git.ipfire.org/?p=people/ummeegge/ipfire-2.x.git;a=blob;f=lfs/pmacct;h=01a120f4ead2f86e80c50380772f81a1705d9321;hb=7756aac6bec7fb8944aab1cf1d83ff32e9fd90d5) which is not the current status (–enable-zmq was also enabled) but i think it should work without other dependencies.

In general, this is not the currently available test version since for example the PAKS scripts are meanwhile adapted for Pakfire this process is not well checked at the moment (have currently not that much time for this project).

Please feel free to test extend and share new knowledge again → [git.ipfire.org Git - people/ummeegge/ipfire-2.x.git/commit](https://git.ipfire.org/?p=people/ummeegge/ipfire-2.x.git;a=commit;h=7756aac6bec7fb8944aab1cf1d83ff32e9fd90d5) .

Best,

Erik

EDIT:

- If you want to use Pmacctd with Sqlite you need to adapt the configuration since the pushed one is held as simple as possible (potential Addon version) .  
OT
- May you know it already but if you administer Cisco machines there seems to be a problem with the certificates  
" Self-signed X.509 PKI certificates (SSC) that were generated on devices that run affected Cisco IOS® or Cisco IOS XE software releases expire on 2020-01-01 00:00:00 UTC. **New self-signed certificates cannot be created on affected devices after 2020-01-01 00:00:00 UTC**. Any service that relies on these self-signed certificates to establish or terminate a secure connection might not work after the certificate expires."  
just as a beneath info 🙂

---

<div class="post-metadata">

**Author:** ![iotapi322](https://community.ipfire.org/user_avatar/community.ipfire.org/iotapi322/32/2958_2.png) [@iotapi322](https://community.ipfire.org/u/iotapi322)\
**Post date:** [21 December 2019 21:38 UTC](https://community.ipfire.org/t/data-usage-monitoring/707/14 "2019-12-21T21:38:38Z")

</div>

> May you know it already but if you administer Cisco machines there seems to be a problem with the certificates  
> " Self-signed X.509 PKI certificates (SSC) that were generated on devices that run affected Cisco IOS® or Cisco IOS XE software releases expire on 2020-01-01 00:00:00 UTC. **New self-signed certificates cannot be created on affected devices after 2020-01-01 00:00:00 UTC**. Any service that relies on these self-signed certificates to establish or terminate a secure connection might not work after the certificate expires."  
> just as a beneath info 🙂

idiots in another division… I’m in the collaboration group. We do the webex Teams. Specifically Logging Metrics and analytics.
