One thing to point out in my post directly above. It took about 12 days after my CU 175 update for the WARN messages to appear. I rebooted yesterday and only one WARN message has appeared so far. So this seems to be an issue that takes a few days to make itself known. Doesn’t make it easy to debug! But when it does happen it seems to explode and I get 100’s of messages.
EDIT: WARN in messages log:
-rw-rw-r-- 1 root syslogd 977084243 Jun 29 14:56 /var/log/messages
Failed during write = 66346
Missing data detected = 12222
Increase values or look . . . = 12222
When you were building pmacct for ipfire, did you ever do a build with ZeroMQ?
I ask because a some of the threads with similar errors as mine (above) mention ZeroMQ. But there is no “SOLVED” type response so I don’t know if it works or not.
Might be time to open another Issue report with pmacct GitHub.
the building time is long ago but at that time i used also Ntopng which needed ZeroMQ and also nDPI so i used both for Pmacct since they where at that time simply available in my building environment but i left them behind since both packages are not part of IPFire systems.
This might be the best, even if ZeroMQ would solve this problem, which i do not think, this would be only a shifting of a not solved problem.
Will have an eye on it if this problem comes by time also here around.
As indicated in the quickstart guide, i’d totally discourage you to use the native version of internal queueing (what you are using right now) in favor of ZMQ. Please see the X. Internal buffering and queueing section of the QUICKSTART document at this propo: pmacct/QUICKSTART at master · pmacct/pmacct · GitHub
so ZeroMQ should be the way to go ? As far as i can see in IPFire building logs only ‘frr-8.0.1’ checks via configure openly for libzmq but beneath also Pmacct may there is more ? Currently not sure about this. What want you to do ?
Yes as before written ZeroMQ (like nDPI) was and is not part of IPFire. Was digging a little through the build logs to get an idea which software can participate also from a asynchronous messaging library and frr was the only one which showed up but this does not mean that others does not have a compile time option for libzmq. This might be the first step to decide →
if it might be a good idea to integrate a new library into IPFire (merge request) to find good arguments.
On which place should ZeroMQ be placed in make.sh .
Should it be part of the core system or delivered as a package (Pakfire).
If checked which already running software can participate from ZeroMQ, adapt the appropriate LFS file.
This should then be the first steps before asking in the developer mailinglist in my opinion.
First thing that I found when looking at the zmq github page is that the last official release was in Jan 2017. However large numbers of commits have been done in the meantime with many raised issues being fixed.
I have found this with at least one other Open Source project that IPFire uses for an addon (lcdproc) where the last official tarball was in 2017 and you have to decide to do an extraction from github at a certain commit to use the most up to date option. I had to do that with lcdproc as a user had experienced a bug which had been fixed but only as a commit.
This is really not an ideal situation as you have to choose the commit carefully to make sure you have a self consistent set of changes.
I believe that if it is needed for pmacct then it should be installed with that package not as a core program.
As a dependency of pmacct place it just before pmacct in make.sh would be my choice.
Why would we want to do that. Is there a problem in the communication between different packages currently. If we do that are we sure that it doesn’t bring any security vulnerabilities or weaknesses.
Anyway, my view is that it should be used with pmacct if that needs to use it and nothing else currently.
If there is some generic structural benefit in using zmq for IPFire, then that should be considered for IPFire3.x
The last one from Github is from 2021 → Release libzmq 4.3.4 · zeromq/libzmq · GitHub but in principal you are right two years have a lot of stuff in it since the last commit is two weeks ago and the project looks quite busy.
For the rest do it like you want, am anyways good with it also without ZeroMQ since i do not have the problems which Jon have. According to your security concerns, i think ZeroMQ uses a good way in cryptography and security → Tinkering with ZeroMQ security - Y Soft Engineering Blog .
The principle of ZeroMQ should not be a problem solver with communication between different packages i think it should be a booster for them but as before, a good more fundamental understanding of what ZeroMQ is in general should be made IMHO.