Hello,
Does anyone have any experience with shipping ipfire logs to logstash?
Hi,
we did that longer time ago with Filebeat --> https://github.com/elastic/beats/tree/master/filebeat#filebeat at that time with OSSEC --> https://forum.ipfire.org/viewtopic.php?f=4&t=4924&hilit=logstash&start=45#p110744 and a fast one with Softflowd --> https://forum.ipfire.org/viewtopic.php?f=50&t=19263&p=109986&hilit=logstash#p109986 .
At that time Go was not available in the IPFire dev environment which has been changed so there should also be a possibility to build it for IPFire but i haven´t checked that.
Best,
Erik
@ummeegge Thanks Erik,
Yes filebeat would be perfect, what would be the best way to get filebeat onto ipfire as I believe there is a restriction to import third party binaries? (no apt and can’t seem to unpackage a deb file) Is building from source the only way?
Hi Ian,
you are welcome. For testing purposes at that time we used the linux-64bit binary --> https://www.elastic.co/de/downloads/beats/filebeat but let me say as ever, do not use pre-compiled binaries from 3rd party 
. Another one might be to send syslog messages remotely to logstash --> https://support.halon.io/hc/en-us/articles/360000700065-Remote-syslog-to-Logstash <-- there is Rsyslog in usage which IPFire do not provides but it is may also possible with sysklogd --> https://wiki.ipfire.org/configuration/logs/logsettings but am not sure about this.
1 Like
Can you believe what I am doing instead. I noticed ipfire has remote syslog sending baked in, so I am just dumping suricata logs into syslog and shipping them to rsyslog remotely (which then pipes it locally to logstash).
No need for me to install filebeat this way.
But a concern I have is currently these are sent unencrypted and unauthenticated. I would like to enable TLS to send the logs but it looks like ipfire does not have the options at least from the GUI. Do you know of a way I can enable the syslog sending over TLS?
sysklogd do not provides TLS nor TCP for sending logs to a remote machine. I would use a VPN for this.
Best,
Erik
Hmm that is unfortunate.
I think I might return to the original plan of using filebeat in that case. I think is cleaner than using VPN and I already have the PKI in place with certs and logstash authing certs etc.
Hey mate, I was unable to get Go working on ipfire I think it is not included and I don’t see it as a pakfire.
What I did find tho is someone has created a docker image that compiles the filebeat repo for raspberry pi. So in an external device I am building filebeat 7.11 using the output from that docker image, and I stick it in a github repo that I pull to ipfire and I run the compiled filebeat on ipfire.
It is working, but yea had no luck getting golang working on ipfire so gave up on the thought to compile on device.
And I am using certificates to encrypt and authenticate to logstash so this is so much better than the remote syslog option 
More info on this thread
Does logstash support remote syslog?
Hi @ms
Yes it does, and in fact I initially was using the remote syslog capability in ipfire to send to logstash. I configured suricata to dump into syslog and shipped it all to logstash using the inbuilt remote syslog capability.
The problem with this however is apparently the sysklogd being used does not support TLS, which means the logs are being sent in plain text, and also there is no authentication being done at logstash you have to leave it wide open so any john doe can start flooding you with fake logs unless you restrict by IP etc.
By using filebeat we can leverage a TLS connection to logstash which both encrypts the logs, and provides auth as only the cert I give to filebeat is allowed to send data to my logstash.
Yes that is true. We are on a kind of more traditional version of sysklogd which never was a problem because most people do not need any special logging requirements.
There are some developers working on replacing it with an alternative. Would you be able to contribute to/sponsor that?
Do you mean this? Recommended way to install filebeat - #6 by knightian
I’ve put my hand up to sit down and setup a build env and submit a pakfire for filebeat now that I have it working in my ipfire.
Although I will say, the forcing of default port 514 doing the rsyslog -> logstash and no way to change that port made it terribly bothersome, because it means making logstash and therefore java (not just the normal java as I found out, specifically the java bundled in with logstash) being able to run on a privileged port, I managed to get that sorted but it was rough. Of course we can NAT that problem away if we’re behind a NAT but that was a hurdle worth mentioning.
Yes, the current solution isn’t very flexible, but I would want to avoid to run any other software on the firewall. Increasing the software stack - especially as it needs Go or Java - is probably not very flexible and will increase memory consumption massively.
The technically easiest choice seems to be upgrading the syslog daemon so that it supports TLS and make that port configurable. Or did I miss anything?
The technically easiest choice seems to be upgrading the syslog daemon so that it supports TLS and make that port configurable. Or did I miss anything?
Yea if it’s an option, as long as the replacement syslog not only uses TLS for encryption by server supplied certificate, but also allows for the syslog client on ipfire to present a cert that gets validated by logstash in the TLS handshake (this may even be part of the TLS spec I don’t know) then really that, and port customisation should be all as you say. The formatting is done on logstash side, it’ll maleate it into JSON using specified grok patterns for elastic ingestion.
If you guys are planning to do that then I will just use my filebeat until that solution is ready and I won’t bother with the filebeat pakfire submission if it can and will be done with a new drop-in for syslog.
Happy to setup a test endpoint on my logstash and give certs for testing if wanted.
No it isn’t on the roadmap yet. We thought about it but there was no technical need to change.
So what should I do? Should I submit a pakfire for filebeat or not?
Hello @ms can I please get an answer because I do not want to go to the effort to submit a pakfire only for you to reject it.
Hello Ian,
as mentioned before, I understood that the easiest version would simply be to configure a syslog server using TLS to connect to AWS’ infrastructure somewhere. That would save you a lot of work in compiling and maintaining any additional software.
Did I get this right?
If you want to submit a patch that integrates additional software, you are of course welcome to do so. But we would require you to maintain it and regularly update it as well as deal with any bug reports that might be coming in for it. We cannot have unmaintained software that never gets updated on the firewall.
If you are the only user of this, this is a lot of work for very little benefit and therefore I would prefer the option above.
2 Likes