Recommended way to install filebeat

Ah I think I understand now. I’m submitting the build process to a pipeline that’s going to compile for all architectures. Yep I see how that works, yea should be no issue then as I believe the build process is the same across architectures.

Good luck in your build and submit process and welcome to the add-on development team.

1 Like

That is the basic process of cross-compiling.
And concerning go, this is a compiler for the go language. Compilers are contained in the build package, not the production package.

Yes this is where I went wrong, I thought by submitting the build process as the add-on, I was essentially submitting instructions for the ipfire device to build the pakfire at add time, which of course it cannot do withou the go compiler. But now I can see that it is instead cross-compiled when received by me and then distributed.

There is another problem with compiling on the fly. IPFire modules are signed. This signing is done at the system build process. If I remember right the signing key is thrown away after the build. Therefore you cannot add an additional module, you don’t know the key of the system.

Ah yea that makes sense.

I’m sure there is some kind of gpg based pki system in place, likely the certificate containing pubKey is in the production ipfire image, and wherever is doing the compiling and packaging is signing with the private key corresponding to that cert. Same thing as for iOS and iPhone apps.

For kernel module signing see Michaels blog article .

pakfires ain’t kernel modules tho, 100% userland I’d be reckoning.

Seems like devs would prefer to upgrade the syslog used in ipfire instead of using filebeat, which I’m happy if it does TLS encryption and authentication to logstash. So holding off making pakfire now as devs have signalled their intent to instead upgrade syslog see here Shipping logs to logstash - #14 by ms

Now he says they have no roadmap to do it… I do not understand this guy, he comes and says there is a better way to do it but then says probably they wont do it… well which is it? Shipping logs to logstash - #17 by ms

So it is likely I will not bother with a pakfire, I wanted to contribute to the project but seems not possible, I get that there is a better way to do it, but if you’re not going to do it then how is it better? My way is better than no way at all (or completely insecure as it is currently).

What to do? I don’t care I have it working for me, I just wanted to give back to the community with a pakfire for others who want to ship logs to a remote SOC, currently not possible with ipfire unless you want to send logs plain text which I imagine no one would want to do unless it was in their own network, or over a VPN as was suggested by someone else. I feel maintaining a VPN connection just for shipping logs is very overkill.

I would hate to see you not contribute this Plugin.
Ipfire is truly a very small team of developers.
It would be sad to see someone like your self leave this community.
Please do not feel that your contribution is not wanted.
I Do not think this is the case.

If you can find out for sure that if my pakfire was passing the criteria it would be added then I would. But the conversation to me seemed more like, it is to be avoided, and we want to do something else.

The conversation does not sound like your addon ( pakfire is the package management software ) isn’t wanted. Michael stated clearly, that additional addons should be maintained by someone ( best the initiator of the addon ).
If you are willing to go through the development process of an addon ( which isn’t too big ) and to maintain this addon ( for a while ), you are welcome to do this.

Being there other ways to achieve your goal, sending logs encrypted, you cannot expect some of the handfull core devs does the job for you. Some ways to send the logs are:

  • send with rsyslogd to a trusted client in the LAN, from this client you can send with your favourite tool to the remote.
  • send per VPN, which is encrypted by default.
2 Likes

At no point did I expect that, I have filebeat working for me as stated, I listed my working example and suggested I make a filebeat pakfire for others who want to ship to logstash which I believe is a pretty common thing nowadays, just maybe not from ipfire, I might be a trend setter lol.

If it is like you have stated, why not make an addon and present it in the development mailing list?
That should be easy. :wink:
We like trend setters, integrating ‘modern’ tools into IPFire project.

Yes that’s what I was proposing but I was getting the impression that it would be rejected for consuming more resources than an alternative way, but actually it has been said to me now that I am able to submit a pakfire so I most likely will do that. But also I am exploring by trying to compile rsyslog on the pi and maybe use it instead of filebeat, as I do agree with ms that it would be leaner. Maybe rsyslog could be the pakfire instead of filbeat.

Leave it with me I’ll come up with something.

Bit annoying that we can’t just pull the compilers and stuff onto an ipfire device i.e. my home device. I need to make a seperate dev environment, my macbook isn’t really up for the task so it’ll have to wait till I get my new PC then I can make a VM for it.

Two points to consider:

  • compilers and other dev tools should have no place on an internet appliance. This would open doors, you don’t want to open.
  • an internet appliance is dedicated hardware, which not necessarily have devices needed for development ( ‘fast’ processor(s), fast and big hard disks, large amount of memory )
1 Like

FWIW, If Filebeat were available as an addon, I’d use it.

So what is the final verdict here on how to get logs to an ELK stack? I have tried the “remote” setting and nada. the whole point is to start agregating traffic for an ML training to complete the cycle.