I would like to contribute some of my time & skill at building images for arm devices.
I have been reading about how IPfire is build and installed.
I found out how it is build and also nightly build rootfs image which I can re-use over the device specific boot partition. I am not sure if the kernel is present or not. I will try it on VM and also flash it on a drive to see what all is present in the image.
We can have detailed discussion on this thread. Hoping to get help from the developers as well as the community.
Thanks.
Update: VM attempt didn’t work out as I only have 2 lan while I might need 3 to make us of this over a network with the host pc having one connection.
At the moment we have decided not to spend much time into the arm support because there are less than 2 percent arm machines active and there are no affordable sbc with more than one real gigabit nic. (VLAN switched and USB are not counted).
IPFire need a kernel with many network and hardening settings and rely on some patches to get full support. (e.g. Layer7 filtering) so it not suggested to use the kernels from the sbc manufacturers.
We don’t plan adding device specific kernels again. (Only patches and dtb’s are possible)
The build system should work on every armv7 or aarch64 sbc that have enough cpu power, disc space and ram (1GB works but only with enough swap). I use a Odroid-C2 with armbian aarch64 as buildsystem but we also use debian aarch64 VM’s at amazon for the nightly builds. If the kernel on the aarch64 system has enabled COMPAT mode it can also build the armv7 image. (with downloaded toolchain)
There is no more percentage of arm because until now, there was no hardware that did not need an rj45 adapter via USB and now that there is one (nano R2s) IPFire does not have or is not compatible due to lack of uboot.
The nano R2s seems to me for the price it has and for certain scenarios it is a killer machine and I would install a lot of them.
I have in a Client a 32bit R1s (with 512mb of ram) with FW, Geoip, IDS rules which is working really well.
It is a pity that it is not implemented for her. I wish there was a Crowdfunding to implement it.
I am sorry Roberto, but this really isn’t the hardware that is suitable to run IPFire in a professional environment.
To keep it short, I would like to say that I have commented on this matter before and I have even written a few lengthy blog articles:
To sum them up, it simply isn’t worth it to port IPFire to ARM. At all.
We all had big dreams when ARM as an alternative finally took off. That was a little bit before the first Raspberry Pi made this an absolute mess of a market. There were good designs out there (look for Globalscale’s Dreamplug for example), but of course they come at a certain price point. That was around $200 at that time, which isn’t bad at all for a firewall.
Now, everybody is looking for a cheap ARM board with performance and loads of features. The Raspberry Foundation is a charity that pays probably no tax at all, but somehow is selling lots and lots of boards at an absolutely “amazing” price.
Amazing because nobody else in Europe can compete with them. Paying no taxes helps. The second step is that they have almost completely outsourced their software development. They call it Open Source-ed, but that is not the same.
Over many years, there has never been a release of that piece of hardware that was supported by a mainline kernel. Neither Linux nor any other of the *BSDs. They simply do not care what software runs on it.
That might be fine, but in the end we need a product that works. For that it needs hardware and software to work together.
There is a ton of patches that I do not know where they are coming from; and a number of projects who simply try to make things work. Most famously Raspbian - a version of Debian for the Raspberry Pi. The code quality of those modifications is so bad that there is never ever any chance to upstream these things into the Linux kernel and Debian. You create a side-project for just a single hardware platform.
Then there is there extra-ordinarily short release cycles of the hardware. As soon as we got IPFire stable on a version of the Raspberry Pi, the next one was already out. Of course it was incompatible enough that only little of the work could be taken with us. Because of the cheap price-point, users bought the new models and the end result simply was a lot of frustration: Users end up with a brick and developers have to put in days after days to get those boards to work.
The market is so fragmented that there are special needs for pretty much every board. Literally megabytes of patches for some of them. Code that is bad, badly reviewed and usually not maintained, but rather patched and patched and patched until it finally works.
We have now ended up with an IPFire kernel that runs on some boards. Arne has been fighting many many nights with those and he is doing a really good job. But there is unfortunately virtually no users.
As of today, fireinfo is reporting 1.71% of ARM systems. You can do the maths now how many users there are for each of the boards that we support. You will end up with really low numbers. I am sure that there must be some that are only running in Arne’s test lab, because users have moved on to something else.
I was really excited about ARM when we started. There is a point for it and there are servers that play in a totally different field. They are worth our time, the single board computers are not. They are unfortunately toys.
We are doing the homework of the vendors. A race that is expensive and almost impossible to win. The goal is that we support hardware that simply isn’t suitable for IPFire anyways. There are no network controllers on it. Not in numbers and certainly not in quality. They are connected via USB and add lots and lots of latency. Not even the cheapest plastic ISP router has network interfaces as bad as that.
We have tried to seek conversations with the vendors. For asking them to change their hardware design to suit our needs better. And to do their role by adding required drivers to the Linux kernel. No response.
If you are looking for cheap hardware, then do not forget that someone needs to write the software to run it. That cost has fallen on other projects that do not make the money.
I am absolutely disillusioned about what the ARM market has become. I regret that we spent time on it at all. It was a waste of time. A fun time, but unfortunately wasted because in the end there is not a single piece of hardware that is cheap and runs like a charm.
A couple of times we were close. But when you have one player who is evading tax by being a “charity” against a vast number of players from Asia, how are you going to compete?
Everyone working from the “western world” is going to be too expensive to fly the “ARM is cheap” flag. It isn’t. It just works by not doing half of the job.
To bring my whinging to a conclusion: I have no interest in adding support for more hardware of the category outlined above. I also do not care about the “good” hardware that is outpriced by x86 designs and therefore nobody will buy.
That leaves me with saying that ARM is dead for me. We have support for what we have now and that won’t change. If we have a free moment, we will maybe play with things that we could get our hands on.
Nobody will have a good firewall with any of this. It hurts me to say it, but there is still no alternative to x86 (yet?).
In Spain, 56.36% of companies have 0 employees, that is, self-employed. These companies (not all, but most) cannot afford to contract Professional Services to implement a Firewall in conditions. For them, it is an expense, not an investment. It’s a shame they think like that, but they don’t have the resources to implement Security.
39.25% are companies with less than 10 workers. These, possibly if they can afford a cheap Firewall, but there is everything, many of them also think that it is an expense. When a Cryptolocker or Ramsonware enters them, they radically change their minds and become sensitized, but it is difficult since there are commercial products (SonicWall, Fortinet, etc …) that are what their technology partners prescribe, but they are so expensive that few they implement it.
I said of the Nano R2s for that 56.36% that they only need a Firewall, VPN Access, Geoip and, at most, an IDS. I think this gadget may (with its shortcomings) be an appropriate solution for this fork of companies.
Roberto,
I can understand your statements. But even an IPS needs much effort in maintaining. Thus, if one can’t afford an adequate hardware equipment, how could be done the maintenance?
There are many “cheap” SBCs based on x86, which are supported by the Linux community and thus IPFire.
Thanks for your opinion, is interesant. Nano R2s cost 22 Euros + Shipping. What other devices do you mean that have 2 Gb ethernet, 1Gb ram, Wifi and quadcore with 1200 Mhz for this price?.
The data of the Nano R1s 32bits that I have installed are the following (imagine with twice the ram):
Well, but then you cannot have those things. It either is worth it that you have a network that you can rely on and that secures your own and your customer’s data or it isn’t. You have that choice. But my point is that the project and the developers are now swallowing that cost of making IPFire run on cheap hardware.
IPFire costs zero. We cannot make it any cheaper than that. Commercial products cost you thousands a year and it normally adds lock-in support contracts.
Compared to that, spending 300€ on some firewall hardware is cheap and will give you something that will work for a long time.
If your business cannot afford that it either isn’t a business or doesn’t know what the danger is.
Providing rootfs and uboot can be one of the reason for this and ipfire is not promoted much I guess. If I understand correctly a normal user with little bit knowledge of IT might just need a flashable image, So maybe we should provide it instead of providing 2 files to be flashed manually. BTW dd is risky for those who do not understand what is dd.
Reason is you support boards which are obsolete and not available for sales anymore and they are low end SOC which were only built for experimenting with the idea of open source hardware and the response it can get.
I have replaced my servers with arm sbcs, Currently running and tiny NAS with , Jellyfin for Media playback and Nextcloud server for filesharing along with samba all this running on a tiny NanoPi Neo Plus2, BTW I am typing this post from an ARM Based laptop, Pine64 PineBook Pro, I can compare it with my I5 Lenovo laptop.
Maybe you have not tried the right boards hahahaha.
Please use pine64 webserver, chat server and blog, Let me know how you feel about it, Maybe that way you will understand that the whole setup is running on around 45 RK3399 based Pine64 RockPro64.
Yes, I agree but those are the boards of the past now the vendors are building boards with router and firewall appliance in mind, Example: NanoPiR1s (Second Lan over USB) causing latency, BananaPi R2(armv7), BananaPi R64 (Aarch64), NanoPi R2s (2xGB Lans), Radxa RockPiE (1xGB Lan and 1x100mb - not over usb controller).
Yes before they were sleeping but now they see the potential, in the world of ARM Soc, vendors are not responsible for the driver as per the open sourced hardware scenario and there are a lot of kernel devs contributing to the mainline kernel for such boards.
The current situation is that the Chip Manufacturers are giving incentives to part time kernel devs as well as companies to contribute drivers for their boards to mainline kernel, Example Amlogic with BayLibre while Allwinner have their own contacts with devs and Rockchip tries to contribute themselves. So the current situation is completely different then it was before.
Agreed and there are many devs willing to help but it takes time and in open source scenario there is not timeline or deadlines.
I cannot deny this as I felt the same initially when I started playing around with arm SBC’s. Now things have changed and I think arm is going to take over in the near future for many use cases.
IDK what you trying to say by this but ARM is still cheap in terms hardware cost as well as power consumption.
I do not think that asking you to support it is a right thing to do, as I understand the time it takes to get a single board to work at the first place, I can only request for your support when needed as long as the aarch64 packages are maintained in the repo.
Again I can never accept this statement, I have met people at fosdem who are using arm based devices to run a complete firewall appliance but they are senior devs and have in depth knowledge of linux and networking so they don’t really need any UI they can handle everything from the cli.
You’re not confused you’re right with what you’re trying to explain.
ARM SBC can make a good low cost, low maintenance firewall appliance but we should not expect it to do complex routing.
BTW little bit about myself, I am a software consultant for an IT company which provides open source business software, I strongly believe in OSS and there is nothing better than OS hardware to support the concept of OSS and the privacy benefits. I am a maintainer at Manjaro Linux ARM and maintaining support for more than 6 devices which are from different Vendors and different SOC manufacturers. I understand how hard it to to maintain an ARM device support but it is not too hard once a proper tools in build to add support to devices. BTW I don’t expect arm firewall appliance to replace x86 appliance anytime soon, but it can still be used at home and SOHO level.
To add support to any arm device all you need is a working Uboot which most of the vendors provide these days and its drivers in the kernel which we have to find the right kernel dev who is supporting that soc. IDK what config is needed in addition to the mainline kernel to work. With mainline I mean pure/vanilla Mainline with patches.
Let us just discuss what is needed for to add support instead of arguing on whether we should support it or not.
BTW We can run Opnsense and PFSense on ARM SBC and that too on armv7 not to forget aarch64 is much more powerful then armv7 soc’s. If it is a matter of not having enough users then it is a complete different case.
So to conclude, All I want to say is I am ready to contribute to this project with my time and knowledge of arm devices. I can get a working uboot for the devices I have, I can get working Mainline kernel for it. I will go through the build process again and see what else is needed for IPFire to work.
I only expect a little support from ipfire team that too only whenever I am stuck, rest I think I can manage.
No hard feelings please, I am just sharing my opinion.
Thank you for making ipfire such a stable firewall OS.
Much Appreciated.
Cheers.
In Spain (I do not know in other countries) the self-employed (removing exceptions that earn a lot of money), cannot allocate 300/400 euros a year in a Computer Security Service (which is the service I provide). but 80/100 euros a year, possibly yes.
If it is a Company with employees, possibly yes, since 300/400 is not much and they can afford it.
That is what I mean, a hardware that can serve a self-employed person to give it some security and to be able to access their equipment from outside to consult existing data on their NAS or computer when they are away from their office / home.
So it was the Crowdfounding thing. To allocate money to be able to implement it in this mini-hardware.
I am in this range of self-employed and cannot contribute much (I have no idea of programming), but I can supply a couple of Nano R2s machines and a little money if necessary.
You can not support every single low cost arm board without any power and ram, but I would like to see a raspberry pi 4 support, because with 4-8 GB of ram you may use ipfire a little better.
I don’t see anyone asking for support for all the arm boards it is nearly impossible for even a commercial company so forget about an open source project trying to achieve this.
I agree that we should only concentrate on known and powerful devices. BTW RPI is not the only device having 4gb RAM, IDK why everyone is only interested in RPI, I have only 1 rpi which is a 3b+ and I didn’t like it when I compared it with other sbc’s. Example Khadas Vim1, Vim2, Vim3, Edge-V, Pine64 RockPro64, the last three do some in 4gb variant.
RPI4 support is not too hard as they have 4.19 and 5.4 kernel support already only thing is it won’t make and ideal firewall appliance unless you use a very high quality usb to lan adapter.
I don’t think we should support sbc without dual lan hardware setup as it will be a headache to answer users who will complain about some usb2lan convertor now working.
It is better to add support to devices with minimum 2 lan.
I do not think that is makes any sense to provide custom images for SBCs. There are too many versions and too few users. We would waste gigabytes of disk space on the servers and mirrors for each release and nobody will download them.
I have to assume that people who want these things to work can use dd. This is a firewall and should not be used by non-experts.
They were not obsolete back when we added support, and we still have users who use them. Discontinuing support at that pace makes no sense for long-lived systems like firewalls.
And what makes a board right?
I am looking for something that is suitable for my purposes which would be: long-term availability by the vendor (around 5 years from initial release), four gigabit network controllers, a decent amount of memory (2-4 GB) and a beefy processor that can run the IPS at gigabit speed.
Those exist, but they all run an x86 processor or come at an unaffordable price.
I really do not want to examine every single board out there. There are too many and they suffer from the same problems anyways. Almost all of those you mentioned have only one network interface.
A firewall receives packets on one interface and sends them out on another. What is the point that one interface can do 1 GBit/s when the other can’t?
Are they not? Who is then? You cannot just dump some hardware onto the market and expect that someone else will deal with it then. You need to provide some basics if you want me to use your stuff.
There might be some. The board most sold is the Raspberry Pi and the foundation has not done much to upstream any changes into the Linux kernel.
The boards might be, because they come from China. Nobody is paying the required WEEE fees or gives you any warranty. That is comparing apples with pears.
If you add the overhead of getting software to work, they are not cheaper. Only very very few users would have to share that cost and that makes it not worthwhile.
If you are happy to do that work for those users I am happy to hear that of course.
And power consumption is no longer relevant. It makes no difference on your electricity bill if the device is consuming 4 or 6 watts.
Those will be maintained and I am happy to help where I can.
Just be sure that if support for certain hardware is being added it has to be maintained for many years too and cannot be simply dropped because the next thing is out.
Oh it does boot on it. IPFire boots on a lot of hardware. But how does an ARM board cope with 1 GBit/s of bandwidth? That is what we are looking at. Most SBCs do only a fraction of that.
You are looking for the little toy projects at home. I am looking for something that a business can rely on. IPFire has a very vast range of users and different requirements. The same software that is running on the SBCs is running on a 19" rack servers pushing tens of gigabits a second.
Yes, IPFire runs on those too. We have a working 32 bit and a working aarch64 port. The latter isn’t linked on the download page for various reasons.
Do not take my arguments as a no. I am not running the show alone here. I am more than happy to welcome you to the development team and looking forward to see you helping to make IPFire better.
I am just looking that everyone is investing their time into the right things that most users benefit from. If you believe in a brighter future for ARM then I am looking forward to seeing that happen…
I want to add some extra comment about ARM architecture.
Disclaimer: a am no developer nor system designer nor cpu/silicon specialist.
One of the most interesting things about ARM is if you pay enough to ARM Holdings and the foundry, you can design whatever you want as SoC, adding few or more modules to suit your code.
Yep… “your code”.
Most of the work of IPcop (before) and IPFire (now) rely on the kernel and packages of Linux, therefore most of the code (and related optimizations) come from x64 platforms (once again: 32Bit is bad at cube or more).
This make develoment less valuable? Not at all, but as the job is quite “not payed” through subscription of the distro, cannot be that broad for all architectures.
Moreover: recently a computer company with a fruit as logo decided to extend it’s ARM CPU environment also on Laptops and Computers, because they do have proprietary code which is under their complete control therefore they can design an SoC which can boost all the bells and whistles about their needs. Oh… if forgot: the fruit company do have all the patents to design anything needed on ARM architecture without paying a penny of royalties… Seems cheaper than go buying Santa-Clara-Branded silicon…
Currently IpFire project cannot do that. Who want to believe to the “most beautiful ARM board in the world for IpFire” should do one thing only: found the project with donations. And call some friends for that.
x64 CPU/APU boards are not cheap. What they cost for hardware is money saved on power bill. Because they run about 25-40W of power instead of 120-180 of a desktop computer with an efficient PSU, SATA SSDs, a bunch of ram (4gb at least) and from 4 to 8 cores.
Wanna run on ITX? Be my guest: less variety of hardware, smaller spaces, higher temperatures, less expansion, not so cheaper.
IMVHO is better to head on Single Boards Computers, with embedded x64 APU (i would like to see soon the Zen2-Based embedded APUs…)
@roberto free of charge, feature rich, power efficient: pick two of them.
You can buy an appliance from any security product reseller, it will be feature rich and power efficient, but wont’ be free of charge.
You can choose to build your “refurbished” business grade desktop with some nice network cards, a small SSD: won’t be power efficient, but it will be feature rich and free of charge.
You can choose to use a x64 Computer Board for more power efficiency, but it will cost two-to three times the price of your business grade pc.
The “bad” thing of wasting power on a Firewall made by a computer is that into warm places like brasil, spain (most), italy (most) you can’t run the cables into the coldest room of your house/facility to… gain thermal confort from heat dissipated (from firewall, server, Network Switch and also UPS…)
I cannot deny this but we can always try to give the best for a normal user to try that way it increases the users which in turn increases the burden for support.
Fair enough, I understand.
Lets look at BananaPi R64, It is powerful, have dedicated network chip, I think 5x 1GB lan and onboard wifi. Kernel support is there but still WIP.
Currently I have R2 which is only 1GB Ram though.
We cannot try all the devices ofcourse but we can always look for good ones. Normally projects can request for sample boards. If it does’t meet the expectation then there is not need to work on it.
That is the reason I don’t like RPI lol, Rockhip and allwinner is doing alot to push upstream changes.
That is what I have been doing at Manjaro Linux ARM project and yes I will be happy to look into it, Currently building ipfire on VIm3 while posting this from VIm3 itself.
Yes ofcourse and there is no harm in dropping the support, if there is not many users for it. For me I support many devices and they do get updated regularly.
Have you seen the cluster concept, Maybe you should see if that can be applied with ipfire, just saying though.
HAHA, Don’t worr, coz even if you say no, I will always do my part of trying to build it as per the guides and see where it leads me. That’s the beauty of open source projects
No more arguments, back to work.
Currently downloading Src and it is failing, I suspect my internet or some certificate error.
Will get back here with the build progress.
Sample Screenshot to show you that ARM SBC can handle alot of load.
first of all, I did not expect the ARM situation to cause such a discussion, but am delighted to see it does. (The community is too quiet anyway.)
Second, just two minor points:
Regarding @roberto’s arguments: It is interesting, although unfortunately quite foreseeable that socioeconomic factors matter when it comes to a firewall. This is bad, as those countries tend to have a higher amount of compromised PCs per capita, and in my humble opinion, we should not ignore them as they might be somewhat underrepresented in Fireinfo.
Compared to other distros, however, we are doing a good job as we have rather basic hardware requirements, which can be satisfied by older or cheaper hardware - at least for the basic functionalities of IPFire.
Similar to Michael and other (core|community) developers, I am fed up with Intel. AMD might be somewhat less worse in terms of CPU insecurity, but in the end, I am looking forward to ARM64 and RISC-V.
Perhaps we could focus on the first one, if beyond-low-end ARM SoCs require too much time, but I think we agree dropping ARM altogether is not a good idea at the moment. If we are looking for trustworthy hardware, x86 certainly is not the ne plus ultra.
I think the main problem is further, that the x86 world has its “standardized” board support package called xyz-BIOS. Other architectures require ( often ) a new build of such a BSP, based on uboot.
Manufacturers of those boards do not ever publish the information for this building process. Especially, if it is a proprietary SoC.That makes it not as easy as with the “standard” boards.
And yes, the Intel architecture is the quasi-standard in processors, despite the fact that others may be better in design.
To get most support from the community of an open-source OS, we should concentrate on the processor main stream.
I do not see a developer community, big enough to do those special BSP builds.
Thank you very much for the clear response. I totally agree with you and IDK what makes people think arm soc takes alot of time. It does until we get to know all the requirements needed for it.
You just need someone who understands arm arch and its bootloader.
Most SBC try to push their board support to mainline uboot or maintain patches for uboot, which was not the case before.
Yes but if we shortlist the boards then this won’t be a problem, Like Rockchip and allwinner have good mainline uboot and kernel support already, If we look at RK3328 and RK3399 which already have good support so I don’t see any reason for it to take alot of time.
Let me cut it short, I have a couple of device which I think can be a good router boards, I am already trying to get working Freebsd and Opnsense for it and I would like to try ipfire also.
Boards:
Radxa RockPiE - Aarch64
BananaPi R2 - Armv7
NanoPi R2s (Waiting for its delivery) - Aarch64
Currently I am following the build steps as per the ipfire wiki. There is no details after the build command.
Where does it build it and what file do we get and how do I use it ? ATM it is still downloadingsrc as my net sucks tonight.
Lets not turn this thread into ARM Vs X86 Vs Risc-V please.
UPDATE:
downloadsrc is fetching linux-4.14 and I think this will not be helpful in building a device specific image as every board will need different configs.
is there anyways I can use kernel 5.4 build which I will compile for the device I want to test it for ?
Currently I am following the build steps as per the ipfire wiki. There is no details after the build command.
unless cross-compiling is not an option for you (not sure if this works with ARM), you need to execute those commands on a host using the same architecture - in this case, your PineBook.
Now I see it is downloading u-boot (2018.03), which means it is fetching all the packages which are needed to build an aarch64 rootfs of ipfire, So I will have to know how to build pkgfire package this way I can build uboot-rockpi-e and linux-rockchip which I normally do for Manjaro using PKGBUILD.
Please advice on how I can build package for ipfire.
