Yes I have qemu and qemu static for aarch64 and binfmt I am on manjaro which is based on arch linux.
This package was enough for me to install on Debian for testing.
Do you have this set up properly? There should be some virtual file system being mounted:
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
This is what I have in there:
root@michael:/build/ipfire.org# ll /proc/sys/fs/binfmt_misc
total 0
-rw-r--r-- 1 root root 0 Aug 5 16:52 python2.7
-rw-r--r-- 1 root root 0 Aug 5 16:52 python3.7
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-aarch64
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-alpha
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-arm
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-armeb
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-cris
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-hppa
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-m68k
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-microblaze
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-mips
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-mips64
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-mips64el
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-mipsel
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-ppc
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-ppc64
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-ppc64abi32
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-ppc64le
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-riscv32
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-riscv64
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-s390x
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-sh4
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-sh4eb
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-sparc
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-sparc32plus
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-sparc64
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-xtensa
-rw-r--r-- 1 root root 0 Aug 5 16:52 qemu-xtensaeb
--w------- 1 root root 0 Aug 5 16:52 register
-rw-r--r-- 1 root root 0 Aug 5 16:52 status
Debian created these automatically for me. make.sh searches through this directory to find the correct executable for aarch64.
Searching for âmanjaroâ and âbinfmtâ shows a couple of articles where people are struggling and need additional packages: Is a re-install needed after a cpu/main board swap? - Support - Manjaro Linux Forum
I do have the same, as I checked it before also we do package arm stuff using a similar approach with qemu.
This is the tool we use to build packages and also images for arm64 devices.
Even we need binfmt service to build package using our tools as explained in the git project link whic I quote below.
Install binfmt-qemu-static package and make sure systemd-binfmt is running
AFAIR we have binfmt as a service which is enabled when we install the package.
I will look into this again once I get home.
If it is working on your side with debian then I am sure there is some issue on my build server maybe location of the files are not the same, I will also try to build a package using manjaro-arm-tool and then test ipfire make script.
Thank you for the response.
The /proc file system should always look the same.
make.sh is looking for the executable in that directory. That can be anywhere and hence we run the search function.
Please post the error messages in full. make.sh performs some more checks on qemu to check if it is suitable for use. Maybe that fails.
1 Like
Hello,
Here is what I have which is similar to yours.
ls -l /proc/sys/fs/binfmt_misc
total 0
-rw-r--r-- 1 root root 0 Aug 6 23:16 appimage-type1
-rw-r--r-- 1 root root 0 Aug 6 23:16 appimage-type2
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-aarch64
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-alpha
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-arm
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-armeb
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-cris
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-m68k
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-microblaze
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-mips
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-mipsel
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-ppc
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-ppc64
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-ppc64abi32
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-riscv64
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-s390x
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-sh4
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-sh4eb
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-sparc
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-sparc32plus
-rw-r--r-- 1 root root 0 Aug 6 23:16 qemu-sparc64
--w------- 1 root root 0 Aug 6 23:16 register
-rw-r--r-- 1 root root 0 Aug 6 23:16 status
Also I just build uboot for rockpi4c over our manjaro-arm-tools which uses qemu too and it was built without any problem.
Log Here
After this build is complete I will try again with ipfire build process.
Update:
Build complete on our tools.
CC tpl/drivers/sysreset/sysreset-uclass.o
CC spl/lib/uuid.o
CC spl/lib/rand.o
CC spl/lib/panic.o
LD tpl/lib/built-in.o
CC tpl/drivers/serial/ns16550.o
CC spl/lib/tiny-printf.o
CC spl/lib/strto.o
CC tpl/drivers/sysreset/sysreset_rockchip.o
CC spl/lib/date.o
CC spl/lib/elf.o
LD tpl/drivers/sysreset/built-in.o
LD spl/lib/built-in.o
LD tpl/drivers/serial/built-in.o
LD spl/u-boot-spl
OBJCOPY spl/u-boot-spl-nodtb.bin
CAT spl/u-boot-spl-dtb.bin
COPY spl/u-boot-spl.bin
LD tpl/drivers/ram/rockchip/built-in.o
LD tpl/drivers/ram/built-in.o
LD tpl/drivers/built-in.o
LD tpl/u-boot-tpl
OBJCOPY tpl/u-boot-tpl-nodtb.bin
CAT tpl/u-boot-tpl-dtb.bin
COPY tpl/u-boot-tpl.bin
MKIMAGE tpl/u-boot-tpl-rockchip.bin
CAT idbloader.img
CAT u-boot-rockchip.bin
CFGCHK u-boot.cfg
==> Entering fakeroot environment...
==> Starting package()...
==> Tidying install...
-> Removing libtool files...
-> Purging unwanted files...
-> Removing static library files...
-> Stripping unneeded symbols from binaries and libraries...
-> Compressing man and info pages...
==> Checking for packaging issues...
==> Creating package "uboot-rockpi4c"...
-> Generating .PKGINFO file...
-> Generating .BUILDINFO file...
-> Adding install file...
-> Generating .MTREE file...
-> Compressing package...
==> Leaving fakeroot environment.
==> Finished making: uboot-rockpi4c 2020.07-2 (Thu Aug 6 20:41:05 2020)
==> Cleaning up...
==> Package Succeeded...
-> Extracting finished package out of rootfs...
==> Package saved as {uboot-rockpi4c} in {/var/cache/manjaro-arm-tools/pkg/aarch64}...
-> Cleaning build files from rootfs
==> Time : 11.68 minutes...
Next is ipfire.
This is what I get
sudo ./make.sh --target=aarch64 clean
[sudo] password for spikerguy:
ERROR: Could not find a binfmt_misc helper entry for aarch64 [ FAIL ]
Check /home/spikerguy/Desktop/pkg-build/ipfire-2.x/log/_build.preparation.log for errors if applicable [ FAIL ]
[spikerguy@spikerguy-gtr ipfire-2.x]$
There is no log file though.
[spikerguy@spikerguy-gtr log]$ ls
[spikerguy@spikerguy-gtr log]$
I removed all my changed that I had done to the make script in
#Set default qemu options where I added QEMU Cpu with cortex-a57 and in qemu_is_required call to added x86_64,aarch64
After that I get env error.
cat _build.base.log
Aug 6 21:55:36: Building stage2 chroot: failed to run command 'env': No such file or directory
I tried binfmt-user-static from aur as well as Manjaro-user-static package but still no luck 
Thank you.
Donât waste your time getting this work. We have not using qemu-user builds since years because it is so slow. Also the current version of perl is not compatible with this method.
even my Odroid-c2 is more than twice fast than a ryzen5 1600AF. (tested up to failing perl)
Good point.
I just receive my replacement device as my build was dead.
I will start building it again on native aarch64 device.
Hopefully this weekend.
Thank you.
UPDATE:
Had put it to build last night and now it have completed took approx 16 hours and 46 minutes to complete.
Now I can start looking into device specific kernel package while have the uboot image to flash but I need to know how the ipfire boots after the uboot is read by the device.
I will flash an image of aarchc64 on sd card and see whats inside the boot partition.
Update2:
Linux Kernel 4.14 have support for RK3328 already so we just need to add rockpie overlay to it.
I have created a gist for now, I can test the uboot while kernel we have to do it together alone with ipfire configs.
Here is the gist for patches for kernel 4.14 and mainline uboot.
Can anyone from ipfire look into the patch I have got ready for 4.14 @ms or @arne_f, Just let me know how I can add it to the existing kernel build for ipfire then I can get it done from next time, I also have the mainline uboot patch ready.
Please check and let me know if there is anything more needed to add rockpie support.
Thanks.
1 Like
That sounds good. But please follow this to submit any patches for review:
That way, we have a proper papertrail on our mailing list, everyone from the development community is involved and our tools automatically pick up the patches from the list and make them easier to merge.
1 Like
Thanks,
I will join the Mailing list and push this patch as per the guide.
Quite excited to try ipfire on a board that i can support.
Hello,
I am looking into ways to push the patch, I have found where to place the uboot patch but not sure about kernel patch location, I will look into it further as I get time on the weekend.
To test the build i tried using rockc64 dtb as it is a similar device like RockPiE except this one have 2 onboard lan. I used the correct uboot image and flashed it over the sd card.
Now It does boot into the image but fails to load the kernel.
Log below.
INF [0x0] TEE-CORE:init_teecore:83: teecore inits done
INFO: BL31: Preparing for EL3 exit to normal world
INFO: Entry point address = 0x200000
INFO: SPSR = 0x3c9
U-Boot 2020.04-1 (May 22 2020 - 13:38:59 +0000) Manjaro ARM
Model: Pine64 Rock64
DRAM: 1022 MiB
PMIC: RK8050 (on=0x40, off=0x00)
MMC: rksdmmc@ff500000: 1, rksdmmc@ff520000: 0
Loading Environment from MMC... *** Warning - bad CRC, using default environment
In: serial@ff130000
Out: serial@ff130000
Err: serial@ff130000
Model: Pine64 Rock64
Net: eth0: ethernet@ff540000
Hit any key to stop autoboot: 0
Card did not respond to voltage select!
switch to partitions #0, OK
mmc1(part 0) is current device
Scanning mmc 1:1...
Found U-Boot script /boot.scr
2915 bytes read in 3 ms (948.2 KiB/s)
## Executing script at 00500000
Loading Linux 4.14.184-ipfire ...
error: invalid magic number.
Loading initial ramdisk ...
error: you need to load the kernel first.
Press any key to continue...
I have few other questions regarding the default aarch64 image that was created after compiling from source.
Update:
I see that there is hardcoded boot.cmd for few devices, I will have to understand how the loads the dtb and kernel from there.
Why canât we use extlinux.conf as it is much better then boot scripts for individual devices.
Normal this script parse some variables from u-boot environment. The hardcoded variables are only some quirks.
because some socâs like the RPi can only boot from fat32 partitions. (also efi must on a FAT partition)
We ship an uncompressed aarch64 kernel image because the old u-boot has a bug to detect the compressed image. Maybee this is now fixed and the new u-boot need the compressed one.
1 Like
HĂ©llo all,
I have find an image for arm64 here :
https://nightly.ipfire.org/next/latest/aarch64/ipfire-2.25.2gb-ext4.aarch64-full-core150.img.xz
I get it booted up on my arm64 marvell espressobin v7 :
Marvell>> usb reset
resetting USB...
USB0: Register 2000104 NbrPorts 2
Starting the controller
USB XHCI 1.00
USB1: USB EHCI 1.00
scanning bus 0 for devices... 2 USB Device(s) found
scanning bus 1 for devices... 1 USB Device(s) found
scanning usb for storage devices... 1 Storage Device(s) found
Marvell>> fatload usb 0:1 $fdt_addr /dtb-4.14.196-ipfire/marvell/armada-3720-espressobin.dtb
reading /dtb-4.14.196-ipfire/marvell/armada-3720-espressobin.dtb
7400 bytes read in 28 ms (257.8 KiB/s)
Marvell>> fatload usb 0:1 $kernel_addr vmlinuz-4.14.196-ipfire
reading vmlinuz-4.14.196-ipfire
17601024 bytes read in 257 ms (65.3 MiB/s)
Marvell>> setenv bootargs $console root=/dev/sda3 rw rootwait
Marvell>> printenv bootargs
bootargs=console=ttyMV0,115200 earlycon=ar3700_uart,0xd0012000 root=/dev/sda2 rw rootwait
booti $kernel_addr - $fdt_addr
I have manually get networks up :
[root@ipfire ~]# ip link set eth0 up
[root@ipfire ~]# ip link set lan1 up
[root@ipfire ~]# ip link set wan up
[root@ipfire ~]# ip link set dummy0 up
[root@ipfire ~]# cat /var/ipfire/ethernet/settings
CONFIG_TYPE=1
DEFAULT_GATEWAY=10.4.2.1
RED_DEV=wan
RED_MACADDR=22:4f:df:0c:0b:e1
RED_DESCRIPTION='"of: mvneta"'
RED_DRIVER=mvneta
GREEN_DEV=lan1
GREEN_MACADDR=22:4f:df:0c:0b:e1
GREEN_DESCRIPTION='"of: mvneta"'
GREEN_DRIVER=mvneta
GREEN_ADDRESS=10.4.2.250
GREEN_NETMASK=255.255.255.0
GREEN_NETADDRESS=10.4.2.0
GREEN_BROADCAST=10.4.2.255
RED_DHCP_HOSTNAME=ipfire
RED_DHCP_FORCE_MTU=
RED_ADDRESS=0.0.0.0
RED_NETMASK=0.0.0.0
RED_TYPE=DHCP
RED_NETADDRESS=0.0.0.0
RED_BROADCAST=255.255.255.255
I am now trying to get access to the webui âŠ
I am able to ping from lan1 :
[root@ipfire ~]# ping 10.4.2.1
PING 10.4.2.1 (10.4.2.1) 56(84) bytes of data.
64 bytes from 10.4.2.1: icmp_seq=1 ttl=64 time=2.03 ms
64 bytes from 10.4.2.1: icmp_seq=2 ttl=64 time=0.747 ms
Just made a fresh reboot, then logged as root via the debug console; then
ip link set eth0 up
ip link set lan1 up
ping ok
WEBUI ok via https://:444
1 Like
You can try to set ETH1ADDR= (Not exact sure about the syntax but if there is ETHADDR= it should correct) to a different mac address in uboot and run saveenv.
After this the setup should work correct.
For easier start after a kernel update you should try to load boot.scr and run it from the usb drive istead of manually load dtb and kernel. (but im not sure if the marvell uboot support all needed features but on older marvells this has worked.)
thanks, will try that allâŠ
But from my OpenWrt box, there is only eth0 then wan/lan0/lan1 as subset of eth0 :
root@C3BOX:/# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1508 qdisc mq state UP qlen 1024
link/ether xx:yy:aa:bb:ab:2a brd ff:ff:ff:ff:ff:ff
3: wan@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-wan state LOWERLAYERDOWN qlen 1000
link/ether xx:yy:aa:bb:ab:2a brd ff:ff:ff:ff:ff:ff
4: lan0@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-LAN0 state LOWERLAYERDOWN qlen 1000
link/ether xx:yy:aa:bb:ab:2a brd ff:ff:ff:ff:ff:ff
5: lan1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-LAN1 state UP qlen 1000
link/ether xx:yy:aa:bb:ab:2a brd ff:ff:ff:ff:ff:ff
Ok. Than this is different. Then you are forced to hack this manually because the setup need different macâs. Did you know how this is implemented. For me this looks like a VLAN Switch but i dont see any VLAN IDâs in the config.
Hello,
Finally I have recieved my replacement device of Nanopi R2s
So now i have 3 boards which see potential device to have ipfire for.
Also the fact that now I have another device to build packages i can dedicate a device for ipfire pkg building and it have enough storage on emmc so I donât have to worry about storage issue while compiling.
Update:
Checking Logfiles for new Files
*** Build finished in 19:32:13 [ DONE ]
Logs
It took 19 hours to build aarch64 image 
I have been trying to make it boot but It is resetting the board after grub. Used mainline uboot for the board.
RockPiE3 = RK3328 board with 1gb ram. The first issue is that the uart log is not clear so I am not able to understand what is causing the reboot.
Finally switch uart and not I can read but it is getting stuck at kernel and I think it is cause of baudrate difference
U-Boot 2020.10-1 (Oct 14 2020 - 10:55:00 +0000) Manjaro ARM
Model: Pine64 Rock64
DRAM: 1022 MiB
PMIC: RK8050 (on=0x40, off=0x00)
MMC: mmc@ff500000: 1, mmc@ff520000: 0
Loading Environment from MMC... Card did not respond to voltage select!
*** Warning - No block device, using default environment
In: serial@ff130000
Out: serial@ff130000
Err: serial@ff130000
Model: Pine64 Rock64
Net: Could not get PHY for ethernet@ff540000: addr -1
No ethernet found.
Hit any key to stop autoboot: 0
switch to partitions #0, OK
mmc0(part 0) is current device
Scanning mmc 0:1...
Found /extlinux/extlinux.conf
Retrieving file: /extlinux/extlinux.conf
245 bytes read in 3 ms (79.1 KiB/s)
1: Manjaro ARM
Retrieving file: /initramfs-4.14.208-ipfire.img
7465912 bytes read in 170 ms (41.9 MiB/s)
Retrieving file: /vmlinuz-4.14.208-ipfire
17644032 bytes read in 397 ms (42.4 MiB/s)
append: initrd=/initramfs-4.14.208-ipfire.img console=tty1 console=ttyS2,1500000 root=UUID=52066f7b-5e0a-4ac5-9b5f-ae98c6785ad6 rw rootwait
Retrieving file: /dtb-4.14.208-ipfire/rockchip/rk3328-rock64.dtb
32046 bytes read in 5 ms (6.1 MiB/s)
## Flattened Device Tree blob at 01f00000
Booting using the fdt blob at 0x1f00000
Loading Ramdisk to 3d814000, end 3df32bb8 ... OK
Loading Device Tree to 000000003d809000, end 000000003d813d2d ... OK
Starting kernel ...
2 Likes
The baudrate should not the problem if you have set this via console=ttyXX,baudrate IPFire should use the configured rate.
Donât set console= twice. The IPFire initskripts try to detect if it should use serial or video by this parameter. Iâm also not sure if ttyS2 is the correct serial console. I think it was ttyS0 on my R2S but im not sure.
1 Like
Yes I think your right, i will make some more changes and I will have to use the accurate dtb for my boards.
I will fetch dtb from official image and reuse it over ipfire image for testing and see if I can get ipfire to start if that works then it will me a simple fix with dts patch of their device.
Secondly, I would like to propose to move all devices to extconfig to make booting process very simple and easy to maintain, as most of the devices are already supported by mainline linux and it can read extlinux where we can define all the parameters for boot which can look grub.
Let me know your thoughts on extlinux idea, as maintaining boot script for devices is alot of work.
Thanks