Improving aarch64-image booting (+ ESPRESSObin network setup)

Thanks @gandalf and @arne_f for the initial findings and recomendations to boot IPFire on expressobin hardware.

I have an 1GHz espressobin board (the regular, not the ultra version) which is confirmed to “only” come with one internal 1Gb duplex connection to an internal 3x1Gb ethernet switch, but it also has one separate mPCIe slot for an additional ethernet, Wifi, or SATA-controller. Unfortunately, the slot does not directly support mSATA cards. And there does not seem to be a (battery buffered?) real-time clock.

I hope for QoS to decently manage an (occasional) 900Mbit speed (backup/restore) between green<->orange, while routing the consumer grade VDSL home internet of red.

Not all my attempts to boot from sdcard and to follow your recommendations did work out.
But I could reproduce @gandalf’s boot method.

EDIT:

What I could not find out is how to manually (re)run the initial network gui (wiki.ipfire.org - Step 5: Network Setup), or to properly configure the 3 interfaces manually.
How were you able to configure the MACs etc. gandalf?

See solution below: Post #4 - “0). Booting)



Trying the recommended .scr loading:
(not sure how this method works, I’ve set up armbian’s u-boot and environment):

(with addr from openwrt 18.xx example)
load mmc 0:1 0x4d00000 boot.scr; source 0x4d00000
-> this just rebooted u-boot

(with addr from armbian uboot example)
load mmc 0:1 0x6d00000 boot.scr; source 0x6d00000
-> Unknown command ‘bootz’ - try ‘help’
-> Bad Linux ARM64 Image magic!

Trying fatload from sdcard:

fatload mmc 0:1 ${fdt_addr} /dtb-4.14.212-ipfire/marvell/armada-3720-espressobin.dtb
fatload mmc 0:1 $kernel_addr vmlinuz-4.14.212-ipfire
setenv bootargs $console root=/dev/mmcblk0p3 rw rootwait
booti $kernel_addr - $fdt_addr

-> boots only until a message saying to wait for /dev/mmcblk0p3 (forever)
(kernel missing the driver?)


Trying fatload from usb 2.0 port:

fatload usb 0:1 ${fdt_addr} /dtb-4.14.212-ipfire/marvell/armada-3720-espressobin.dtb
fatload usb 0:1 $kernel_addr vmlinuz-4.14.212-ipfire
setenv bootargs $console root=/dev/sda3 rw rootwait
booti $kernel_addr - $fdt_addr

-> works :blush:


But now I’d need an idea how to redo and fix the network config.

Looks like I got the espressobin network working.

1.) IPFire Network Settings

Found the basic configuration can be re-done by executing setup, but for this workaround I still had to edit /var/ipfire/ethernet/settings manually, and take care to:

  • Only specify the mv88e6085 driver in this file (and for all switch ports), to avoid that some early boot process sets up the internal mvneta network controller as an ipfire device.

  • Not configure any MAC addresses for switch ports. They should first be set up by an /etc/init.d/swconfig.user, and can then be selected afterwards using the NIC assignments in the web-config under Network -> Zone Configuration.

Example/var/ipfire/ethernet/settings:

CONFIG_TYPE=2
RED_DEV=eth1 
RED_MACADDR=
RED_DESCRIPTION='"of: mv88e6085"'
RED_DRIVER=mv88e6085
RED_DHCP_HOSTNAME=localhost
RED_DHCP_FORCE_MTU=
RED_ADDRESS=0.0.0.0
RED_NETMASK=0.0.0.0
RED_TYPE=DHCP
RED_NETADDRESS=0.0.0.0
ORANGE_DEV=eth2
ORANGE_MACADDR=
ORANGE_DESCRIPTION='"of: mv88e6085"'
ORANGE_DRIVER=mv88e6085
ORANGE_ADDRESS=192.168.2.1
GREEN_DEV=eth3 
GREEN_MACADDR=
GREEN_DESCRIPTION='"of: mv88e6085"'
GREEN_DRIVER=mv88e6085
GREEN_ADDRESS=192.168.3.1
GREEN_NETMASK=255.255.255.0
GREEN_NETADDRESS=192.168.3.0

2.) swconfig Additions

Added a custom /etc/init.d/swconfig.user, but devs could instead integrate this board’s case into /etc/init.d/swconfig.

Note that instead of using the proper MAC addresses from the u-boot environment, my swconfig.user script always falls back to generating a random address range and saving these to the ipfire config.

#!/bin/sh
########################################################################
# Begin $rc_base/init.d/swconfig.user
#
# Description : Workaround script to fix lan switch.
########################################################################

. /etc/sysconfig/rc
. ${rc_functions}

if [ -e /var/ipfire/ethernet/swconfig_mac ]; then
	SWMAC=`cat /var/ipfire/ethernet/swconfig_mac`
else
	# Generate a random local administrated mac address for vlan swconfig.
	SWMAC=`printf "%1x2:%02x:%02x:%02x:%02x" $[RANDOM%16] $[RANDOM%256] $[RANDOM%256] $[RANDOM%256] $[RANDOM%256]`
	echo $SWMAC > /var/ipfire/ethernet/swconfig_mac
fi



case "${1}" in
	start)
		case `cat /proc/device-tree/model 2>/dev/null` in
			"Globalscale Marvell ESPRESSOBin Board")
				# 
				# configure the internal nic
				ip link set eth0 address $SWMAC:00
				ip link set eth0 up
 
				# set up the switch child ports with names 
				# as shown in the web interface
				ip link set wan address $SWMAC:01
				ip link set wan name eth1
				ip link set eth1 up

				ip link set lan0 address $SWMAC:02
				ip link set lan0 name eth2
				ip link set eth2 up

				ip link set lan1 address $SWMAC:03
				ip link set lan1 name eth3
				ip link set eth3 up
			;;
		esac
		exit 0
	;;

	*)
		echo "Usage: ${0} {start}"
		exit 1
	;;
esac

# End $rc_base/init.d/swconfig.user

3.) Zone Configuration

Reconfigure the NIC assignments in the web-config under Network -> Zone Configuration, and reboot.

@gandalf Could you try to reproduce?
@arne_f Would you consider merging the espressobin case into /etc/init.d/swconfig? Just by being in place on the first boot it should already make the manual config workarounds unnecessary.

Here is an edited version of the booting problems, now including workarounds:
(This forum software prevents properly adding it to the first post.)



0.) Booting


The default and recommended boot.scr loading did not work out of the box. (Armbian’s u-boot and environment installed, here.)

(Manual attempt with memory address from old openwrt 18.xx example)
load mmc 0:1 0x4d00000 boot.scr; source 0x4d00000
-> this just rebooted u-boot

(Manual attempt with address from uboot)
load mmc 0:1 ${scriptaddr} boot.scr; source ${scriptaddr}
-> Unknown command ‘bootz’ - try ‘help’
-> Bad Linux ARM64 Image magic!

So, currently booting still needs a number of workarounds, until the boot.scr itself gets fixed.

a) uEnv.txt

See these additions to put into the uEnv.txt in the image:

### Manual uEnv.txt customizations to be loaded by boot.scr (compiled version of boot.cmd). ###

## board type
fdtfile=marvell/armada-3720-espressobin.dtb


## Workaround
## boot.scr currently depends on $boot_dev to be properly set from u-boot environment,
## to be able to load from devices other than mmc.
## If not set by u-boot, it is requireded to specify it here.
##
## If the boot environment (e.g. armbian) sets $boot_interface to the
## probed device, assigning that here may also be a workaround,
## until boot.scr can properly check and work with either one.

# But this did not seem to get evaluated here.
# Had to add "setenv boot_dev $boot_interface;" to bootcmd_${target} variables.
#boot_dev=${boot_interface}


## Workaround
## for UUIDs in $root_dev (as above), as long as they do not to work if ramdisk loading fails.
## So until boot.scr can properly fall back the $root_dev when ramdisk loading fails,
## it is required to specify a $root_dev path here in uEnv.txt.
##
## Currently even for booting from mmc when $boot_dev is properly set,
## because then $root_dev isn't initialized.

root_dev=/dev/mmcblk0p3
#root_dev=/dev/sda3


## Workaround
## to avoid kernel loading crashes with boot.scr (at least on espressobin).
## Maybe due to boot.scr using $fdt_addr_r, ramdisk loading, console=$console, ...?

uenvcmd=fatload $boot_dev 0:1 $fdt_addr /dtb-${KVER}-ipfire${kernel_type}/${fdtfile}; fatload $boot_dev 0:1 $kernel_addr vmlinuz-${KVER}-ipfire${kernel_type}; setenv bootargs $console root=$root_dev rw rootwait; booti $kernel_addr - $fdt_addr;

# including this last line comment ensures the previous line gets executed

It’s a little strange why boot.scr can not boot with the commands it uses, and it only works by providing a uenvcmd in uEnv.txt. I did some tests but to no avail:

# boot.scr testing:
load mmc 0:1 ${scriptaddr} boot.scr; source ${scriptaddr}


#with the current boot.scr commands kernel boot crashes:

fatload mmc 0:1 ${fdt_addr_r} /dtb-4.14.212-ipfire/marvell/armada-3720-espressobin.dtb
fatload mmc 0:1 ${kernel_addr_r} vmlinuz-4.14.212-ipfire
fatload mmc 0:1 ${ramdisk_addr_r} uInit-4.14.212-ipfire

# this line crashes, but works after setting bootargs as below
booti ${kernel_addr_r} - ${fdt_addr_r};


# try same without using _r address?:

fatload mmc 0:1 ${fdt_addr} /dtb-4.14.212-ipfire/marvell/armada-3720-espressobin.dtb
fatload mmc 0:1 ${kernel_addr_r} vmlinuz-4.14.212-ipfire
fatload mmc 0:1 ${ramdisk_addr_r} uInit-4.14.212-ipfire (works if replaced by set bootargs)
booti ${kernel_addr_r} - ${fdt_addr}

#works:
fatload mmc 0:1 ${fdt_addr} /dtb-4.14.212-ipfire/marvell/armada-3720-espressobin.dtb
fatload mmc 0:1 $kernel_addr vmlinuz-4.14.212-ipfire
setenv bootargs $console root=/dev/mmcblk0p3 rw rootwait

# and all these alternatives are also working:
setenv bootargs root=/dev/mmcblk0p3 rw rootwait
setenv bootargs root=/dev/mmcblk0p3 rootwait
setenv bootargs rootwait
setenv bootargs console=$console root=/dev/mmcblk0p3 rw rootwait

booti $kernel_addr - $fdt_addr

b) u-boot environment

  • It needs to support loading from a FAT formatted filesystem, if not already.
    For example, in the armbian environment, prepend the boot_a_script variable to first try "load... ; " before its usual "ext4load...;"

  • And it needs to set the variable $boot_dev properly.
    For example, in the armbian environment, check the current environment and insert an additional “setenv boot_dev $boot_interface; ” to the bootcmd_${target} variables:

    printenv

    setenv bootcmd_mmc0 ‘setenv devnum 0; setenv boot_interface mmc; setenv boot_dev $boot_interface; run scan_dev_for_boot;’

    setenv bootcmd_mmc1 ‘setenv devnum 1; setenv boot_interface mmc; setenv boot_dev $boot_interface; run scan_dev_for_boot;’

    setenv bootcmd_sata ‘setenv devnum 0; scsi scan; scsi dev 0; setenv boot_interface scsi; setenv boot_dev $boot_interface; run scan_dev_for_boot;’

    setenv bootcmd_usb ‘setenv devnum 0; usb start;setenv boot_interface usb; setenv boot_dev $boot_interface; run scan_dev_for_boot;’

    saveenv

The aarch64 boot script workarounds are more general, so I wanted to adjust the topic, but this is also not possible (anymore).

First, thanks to the mods for adjusting the title and text.

Concerning the espressobin / u-boot serial console I recently I noticed a problem:

Sometimes commands on the u-boot prompt only falsely return “command not found”. Especially when using quotes in the commands like the adjustments in 0b) u-boot environment. What helped was using screen /dev/ttyUSB0 115200 and using the reset button of the board.

Update: The custom swconfig.user (and a proper swconfig integration) needs to create the red0, orange0 and green0 device names to allow things like QoS to work.

#!/bin/sh
########################################################################
# Begin $rc_base/init.d/swconfig.user
#
# Description : Workaround script to fix lan switch.
########################################################################

. /etc/sysconfig/rc
. ${rc_functions}

if [ -e /var/ipfire/ethernet/swconfig_mac ]; then
	SWMAC=`cat /var/ipfire/ethernet/swconfig_mac`
else
	# Generate a random local administrated mac address for vlan swconfig.
	SWMAC=`printf "%1x2:%02x:%02x:%02x:%02x" $[RANDOM%16] $[RANDOM%256] $[RANDOM%256] $[RANDOM%256] $[RANDOM%256]`
	echo $SWMAC > /var/ipfire/ethernet/swconfig_mac
fi



case "${1}" in
	start)
		case `cat /proc/device-tree/model 2>/dev/null` in
			"Globalscale Marvell ESPRESSOBin Board")
				# 
				# configure the internal nic
				ip link set eth0 address $SWMAC:00
				ip link set eth0 up
 
				# set up the switch child ports with names 
				# as shown in the web interface
				ip link set wan address $SWMAC:01
				ip link set wan name red0
				ip link set red0 up

				ip link set lan0 address $SWMAC:02
				ip link set lan0 name orange0
				ip link set orange0 up

				ip link set lan1 address $SWMAC:03
				ip link set lan1 name green0
				ip link set green0 up
			;;
		esac
		exit 0
	;;

	*)
		echo "Usage: ${0} {start}"
		exit 1
	;;
esac

# End $rc_base/init.d/swconfig.user

and /var/ipfire/ethernet/settings also:

CONFIG_TYPE=2
RED_DEV=red0
RED_MACADDR=
RED_DESCRIPTION='"of: mv88e6085"'
RED_DRIVER=mv88e6085
RED_DHCP_HOSTNAME=localhost
RED_DHCP_FORCE_MTU=
RED_ADDRESS=0.0.0.0
RED_NETMASK=0.0.0.0
RED_TYPE=DHCP
RED_NETADDRESS=0.0.0.0
ORANGE_DEV=orange0
ORANGE_MACADDR=
ORANGE_DESCRIPTION='"of: mv88e6085"'
ORANGE_DRIVER=mv88e6085
ORANGE_ADDRESS=192.168.2.1
GREEN_DEV=green0
GREEN_MACADDR=
GREEN_DESCRIPTION='"of: mv88e6085"'
GREEN_DRIVER=mv88e6085
GREEN_ADDRESS=192.168.3.1
GREEN_NETMASK=255.255.255.0
GREEN_NETADDRESS=192.168.3.0

Another finding:
I got command not found errors with the u-boot commands, and it seemed related to using the single quote “ticks” in the commands listed above.

In the end, it worked after connecting to the serial console not with gtkterm, but with screen /dev/ttyUSB0 115200 in an xterm window.