I have run some tests on a physical device to see if I could replicate what you are seeing.
I installed CU197 on the device and then I ran the update to CU198 from the console command line with pakfire upgrade
Log showed the following
12:41:40 pakfire: DOWNLOAD STARTED: paks/core-upgrade-2.29-198.ipfire
12:41:40 pakfire: MIRROR INFO: 24 servers found in list
12:41:40 pakfire: DOWNLOAD INFO: Host: mirror.cedia.org.ec (HTTPS) - File: ipfire/pakfire2/2.29-x86_64/paks/core-upgrade-2.29-198.ipfire
12:41:42 pakfire: DOWNLOAD INFO: ipfire/pakfire2/2.29-x86_64/paks/core-upgrade-2.29-198.ipfire has size of 60046479 bytes
12:42:15 pakfire: DOWNLOAD INFO: HTTP-Status-Code: 200 - 200 OK
12:42:16 pakfire: DOWNLOAD INFO: File received. Start checking signature...
12:42:18 pakfire: DOWNLOAD INFO: Signature of core-upgrade-2.29-198.ipfire is fine.
12:42:18 pakfire: DOWNLOAD FINISHED: ipfire/pakfire2/2.29-x86_64/paks/core-upgrade-2.29-198.ipfire
The download time was 38 seconds.
I then re-installed CU197 on the same system and ran the update via the WUI and got the following log
13:11:59 pakfire: DOWNLOAD STARTED: paks/core-upgrade-2.29-198.ipfire
13:11:59 pakfire: MIRROR INFO: 24 servers found in list
13:11:59 pakfire: DOWNLOAD INFO: Host: mirrors.ocf.berkeley.edu (HTTPS) - File: ipfire/pakfire2/2.29-x86_64/paks/core-upgrade-2.29-198.ipfire
13:11:59 pakfire: DOWNLOAD INFO: ipfire/pakfire2/2.29-x86_64/paks/core-upgrade-2.29-198.ipfire has size of 60046479 bytes
13:12:08 pakfire: DOWNLOAD INFO: HTTP-Status-Code: 200 - 200 OK
13:12:09 pakfire: DOWNLOAD INFO: File received. Start checking signature...
13:12:11 pakfire: DOWNLOAD INFO: Signature of core-upgrade-2.29-198.ipfire is fine.
13:12:11 pakfire: DOWNLOAD FINISHED: ipfire/pakfire2/2.29-x86_64/paks/core-upgrade-2.29-198.ipfire
The download time was 12 seconds.
A bit faster but note that the mirror was different. First time was a mirror from ecuador and the second time from USA. The routing paths could easily make a difference in the download times, especially as there was a 18 minute gap between the two tests due to the full installation step.
So I then did another fresh install of CU197 on the device and then ran the update again but with the mirror defined as mirrors.ocf.berkeley.edu again. Logs showed
14:13:29 pakfire: DOWNLOAD STARTED: paks/core-upgrade-2.29-198.ipfire
14:13:29 pakfire: DOWNLOAD INFO: Host: mirrors.ocf.berkeley.edu (HTTPS) - File: /ipfire/pakfire2/2.29-x86_64/paks/core-upgrade-2.29-198.ipfire
14:13:30 pakfire: DOWNLOAD INFO: /ipfire/pakfire2/2.29-x86_64/paks/core-upgrade-2.29-198.ipfire has size of 60046479 bytes
14:13:41 pakfire: DOWNLOAD INFO: HTTP-Status-Code: 200 - 200 OK
14:13:42 pakfire: DOWNLOAD INFO: File received. Start checking signature...
14:13:44 pakfire: DOWNLOAD INFO: Signature of core-upgrade-2.29-198.ipfire is fine.
14:13:44 pakfire: DOWNLOAD FINISHED: /ipfire/pakfire2/2.29-x86_64/paks/core-upgrade-2.29-198.ipfire
giving a download time of 15 seconds. This compares very similarly with the WUI time with the same mirror of 12 seconds and especially as there was a gap of 58 minutes between the two tests with the same mirror due to the full installation step plus I had to go and work on some other stuff.
So from my perspective I don’t see any big difference between the pakfire upgrade download via the console command line or the WUI.
Just as a confirmation, when you say you are using the console is this via hdmi for the screen and usb for the keyboard or are you using the serial console.
If the latter then I could believe that there might be an impact but I am not currently able to test this and if the WUI is faster for you then that seems fine as the WUI is the intended upgrade route. The console approach for pakfire is there to be availabe as a backup route if something goes wrong with the WUI access.