Yes, I am aware of these kernel perf knobs, it is totally fine to adjust these settings while trouble shooting performance issue and collect performance data, and turn these settings back to security setting after trouble shooting. perf is very common tool being used in all kinds of enterprise Linux os system.
perf can be built statically so no compiler or linker required in running system. in IPFire case, perf can be built when IPFire kernel is built and the result perf binary can be copied to running IPFire system
diff --git a/lfs/linux b/lfs/linux
index 51b0804e9..7b7220319 100644
--- a/lfs/linux
+++ b/lfs/linux
@@ -171,6 +171,9 @@ else
cd $(DIR_APP)/tools/bpf/bpftool && sed -i -e 's/^prefix ?= \/usr\/local/prefix ?= \/usr/' Makefile
cd $(DIR_APP)/tools/bpf/bpftool && make $(MAKETUNING)
+ # Build perf tool
+ cd $(DIR_APP)/tools/perf && NO_JEVENTS=1 NO_LIBTRACEEVENT=1 make $(MAKETUNING)
+
# Install the kernel
cd $(DIR_APP) && cp -v arch/$(KERNEL_ARCH)/boot/$(KERNEL_TARGET) /boot/vmlinuz-$(VER)-$(VERSUFIX)
cd $(DIR_APP) && cp -v System.map /boot/System.map-$(VER)-$(VERSUFIX)
@@ -180,6 +183,9 @@ else
# Install bpftool
cd $(DIR_APP)/tools/bpf/bpftool && make install
+ # Install perf tool
+ cd $(DIR_APP)/tools/perf && install -m 755 perf /usr/sbin/
+
1 Like
They changed the Kernel that has the bug is in 188 and 189. As far as I know , 190 has the same kernel as the other two and its going to duplicate the same issue with the same hardware on any computer.
CU187 had kernel 6.6.32.
This was updated to 6.6.47 for CU188 & 189.
CU190 (Unstable) has kernel 6.6.56.
ok, cu187 iso install was a working on my AMD machine, but security patch for bugs Grimm discovered wasn’t implemented.
6.6.47 broke the spector patch and caused my AMD system to go crazy like what this guy is expirencing. Even though I had hyperthreading turned off. Patching with 6.6.56 has fixed my current cu188 install.
So what is the difference between CU188 and CU190, and where is cu190 download?
There hasn’t been many issues like this happening other than bad drivers introduced by vendors, but the last set that I know of that caused intermittent issues was broadcom drivers about 4 years ago.
But what version for it worked correctly for the thread starter?
I wonder what driver green0 is using:
ethtool -i green0
Possibly, the netcard settings in firmware could be out of sync with the driver. Invoking a setting would fix it if that is the issue, but the interface setting would have to be passed at boot time for it to be permanent. As the card’s firmware is ignoring default driver profile.
Then this would fix the issue:
ethool -s green0 speed 1000 duplex full
if this turns off the interface, then the auto-negotiation needs to be turn off, but that command is driver specific.
Then you would have to download the manufacturer’s drivers and management software to manage the ports. Which is common to do anyways with a multi port card.
Hello @bonnietwin, @dr_techno, @ms, @bbitsch, @vincentmli and all,
after several days (difficult to intervene in customer on my production equipment) and many cross-tests, I come back to you with new elements.
The original symptoms: Degradation of IPFire performance from startup, high disk load from startup of the RED network interface - results: Lack of responsiveness of network cards (very high latency > 3000 ms), GUI regularly inaccessible even on GREEN, SSH manipulations greatly slowed down, etc.
What I noticed:
-
Lithium battery out of order, replaced
-
Problem is not related to network equipment (same symptoms with or without Ethernet switch, directly connected behind GREEN interface, etc.)
-
On a test machine, IPFire operation is normal in CU183, 186, 189 or even 190 [nightly]) from scratch, as long as a configuration restore has not been performed.
-
After restoring the IPFire configuration and rebooting, the disk (SSD) is heavily used and this from the start of the RED interface (the disk heats up and becomes burning a few hours later)
Notes:
On the production IPFire, I suspect an anomaly related to the IPFire storage (file system integrity) which would have been reflected in the configuration backup…
On the production IPFire, I performed several reboots and file system checks proposed by IPFire but without obvious evolution.
Although time-consuming, I can redo the entire IPFire configuration without going through a restore, however my client uses OpenVPN and it is not possible for me to regenerate and copy new certificates on all user machines (Windows and Mac, most of which are mobile)
As long as the problem you are experiencing is not related to any of the files related to OpenVPN in your backup then there is an option.
Manually redo the configuration for everything including creating a new OpenVPN setup for the server and one client configuration.
Then take your backup and run the following command on it
tar -axf /path/to/backup-file.ipf
The .ipf file is just basically a tar file and the above command will extract everything into the directory that you ran the above command from.
Then copy all files with non-zero size from the var/ipfire/ovpn/ directory in the extracted archive to your IPFire system.
That will replace the server ca and root certificate from your backup and all the client connection certficates as previously used and any other files that represent your original setup just as if you have done a restore but just for the ovpn portion.
I believe that having created an OpenVPN configuration before copying the files from your backup should mean that all the links that are mentioned in the backup will be in place but I haven’t tried that to be certain.
Later today I will try and test this out on my vm setup to confirm what I have described above.
I have tested it out and it works.
You don’t even have to create any initial OpenVPN settings.
I tested out the openvpn connection on a clone of my testbed system. The connection worked. then did a backup of the system and moved the file off of IPFire.
I then did a fresh install of CU189 onto that system.
I extracted the backup file and then copied the /var/ipfire/ovpn directory and its contents recursively to the /tmp/ directory on my vm IPFire test system with CU189 freshly installed. I used scp for that but you could also put the contents onto a usb stick and add it to the Extra HD entry presuming the system you are working on is a physical system.
If you have the ovpn contents from your backup file on a usb stick then after mounting the usb stick you can copy the contents of the usb stick ovpn directory recursively into the /var/ipfire/ovpn directory.
You probably will then need to recursively chown the /var/ipfire/ovpn directory to nobody:nobody. When I did my cp some of the entries became root:root.
I then tested again the openvpn connection with the same client and it successfully connected, showing the root and host server certs and the client connection certs were correctly restored.
4 Likes
Hello @bonnietwin,
I just tried your method of restoring OpenVPN settings and certificates and it works perfectly, great !!!
It has the big advantage of being simple and quick to implement, without reboot !
Note : As @bonnietwin rightly pointed out, after copying the files from /var/ipfire/ovpn, make sure that the file rights are set to nobody:nobody because those that were still at root:root did not allow the connection to be successful (especially the files in the /var/ipfire/ovpn/certs folder).
Many thanks to Michael, Bernhard, Dave and Vincent who supported me in solving this problem, and in particular @bonnietwin for his responsiveness and the relevance of his solution (!!!).
To all, have a good week-end,
Stéphane
4 Likes