Hi all, @tphz what is your resume according to your investigated data ? And a question to all, what metrics might be also useful in such a line check ? Dwn-Speed, Up-Speed, Ping only ? Or may something like Jitter, Packet loss, Latency and Hops information may (mtr for example is also available) ? May you have also other ideas ?
Am thinking about in general of how we can get different tools from IPFire together for a good information gathering which is in this kind of segment currently not available.
But may i´ am thinking to complex ? Anyways, nice that you guys are hacking a little around .
Latency at idle, then latency during download and upload. Similar metrics to used by bufferbloat testers. The goal being for latency to not increase during data transfer. Could be built into the QoS page?
Hi all, @bloater99 Yes, this may also be a possible result—nice idea! However, the point might be to get proper information. Latency at idle should indeed be checked first to ensure that the system is in an idle state. If there are background processes or applications using bandwidth, this can lead to inaccurate measurements.
Even a speed test can introduce noise into the results, as it often involves multiple variables that can affect latency. Therefore, it’s crucial to isolate these factors when measuring latency may with ping only ?!
The next point is, how can one be sure to get good results from the speedtest itself ?
Possible idea of how to check if a speedtest or when to test latency at idle →
#!/bin/bash
# This script evaluates network conditions for a speed test and idle state.
# It performs the following checks:
# 1. Checks the status of QoS (Quality of Service) and advises on its impact on speed tests.
# 2. Identifies the active WAN interface.
# 3. Performs a ping test to measure latency and packet loss.
# - Acceptable average latency: < 50 ms
# - Acceptable packet loss: < 1%
# 4. Checks network usage on the identified WAN interface.
# - Acceptable network usage: < 10 Kbps
# 5. Checks CPU usage on the system.
# - Acceptable CPU usage: < 20%
# 6. Evaluates whether the conditions are favorable for conducting speed tests.
#
# 07.10.2024
#############################################################################################
#
# Define color codes
GREEN='\033[0;32m' # Green color for favorable conditions
RED='\033[0;31m' # Red color for unfavorable conditions
NC='\033[0m' # No Color
# Define symbols
CHECK_MARK="${GREEN}✔${NC}" # Green check mark
CROSS_MARK="${RED}✖${NC}" # Red cross mark
# Function to check the status of QoS
check_qos_status() {
echo "Checking QoS status..."
qos_status=$(/usr/local/bin/qosctrl status)
# Check if QoS is active by looking for specific output in the status
if echo "$qos_status" | grep -q "qdisc"; then
echo "QoS is currently active."
echo "It is recommended to disable QoS during speed tests for accurate results."
qos_check_result="${CROSS_MARK}"
else
echo "QoS is not active."
qos_check_result="${CHECK_MARK}"
fi
}
perform_ping_test() {
local target="8.8.8.8" # Target IP address for ping test (Google's public DNS)
local count=10 # Number of ping requests to send
echo -e "\nPerforming ping test..."
ping_result=$(ping -c $count $target)
# Extract average latency from ping results
avg_latency=$(echo "$ping_result" | grep 'avg' | awk -F '/' '{print $5}')
# Extract packet loss percentage from ping results
packet_loss=$(echo "$ping_result" | grep 'packet loss' | awk '{print $6}')
# Output results with symbols indicating success or failure
if (( $(echo "$avg_latency < 50" | bc -l) )) && [[ "$packet_loss" == *% ]]; then
packet_loss_value=$(echo $packet_loss | tr -d '%')
if (( $(echo "$packet_loss_value < 1" | bc -l) )); then
echo -e "Average latency: ${avg_latency} ms ${CHECK_MARK}"
echo -e "Packet loss: $packet_loss ${CHECK_MARK}"
else
echo -e "Average latency: ${avg_latency} ms ${CROSS_MARK}"
echo -e "Packet loss: $packet_loss ${CROSS_MARK}"
fi
else
echo -e "Average latency: ${avg_latency} ms ${CROSS_MARK}"
echo -e "Packet loss: $packet_loss ${CROSS_MARK}"
fi
}
check_network_usage() {
echo -e "\nChecking network usage..."
# Get the RX bytes for the identified WAN interface before measuring
rx_bytes_before=$(cat /proc/net/dev | grep "$wan_interface" | awk '{print $2}')
sleep 1 # Wait for 1 second to measure usage over time
# Get the RX bytes again after the wait
rx_bytes_after=$(cat /proc/net/dev | grep "$wan_interface" | awk '{print $2}')
# Calculate the difference in RX bytes and convert to Kbps
network_usage=$(( (rx_bytes_after - rx_bytes_before) * 8 / 1024 )) # Convert to Kbps
# Check if network usage was calculated correctly
if [[ -z "$network_usage" ]]; then
echo "Error: No valid data found for network usage."
network_usage=100 # Set to a high value for evaluation if no data is found
echo -e "Average network usage: ${network_usage} Kbps ${CROSS_MARK}"
else
if (( $(echo "$network_usage < 10" | bc -l) )); then
echo -e "Average network usage: ${network_usage} Kbps ${CHECK_MARK}"
else
echo -e "Average network usage: ${network_usage} Kbps ${CROSS_MARK}"
fi
fi
}
check_cpu_usage() {
echo -e "\nChecking CPU usage..."
# Get CPU usage percentage, calculating from idle time
cpu_usage=$(mpstat 1 1 | grep 'Average' | awk '{print 100 - $12}')
# Check if CPU usage was calculated correctly
if [[ -z "$cpu_usage" ]]; then
echo "Error: No data found for CPU usage."
cpu_usage=100 # Set to a high value for evaluation if no data is found
echo -e "CPU usage: ${cpu_usage}% ${CROSS_MARK}"
else
if (( $(echo "$cpu_usage < 20" | bc -l) )); then
echo -e "CPU usage: ${cpu_usage}% ${CHECK_MARK}"
else
echo -e "CPU usage: ${cpu_usage}% ${CROSS_MARK}"
fi
fi
}
get_active_wan_interface() {
echo -e "\nDetermining active WAN interface..."
# Use the command to determine the active WAN interface based on common names
wan_interface=$(ip -o link show | awk -F': ' '$2 ~ /red0|ppp.*|ens.*|vlan.*/ && $3 ~ /UP/ {print $2}')
# Check if an interface was found
if [[ -z "$wan_interface" ]]; then
echo "No active WAN interface found."
return 1 # Return error status if no active WAN interface is found
else
echo "Found active WAN interface: $wan_interface"
return 0 # Return success status if an active WAN interface is found
fi
}
main() {
# Print header with bold formatting and add a blank line below it.
echo -e "\n\033[1mEvaluating conditions for speed test and idle state...\033[0m\n"
check_qos_status # Check and report the status of QoS
# Get the active WAN interface before proceeding with other checks
get_active_wan_interface
# Check if the WAN interface was found before proceeding with other checks
if [[ $? -ne 0 ]]; then
echo "The script cannot proceed as no active WAN interface was found."
exit 1 # Exit with error status if no WAN interface is found
fi
echo ""
echo "Performing Tests:"
echo -e "QoS Status: ${qos_check_result}"
perform_ping_test # Perform ping test with symbols
check_network_usage # Check network usage with symbols
check_cpu_usage # Check CPU usage with symbols
# Evaluate conditions based on defined thresholds:
local favorable_conditions=true
if (( $(echo "$avg_latency < 50" | bc -l) )) && \
[[ "$packet_loss" == *% ]] && \
(( $(echo "${packet_loss/\%/} < 1" | bc -l) )) && \
(( $(echo "$network_usage < 10" | bc -l) )) && \
(( $(echo "$cpu_usage < 20" | bc -l) )); then
result_message="Favorable conditions for tests"
else
result_message="Unfavorable conditions for tests"
favorable_conditions=false
fi
echo "" # Add a blank line before the final evaluation output
# Output results with symbols indicating success or failure
if [ "$favorable_conditions" = true ]; then
echo -e "${GREEN}${result_message} ${CHECK_MARK}${NC}"
else
echo -e "${RED}${result_message} ${CROSS_MARK}${NC}"
echo "Current conditions:"
echo "- Average latency: ${avg_latency} ms"
echo "- Packet loss: $packet_loss"
echo "- Network usage: ${network_usage} Kbps"
echo "- CPU usage: ${cpu_usage}%"
echo "- WAN interface: $wan_interface"
fi
}
# Execute the main function to start the script's operations.
main
# EOF
don´t know if this is suitable or hopeless overdosed and even not enough to check ?
There is no way to guarantee avoiding all noise in the results. That is up to the user when they click the “Go” button. At least that is how the three links above do it. They warn the user to test during quiet network times.
For sure you are right it is always up to the user to click something but by collecting automated data it can also be up to a, let´s say good time/circumstance for the test, above all, if you go for a automated configuration of something like QoS (if you meant this) and speedtest.net but also cloudflare can nothing do about this without making remarks about the needed circumstances but may a local script can ??? But this is also a little OT sorry to the topic creator.
But also in this topic it might be an idea to check line down and up speed and all that if the line is not overdosed with other traffic to simply prevent wrong values, may this is also a problem of the different results in different discussions by using the speedtest´s no matter from which repo they are from. An activated QoS for example can mix up the results sustainable, but not only, there is more which can. So even a simple speedtest needs some good conditions or prerequisites for adequate results, this is beneath the collection of useful data the main thing i´am currently on questioning.
Best,
Erik
P.S. Am trying something with speedtest and mtr by combinating the results but am a little on with the Jitter topic which sivels package does not deliver but iPerf or librespeed which has been already build for IPFire, lets see. The ookla cli is closed source!
I did not mean this, but it’s a good idea! My suggestion was simply to have a “bufferbloat speedtest” button on the QoS page that simply gets results and lets the user use that to configure their QoS. But if it also gave QoS configuration suggestions (as I think you are suggesting), that would be pretty cool. As long as it was a suggestion only, with an option for the user to accept or reject the proposed settings changes.
Yes, there’s a reason: ipFire default install should be minimal to reduce the attack surface and maintenance cost. If you choose to install addons, that’s your responsibility.
You could argue that there should be a stage in the install which suggests installing addons, but that’s a different discussion.
Thanks for all of the feedback. Attached is an updated zip file including suggestions from @siosios to include start button and @bonnietwin to use --secure. If you don’t have speedtest-cli installed, the install script will prompt you to agree to install it. The revision also grays out the start button after starting speedtest-cli so it won’t run multiple copies.
Attached is a package built from IPFire lfs with the speedtest-webgui page and menu entry. I included language files with translations done from web resources so I’m not sure how good the translations are. If there’s anything glaring in the language that should be corrected please let me know.
The package version doesn’t do anything with speedtest-cli except warn if it’s not installed. The page checks if another copy of speedtest-cli is running on the system and messages “try again later” if that’s the case. The Start button gets grayed out when clicked so multiple copies aren’t started and the Clear button resets the page if you want to run another Speedtest.
I’ve tested both the install.sh and uninstall.sh and both have worked for me.
To install
Copy the package to /opt/pakfire/tmp.
Unzip the package.
tar -xvf speedtest-webgui-0.7.0-1.ipfire.tar.gz
Run the installer.
./install.sh
I made a minor update to the speedtest-webgui package. The change is that the ‘Clear’ button is now greyed out while speedtest is running. This fixes the problem of hitting the ‘Clear’ button and reloading the page while a speedtest is running which would detach the running speedtest from the web page and no results would be reported. The installation is the same as before.
Copy the package to /opt/pakfire/tmp.
Unzip the package.
tar -xvf speedtest-webgui-0.7.1-2.ipfire.tar.gz
Run the installer.
./install.sh
.
Best,
