New Speedtest Addon Wiki Page!

jon · 30 November 2019 22:32

I turned off the IPFire QOS (menu Service > Quality of Service) and gave things a try.

This is the speedtest addon running on my IPFire box:
ping: 17.541 ms
Download: 181.33 Mbit/s
Upload: 12.07 Mbit/s

This is the speedtest-cli running on the console on my Mac:
ping: 17.015 ms
Download: 182.24 Mbit/s
Upload: 12.15 Mbit/s

I’m not a Vivaldi user but the test results for Safari running on my Mac:
ping: 11 ms
Download: 239.56 Mbit/s
Upload: 12.01 Mbit/s

And here is the Speedtest app loaded on my Mac:
ping: 11 ms
Download: 241 Mbit/s
Upload: 12.1 Mbit/s

All tests are to the same server.

So I see similar odd results and they seem to point to Speedtest-cli. But I do not know why. Maybe the author could provide more clues than me. https://github.com/sivel/speedtest-cli

yoda · 30 November 2019 22:42

Just an FYI, it appears that speedtest-cli cannot really be relied upon for accurate results…from: https://github.com/sivel/speedtest-cli

" There is the potential for this tool to report results inconsistent with Speedtest.net. There are several concepts to be aware of that factor into the potential inconsistency:

1. Speedtest.net has migrated to using pure socket tests instead of HTTP based tests
2. This application is written in Python
3. Different versions of Python will execute certain parts of the code faster than others
4. CPU and Memory capacity and speed will play a large part in inconsistency between Speedtest.net and even other machines on the same network

Issues relating to inconsistencies will be closed as wontfix and without additional reason or context."

jon · 30 November 2019 23:21

Good find!

ms · 1 December 2019 20:41

That is obviously a normal problem that you have when benchmarking something.

So this speedtest is just as reliable as any other ones. But they are reproducible, easy to run and give a good idea about where you are ending up.

hellfire · 2 December 2019 17:05

So is speedetest limited by IPfire QoS or not?

Edit: it seems so, at least a test revealed this. Question: is there a cmd to stop QoS while running the speedtest and restart it afterwards?

jon · 3 December 2019 00:02

Something besides the IPFire WUI menu Services > Quality of Service, then click Stop?

I see a command /usr/local/bin/qosctrl. I’ve not used it before so I really don’t know if this helps.

[root@ipfire ~]# /usr/local/bin/qosctrl 
No argument given.
qosctrl (start|stop|restart|status|generate)

EDIT: see https://forum.ipfire.org/viewtopic.php?f=27&t=20266

ms · 3 December 2019 11:21

Yes, that is the job of the QoS, to limit certain traffic.

hellfire · 3 December 2019 11:49

I know, Michael, however, when executing speedtest on IPFire’s console, I thought this would probably not be affected by QoS.

Anyway, I’ve already found that QoS interferes there, too…

Thx,
Michael

pike_it · 3 December 2019 12:10

So… two options:

disable QoS while testing
do not speed test

Latter is not that useful… Could a Qos rule override Qos management for speed test starting from IPFire?

hellfire · 3 December 2019 16:22

Idea behind is to create a script that stops QoS at night, run the speedtest and restart QoS afterwards.
The result in CSV format should be inserted into a database for monitoring.

I will see if this works, reliable, besides the mentioned measuring inaccuracy.

ms · 3 December 2019 17:33

Sorry, I am completely missing what you are trying to do here.

Why do you need to run the speedtest once every night?

pike_it · 3 December 2019 18:24

Benchmark the provider during the days/weeks. Most of ISP are in overbooking
And having some stats for understand/know the real performance, QoS can be adjusted and you can bug your ISP asking “Please improve my service”

jon · 3 December 2019 19:15

Hey! I just came across this while looking for something else. Ookla came out with their own Speedtest CLI. Much more better™ then the sivel version.

https://www.speedtest.net/apps/cli

Installed on my iMac:

iMac3:~ me$ speedtest -s 5900

   Speedtest by Ookla

     Server: aFastServer - Arlington Heights, IL (id = 5900)
        ISP: myFastISP
    Latency:    10.95 ms   (0.31 ms jitter)
   Download:   240.29 Mbps (data used: 223.9 MB)                               
     Upload:    11.98 Mbps (data used: 9.5 MB)                               
Packet Loss:     0.0%
 Result URL: https://www.speedtest.net/result/c/bb62347f-3562-47f5-8277-1234567890

Note 1: This is with QOS turned off.
Note 2: Be aware if you have the sivel speedtest installed. The sivel speedtest will continue to run instead of the Ookla speedtest.

EDIT: https://www.speedtest.net/insights/blog/introducing-speedtest-cli/

pike_it · 3 December 2019 19:19

Pro for Ookla: widely know, quite “acceptable” for non-tech people.
Pro for sivel: it’s independent (and maybe not twekable) by Ookla

jon · 3 December 2019 19:41

This is/was the article I was looking for:
I set up my Raspberry Pi to automatically tweet at Comcast Xfinity whenever my internet speeds drop significantly below what I pay for

https://twitter.com/a_comcast_user

After reading the above I started keeping track of Internet speeds. I was not looking for “blazing fast at all times!”. But when the speeds went low it made it easy to complain. (the breaks in the graph are when there was no Internet service)

hellfire · 4 December 2019 11:45

Jon, so you are using speedtest-cli as a base for this graph?

I plan to do the same as mentioned above.

jon · 4 December 2019 17:10

Yes. But running on a Raspberry PI and a 100 MB ethernet port. So the max download speed was near 92.

yoda · 6 December 2019 06:32

@ms Why did you remove from https://wiki.ipfire.org/addons/speedtest-cli the quote from speedtest-cli Github page that provide users with a heads up about the potential for results inconsistency? I think this is very relevant to anyone who uses speedtest-cli.

ms · 6 December 2019 07:17

Hi,

sorry for not leaving a comment on this (I need to add that to the wiki when we do a rollback).

I thought that it would be better to either link this discussion or reference it elsewhere. I do not think that we need to have an extra paragraph on the wiki for this, because I consider these speed tests always as unreliable. Benchmarks are always flawed in one way or another. I highly doubt that this is because of the Python implementation.

Speed tests are just some kind of indication about what has gone through your link at a certain point in time. They are not reproducible at all.

hellfire · 6 December 2019 07:36

I agree, however, for me, running IPFire in my home lab, speedtest-cli delivers enough information to get at least an impression what’s going on or if the speed decreases significantly over a a certain period of time.