That means (my 2 cents view) that probably the currently used test file it's good to test about mirror consistency and age, but a different file (bigger) should be used to test about speed.
thinking, that i fix it..., but how is bigger?
suppose the wrong result depends from some delay starting the download, say 1 seconds delay, to have a meaningful speed result we should use at least a 5 seconds download (allow for an error of approx. 20%)
The problem is that the time depends also from available internet connection speed:
On a low speed connection (less than 500 Kbit/s) I would try 250 Kb filesize,
On a medium speed (5 Mbit/s) => 2.5 Mb filesize,
On higher speed ( > 10 Mbit/s) => 5 Mb filesize or more.
Not a simple task ....