So trying to extract 110% performance from today when it
means having only 70% performance available tomorrow is a
bad deal. You end up with just 77% of your available peak.
Uh, not quite: x(1.1) + x(0.7) != 0.77(2x). Reminds me of the fuzzy math in DHH's startup revenue calculations...
That's not what the article is about - but try scaling it back to 50% and see if it results in 50% drop. The graphs show that the relationship is probably very nonlinear
I would advise against concluding anything with < 20% gain, the changes often impacts readability (the intention becomes less clear) and might as well be measurement errors or just be insignificant for any sort of real application.
Not to mention, of course, these measurements are specific to a given system, implementation, etc.
This is a really neat article. One thing: the author falls victim to a common, unfortunate mistake in calculating the percentage gains: ...120 cycles. That’s a 30-percent speed-up. and then ...98 cycles. Compared to the original code, that’s 60 percent faster.
The right way to calculate this figure is (t1 - t0)/t0, rather than the author's formula which seems to be (t1 - t0)/t1. For instance: (157 - 98)/98 = 60%, but the actual amount is (157 - 98)/157 = a 38% speed up. A heuristic: 60% of 157 will be much more than 60 (since 60% of 100 = 60), which means a 60% speed up would reduce the speed to below 97 cycles.
It gets even more misleading the more efficient it gets: Adding up the cycles, the total was just 1689. I had shaved almost 1200 cycles off of my friend’s code. That’s a 70 percent speed-up! The author has 1200/1689 = 71%, but the correct numbers yield 1200/(1689+1200) = 42%.
Not that I don't think these are significant gains, but it's just misleading to label them like this. If you've removed less than half the cycles, there's no way you've seen a 70% speed up.
Actually, 25% is the right way to judge this improvement. For example, let's say performance was currently at 99.9% and you improve it to 99.99%. That's not a 0.09% improvement (99.99 - 99.9), but rather a ten-fold improvement (.01% errors vs 0.1% errors).
This has to do with the fact that accuracy/errors are not linear.
You should maximize the geometric EV, not the (usual) arithmetic EV. In your average calculation you took the arithmetic mean, but the geometric mean is relevant.
(2.25 * 0.9 * 0.9 * 0.36)^(1/4)
=0.9
Which is below 1!
To test the geometric mean with the earlier example ("+100% is equivalent to -50%"):
reply