Right, the CPython runtime is first trying to be correct (IMHO is also correct). Very few optimizations are made by the compiler/interpreter. This has long been known by Python devs, and exploited by specifically using APIs known to be written in C/C++. e.g. using 'map' instead of a list comprehension. Map was generally faster (at least in the 2.x days) than an equivalent list comprehension.
Just FYI this specific wisdom you're suggesting is exactly backwards, or at least could be depending how how you read it.
timeit.timeit(setup='x = range(10000); l = lambda n: n + 1', stmt='map(l, x)', number=1000)
provides approximately 1.2 seconds to do that. The same with a listcomp:
timeit.timeit(setup='x = range(10000)', stmt='[n+1 for n in x]', number=1000)
runs in a half second.
Although dropping the inlined function runs slower still (1.6s):
timeit.timeit(setup='x = range(10000); l = lambda n: n + 1', stmt='[l(n) for n in x]', number=1000)
The second is the fastest because you drop the overhead of the function call, which does stack pushes and pops, and instead do them locally. The last is the slowest because it does the stack pushes and pops, as well as loads extra globals, which is slow.
On 3.6 the map version runs in 0.00067 seconds, because it doesn't actually do anything, it just constructs a generator object.
The general point is well taken. My favorite bit of trivia like this is the fast matrix transpose: `zip(*matrix)`, which does everything in pure c, and is also likely the shortest way to do the transpose in python.
> Just FYI this specific wisdom you're suggesting is exactly backwards, or at least could be depending how how you read it.
Yeah. map is usually faster if you already have a function to invoke (especially if that function is a builtin or in C). If you have an expression and have to wrap it in a lambda to get it in map, it's going to be way slower than the listcomp because all things considered CPython's function calls are expensive.
reply