I actually donated to the Pypy project in the past but I don’t use it.
Two reasons for my hesitation:
1) Cpython is fast enough for most things I need to do. The speed improvement from Pypy is either not enough or not necessary.
2) Lingering doubts about subtle incompatibility (in terms of library support) that I might have to spend hours getting to the bottom of.
I already work long hours and don’t have bandwidth to tinker. With Cpython, although slow, I can be assured is the standard surface that everyone targets, and I can google solutions for.
It’s the subtle things that i waste a lot of time on. It’s analogous to an Ubuntu user trying to use Red Hat. They’re both Linuxes but the way things are done are different enough that they trip you up.
The only way to get out of this quandary is for Pypy to be a first class citizen. Guido will never endorse this so this means a bunch of us will always have hesitation putting it into production systems.
The main reason I'm not even considering PyPy is that it lags behind CPython by a couple of versions. I'm getting ready to move to 3.7 while PyPy is still on 3.5.
I think that's because when new codebases are written, pypy isn't on the latest version of the Python spec, so people choose CPython. If it got a little more support it could get to parity or n-1, and far more projects would start with it and stick with it.
I don't use it myself, I just see the relatively small difference in investment it would be between a great project that's slightly languishing in obscurity despite incredible talent and effort, and a genuine full alternative to CPython.
After I ran the experimental evaluation, I had similar thoughts. If PyPy ever matches the current version of CPython I'm not sure why one wouldn't use PyPy over CPython. The biggest hurdle is matching support for popular libraries like NumPy, Tensorflow, Pandas, Scipy etc. I know they're working on supporting these, it's definitely a lot of work to do, easier said than done.
Not completely unconcerned, PSF did donate $10k to PyPy last month, so they do recognize some importance for speed, even if it isn't CPython's priority.
(and PyPy does break some python APIs and expectations, speed does come at a small cost, after all)
It's a great question. I'd say for myself I've been hesitant to use CPython or PyPy simply because their documentation seems focused on the extremely technical, rather than a person just trying it out for the first time.
I know Python, and I know C. But I'm worried ending up down rabbit holes in PyPy and competitors. I've not been able to find a really solid tutorial or parse the docs very well.
Perhaps it's just me, though. That's always possible. I just see a large barrier to adoption.
I want to use PyPy, and I have used it for a biggish server-based project I've been a part of.
The issue I've usually had with it is just small things – maybe an module we depend on doesn't work with PyPy yet, maybe PyPy3 isn't 3.3-compatible yet and can't use "yield from", which we use in our code or which our libraries use.
Dealing with those sort of small issues (and/or waiting for PyPy to fix them), worrying about whether the project I'm currently working on can use PyPy or if I need to use CPython instead, etc makes me stop worrying about trying to use it after a while. In my dev environment at least.
I do really love what the PyPy guys are doing though, and when it works it works damn well. Building it from source is also super pretty.
>I would certainly prefer it if PyPy was 100% compatible with the CPython C API even if it was at 80% (maybe even 60%) of the CPython C API speed because then I don't even have to think. I'd be using PyPy because it's faster overall and I can do the analyses I want faster.
While part of me agrees with this, if PyPy starts sacrificing performance for CPython compatibility then pretty soon it'll degenerate into CPython.
What would be the point of filing bugs against PyPy stating that it is slower or more resource-hungry on programs XYZ, when this is ultimately due to PyPy's core design decisions which are a point of pride and never going to be abandoned?
Expecting some positive outcome from that seems incredibly unrealistic to me. I believe it is a waste of time. It isn't my responsibility to provide reasons to PyPy why I'm not using it. Let PyPy - a project receiving no small amount of promotion - show that it is better for my purposes, before I invest big in switching over projects.
What you COULD say is that it doesn't make sense that in practice, people are treated like idiots and flamed if they publicly mention that they don't find in practice that PyPy is always or even generally better than CPython.
Or you could say that they should both work for most purposes, and that the choice is nuanced (measure the difference yourself), and in just a few respects PyPy is not as mature (not surprising given the lengths of the projects' histories). And PyPy is a work in progress and you expect it to get better if it isn't better than CPython now, for some specific purpose.
I don't expect you to say either of those things, because it seems important to the PyPy project to promote it over CPython and if that means selectively mentioning only the cases which are in PyPy's favor, or softly suppressing dissent, then so be it. That is how it seems to me, and I don't understand why it has to be that way.
We tried PyPy without much success (and we really tried, six month project just to get it to run our codebase.) Our workload is very heavy on the CPython C API, which makes life difficult for PyPy or any other alternative Python implementation. PyPy is great for a lot of uses, just not a great fit for us.
To extend what you said a bit: PyPy is not only slow with extensions, it's also massively slower for running short-lived code that isn't repeated much, which is precisely the scenario of many if not most command line tools written in Python. Although I'm not sure there are enough people concerned about Python's performance in this scenario.
And yeah, I would be happy to pitch in too if CPython can be 50% faster reliably. It only takes 500 people each pitching in $100 to reach $50k.
Edit: somehow got $50k etched in mind when it says $500k. That’s more difficult from individual funding.
I don't think PyPy has a chance at becoming the standard the way things are going now.
Their funding has ended, and I don't think they will manage to pony up another million+ in funding after 5 years. PyPy isn't compatible with C modules, and can be up to 5x as slow as Cpython (according to their own benchmarks). Making unladen-swallow the python standard will probably cause a brain-drain and make others hesitant to support PyPy.
I don't think speed is a big enough pain point for the python community at large to justify the pain of moving from cpython.
PyPy needs to use their great implementation stack to develop their own killer feature. Like for instance a small and fast python for embedded scripting or compiling normal python program to a small and optimized native executable.
Hey, don't get me wrong, it's great that you're making strides, and I still hope the project is a success.
My viewpoint comes as someone who was very excited initially at the thought of a drop-in replacement for CPython whose goal was to be "faster than c". It's been a very long time and it seems like pypy still has a long way to go to achieve production-readiness, so it's hard to continue being excited about the project.
I'm reminded of when I first started following the PyPy project, and I thought it would be impressive if they could ever get close to CPython's speed. They've done something truly amazing.
I hope the python community someday coalesces around a single version. I generally can't take advantage of all their awesome work because the libraries I depend on (e.g. pandas) won't run in pypy.
Two reasons for my hesitation:
1) Cpython is fast enough for most things I need to do. The speed improvement from Pypy is either not enough or not necessary.
2) Lingering doubts about subtle incompatibility (in terms of library support) that I might have to spend hours getting to the bottom of.
I already work long hours and don’t have bandwidth to tinker. With Cpython, although slow, I can be assured is the standard surface that everyone targets, and I can google solutions for.
It’s the subtle things that i waste a lot of time on. It’s analogous to an Ubuntu user trying to use Red Hat. They’re both Linuxes but the way things are done are different enough that they trip you up.
The only way to get out of this quandary is for Pypy to be a first class citizen. Guido will never endorse this so this means a bunch of us will always have hesitation putting it into production systems.
reply