It’s more important to prioritize single-threaded performance because it’s much harder to improve by throwing money at the problem.
With multithreaded performance, you can just add another core to (more than) offset whatever overheads there are from using process-based parallelism.
I think that this entire GIL vs No-GIL dichotomy is misguided. The biggest problem people have with multiprocessing is that you can’t share memory. So add virtual processes with an explicit mechanism for memory sharing. Then you can keep all of your single-threaded optimizations like refcounting without barriers because the objects for one thread will stay in that thread.
I think the general priority has been to improve single thread performance and for good reasons.
While there are plenty of problems that are parallelizable, a great majority of problems are sequential. Even if you think of say on the OS level - of running every process on a separate CPU/core/whatever - the single thread performance still comes out being the most important factor
Also, don't forget that threads don't scale beyond one machine; so when you want expand beyond one machine you need to take care of two levels of scaling, threads and processes. This can be avoided when using multiple processes that communicate or share state some other way in the first place.
Even with one CPU, with many cores the internal synchronization that happens to "simulate" a shared memory space can be expensive, if you're not careful your cores get clogged ping-ponging memory pages between each other.
It might sound more convenient to use threads instead of processes, but in the end I'm not sure that all the work to remove the GIL (and introduce shitloads of finer-grained synchronization primitives) is worth it.
Yeah, I'd agree it's kind of stupid to use OS threads if you are going to have a GIL. It does make the implementation simpler, but it comes at tremendous cost to IO bound programs. If you are actually trying to do computationally intensive work, you should really be using multiple processes instead of threads in a language with GC. When writing a UI, moving work to a separate thread can still lag because GC will also block the UI thread. Even if you don't care about latency, having separate memory pools via separate processes often helps GC because then GC is embarrassingly parallel regardless of GC implementation.
I agree with this. I was talking to a someone recently who works on embedded system. They do quite a bit of threaded code to deal with network, UI, and other aspects. I mentioned I prefer message passing and processes to threads. His response was that in the limited resources of their devices, that was not feasible. Shared memory and a single process saved on very valuable limited memory.
I mainly work on server side code where for the most part, the overhead of a separate process is not issue. The overhead of not sharing memory is not an issue.
Significant but as it's freethreaded you only run one large process. That alone allows for all kinds of additional optimizations that would be pointless with a lot of little singlethreaded processes in which sharing anything mutable between them has massive overhead.
I generally prefer to code one active thread per process, with multiple processes when I want parallel work. They communicate through shared memory, generally single-writer enforced by the OS.
So, yes, there can be equally rigorous system-level disciplines that substitute for language-imposed ones, and that can offer much better performance. But it does take a lot of experience to choose well.
I generally agree with this, but it neglects the existence of the class of problems that are CPU bound but don’t just split up neatly into separate and independent workers. For such problems trying to use multiple processes can be considerably harder (and far less performant) than just using multiple threads.
> Single-threaded performance is a toss up and depends on work load.
I would actually say the exact opposite is true. Single threaded performance is much more reliable and every single application can use it. Multithreaded performance is much more workload dependent, and there are many applications that can’t fully utilize it.
Depends on the program. If it requires, say, to synchronize hundreds of thousands of entities every 16ms, it's probably better to go with single threaded instead...
Because people still buy new computers and use those applications on them – that's why single-thread performance is (for the moment at least) still important.
I do understand where you're coming from, but real-world performance is important, especially when that world is imperfect.
If your workloads are limited by single thread performance you need better software. It's why vulkan and dx12 are a thing. (The single thread limitation of committing a frame to the GPU has been reduced by an order of magnitude) It's why C++ has the parallel algorithms library baked into the language.
I get it, threading is hard. But it's honestly not that hard. It's only hard when you're maintaining some super old program with single threadedness engineered into its core architecture. (note: this is my day job) Greenfield applications since 2009 should have had threading built in as a core assumption.
AMD is doing the right thing by optimizing for multithread performance over single thread performance. Moore's Law is dead for single cores. It has been for a decade and a half.
Won’t CPU-heavy tasks still make everything choke because of the GIL? You want to send CPU-heavy tasks to a different process instead of a different thread, don’t you?
Having multiple threads does not mean that they are all doing equally useful work. Single threaded performance is absolutely critical for a desktop machine.
Even in multithreaded desktop applications, it's rare to see them effectively use more than 8 threads.
If we were arguing about designing vehicle safety testing suites for the worst performers (a very real problem that we have right now) we wouldn’t even be having this conversation.
Writing multithreaded applications increases the performance ceiling. If an application can’t take use of multiple threads, but is written in a multi-threaded way, there’s no harm done. It simply runs the multi threaded code in a single threaded way (think of ParArray) with a bit of overhead incurred for “becoming multithreaded”.
Reasoning out of adding multithreaded support for long running actions because “most systems can’t take use of the extra threads” is just irrational, especially since most modern commodity systems could have a linear improvement with the additional threads.
The single core systems are barely hurt by the memory overhead involved with provisioning CORE_NUM of worker threads. But the multi core systems can take massive advantages from it.
With multithreaded performance, you can just add another core to (more than) offset whatever overheads there are from using process-based parallelism.
I think that this entire GIL vs No-GIL dichotomy is misguided. The biggest problem people have with multiprocessing is that you can’t share memory. So add virtual processes with an explicit mechanism for memory sharing. Then you can keep all of your single-threaded optimizations like refcounting without barriers because the objects for one thread will stay in that thread.
reply