I think the rock-paper-scissors dance between throughput, latency, and efficancy that a GC plays is a pretty facinating topic. It really brings out the our biases from the kinds of work we do.
If a service has reduced it's latency that only increases the throughput the overall system. CPU/Memory efficiency of a single process often much less important because there's often headroom and
GC can help leverage underutilized that cpu/memory.
The GC is low latency, but its throughput isn’t great. The key is avoiding the heap by mostly brute forcing trivial data structures on the stack (which is why you see so many repetitive O(n) loops).
I think people generally assume systems programming requires low latency and that using a GC implies high latency. That's mostly, but not strictly, true. If you're writing a web server, as opposed to a missile guidance system, the occasional GC pause is probably not a dealbreaker.
Very good point. Performance folds in latency (determinism) and throughput. There is usually a trade-off between the two. GC might handle throughput reasonably, latency is a bit tougher. You are sort of left tweaking knobs on a black box hoping to get good results in the end.
Also, throughput. But latency and throughput are almost universally opposite ends of the same axis — that’s why it’s great that Java allows for choosing a GC implementation.
It's perfectly passable for web serving and other such high-latency tasks. In a GC, the goals of throughput and latency are diametrically opposed -- optimizing for one makes the other worse. The present simplistic GC is a typical throughput-oriented design.
You didn't have a bad experience with GC in the past, you had a bad experience with a single GC implementation, one which was almost certainly optimized for throughput and not latency and in a language that pushes you toward GC pressure by default. :)
GC is more than just pauses. Throughput matters too; in fact, it often matters more than latency (for example, when writing batch jobs like compilers).
GC is a memory management technique with tradeoffs like all the others.
GC has many different implementations, with widely ranging properties. For example, the JVM itself currently supports at least 3 different GC implementations. There are also different types of GC's, so for example in a generational garbage collection system you'll typically see two or three generations of GCs, depending on the generation (how many GC cycles it has survived) of the objects it collects. The shortest GC's in those systems are usually a couple milliseconds, while the longest ones can be many seconds.
GC isn't always a problem. If your application isn't latency sensitive, it's not a big deal. Though if you tune your network timeouts to be too low, even something that is not really latency sensitive can have trouble because of GC causing network connections to timeout. Even if it is a latency sensitive applicatoin, if GC "stop the world" pauses - pauses that stop program execution, are short it can be OK.
One reason you'll see people say GCs are bad is for those latency sensitive applications. For example, I previously worked on distributed datastores where low latency responses were critical. If our 99th percentile response times jumped over say 250ms, that would result in customers calling our support line in massive numbers. These datastores ran on the JVM, where at the time G1GC was the state of the art low-latency GC. If the systems were overloaded or had badly tuned GC parameters, GC times could easily spike into the seconds range.
Other considerations are GC throughput and CPU usage. GC systems can use a lot of CPU. That's often the tradeoff you'll see for these low-latency GC implementations. GC's also can put a cap on memory throughput. How much memory can the GC implementation examine with how much CPU usage with what amount of stop-the-world time tends to be the nature of the question.
Isn’t one of the benefits of RC better (lower and more predictable) latency than with GC? Basically, you can presict when (and how much) garbage will be collected, and if it turns out that’s too much, you can fix your code... good luck doingthat with GC.
The downside is, of course, that it requires much more care (to avoid cycles)
Yeah, I'm being unfair in naming Go & Java specifically. But these stories of 'fixing' garbage collection come up all too often.
I wonder when we'll see a further GC update that trades latency for throughput...
The problem seems to be that no matter how you tweak GC, you will always have a class of program that it performs terribly for (and it seems to impact a large group of programs, never just some obscure corner case). So I suspect that this latest GC tweak will have unexpected results on some other class of program, leading to another tweak, and so on...
You don't really have a choice in the matter. If you're writing high throughput or low latency applications you are dependent on the JVM's GC behavior, period.
Digression here, but I like the "(nearly)". This bit always amuses me about garbage collection wonks. The pauseless bit is a real time requirement. Saying your GC has great latencies or is "(nearly) pauseless" is tantamount to telling a real time engineer your system only fails some of the time. It makes you look dumb.
GC is great. GC makes a ton of things simpler. GC as implemented in popular environments still sucks for real time use.
Often such flak ignores the differences between throughput and latency.
For long, lived processes you’ll end up writing some kind of garbage collection system.
reply