> It seems true to me that working with the Rust borrow checker is eventually fruitful, but there is a micro-case where working with lifetimes seems like a lot mental overhead with little gain, compared to a built-in garbage collector available in Lisp.
I fully agree. In the past year I have worked in Rust a lot and while quarrels with the borrow checker have definitely gotten more and more rare, they still happen occasionally. It's not that the borrow checker presents a reason which I think is incorrect or invalid, it's rather that it's sometimes difficult or just tedious to encode the proof that what you're doing is actually fine.
That said, there have been situations in which I was absolutely confident my code would have been fine if it weren't for the meddling borrow checker, but it actually wasn't and it helped me catch bugs.
Fortunately there is a bit of an escape hatch: if you want to, you can just start reference counting and get close to the experience you'd have with a proper GC. For example, there's nothing preventing you from implementing a reference counted linked list and writing
> It seems true to me that working with the Rust borrow checker is eventually fruitful, but there is a micro-case where working with lifetimes seems like a lot mental overhead with little gain, compared to a built-in garbage collector available in Lisp.
Fwiw, my shop uses Rust in some basic web applications and lifetimes are not used in day to day programming. It heavily depends on what you're doing. Are there some applications/libraries/etc where we're using lifetimes? Definitely. But for us they're very niche. AND, perhaps most importantly, even in those you _still_ don't have to. If you're okay with the performance trade off, which we most certainly are (we have some Python usage for example), adding in basic reference counting can be a no-brainer tradeoff.
I actually quite like lifetimes. I have some side projects where i use them extensively. But just like standard Generics they're a tool you can use, but that doesn't mean you always should. Just like accepting a `s: &str` instead of `s: S) where S: AsRef<str>` can make your life cleaner and simpler, using `Rc<Foo>` can equally simplify.
Just my 2c :)
[1]: because we like it, no other reason.. really, a team of ~30.
> The most fundamental issue is that the borrow checker forces a refactor at the most inconvenient times. Rust users consider this to be a positive, because it makes them "write good code", but the more time I spend with the language the more I doubt how much of this is true. Good code is written by iterating on an idea and trying things out, and while the borrow checker can force more iterations, that does not mean that this is a desirable way to write code. I've often found that being unable to just move on for now and solve my problem and fix it later was what was truly hurting my ability to write good code.
The latter part of this is true for any strongly statically typed language (with Rust expanding this to lifetimes), which negates the beginning of this paragraph -- once you get things compiled, you won't need to refactor, unless you are changing major parts of your interfaces. There are plenty of languages that do not have this problem because it is a design choice, hardly something Rust can "fix", it's what makes it Rust.
>You'll find yourself constantly fighting the borrow checker.
In the beginning, yes, this is true. But most people learn within a month or two which design patterns lead to problems with the borrow checker and which work smoothly, and often this knowledge translates to good design in languages like C and C++ as well.
If you're fighting the borrow checker in Rust, you'd probably have been fighting segfaults and use-after-free in C / C++. I'd rather spend 30 minutes fighting the borrow checker than spend 4 hours digging around in Valgrind.
> Type signatures are littered with lifetime annotations.
You cannot avoid the concept of lifetimes, without a garbage collector. If you don't want garbage collection, you have to deal with them.
Having explicit lifetime annotations in the code is _vastly_ better than trying to track the lifetimes in your head from scratch every time.
> Compared to Rust? I doubt that. Like I said, Rust lets you dance near the line.
Your problem is that you are trying to imagine what happens in the real world rather than just looking at it.
> This is from experience in various large C++ codebases. I'm not saying people use refcounting a lot, I'm saying it gets used more than Rust.
I count 56 imports of Arc and 37 of Rc in servo, would you wager the equivalent parts of chromium or firefox use more ref-counting?
> Not really. Aside from non-lexical borrows and a couple other nice-to-have things (but not necessary), the borrow checker is pretty precise for what it tries to prove.
The problem is that the borrow checker does not look at the function as a whole. You are a slave to the borrow checker, you can only consider other things when it has been satisfied, even if its trivially safe from a whole-function view. Some simple things aren't even expressible at all without non-lexical borrows, no matter what gymnastics you try.
> It sort of does. If you try to introduce borrows and listen to the suggestions the compiler gives you, and eventually end up nowhere, it's probably not possible to do it that way.
That sounds like a terrible workflow that I don't think Rust programmers use in the real world. I think they usually use shared ownership because they know they have shared ownership, not because they gave up on the borrow checker. Any instance of actually resorting to reference counting because of the inability to make the borrow checker happy actually counts against Rust, not for it.
> in practice IMO the things Rust makes you feel safe to do is a superset of what C++ lets you feel safe to do
People routinely do safe things in C++ that would be impossible to express in safe Rust code. You need to let go if this untruth.
> if we're going to talk about "all possible things C++ theoretically allows you to do", then you should include unsafe -- you were comparing "safe Rust" with "all C++", which is unfair here, since the entities to be compared for what you're theoretically allowed to do are "all Rust" and "all C++".
I focused on safe Rust because your argument was that safe Rust is just as powerful as C++, just proved safe statically. This is far from the truth and you refuse to accept the correction. Most people in the Rust community would probably say they accept the loss in power because they prefer absolute safety. This is a very defensive position. It's impressive just how little power Rust gives up in comparison to languages like C#/Java. But to pretend that you actually have more power in Rust due to psychological effects is just zealotry and doesn't reflect how people use C++ in the real world.
> Examples?
Any abstraction that uses dynamic checking that would be considered a bug to actually trigger. Some examples include borrowing a RefCell, random access of iterators, sequential access of certain kinds of iterator adaptors, etc.
> Personally i feel the concern over the borrow checker is way overblown. 95% of the time the borrow checker is very simple.
I have been using Rust professionally as well and had a different experience. For anything singlethreaded I agree with you. For any asynchronous behavior, whether it's threads or async or callbacks, the borrow checker gets very stubborn. As a response, people just Arc-Mutex everything, at which point you might as well have used a GC language and saved yourself the headache.
However, enums with pattern matching and the trait system is still unbeatable and I miss it in other languages.
> It breaks your intuition, and it constantly feels like the compiler is telling you that you’re writing code wrong.
I had the same experience with Rust, and you know what? It was right. My intuition was in fact wrong, and the code the borrow checker complained about would have lead to memory access violations in other languages. Rust catches it at compile time so my program doesn't fall over at runtime. If you think you know better than the compiler, good on you, more power to you. But in my experience I've found that trusting the borrow checker has lead to rock solid stable code that is performant and never crashes during runtime with segfaults and other symptoms of bad code.
> I have this intuition that the borrow checker might enforce a way of working that fundamentally stops you from doing things that aren't _really_ compatible with the way a CPU actually works on the lowest level.
No, I think you are reading too much into it. One is free to Box-allocate/pointer-chase on every line in Rust without any trouble. Sure, as other low level languages, rust also prefers/makes stack-allocation more ergonomic/the default, which is a quite cache-friendly way of operation but not even the stack is “native” in and of itself - it is just a frequently and extensively used part of the “heap”.
>You still have to fight with borrow checking in Rust as in C++, it’s just not automated.
It's not that easy. The borrow checker gives you memory safety and thread-safety. In C++, you always have to worry about memory safety, so here the ownership model and borrow checker is a pure win. However, in C++ you don't have to worry about thread safety in a single thread...there's no barrier to passing pointers around and mutating things from different places in your code. In rust, the borrowchecker disallows shared mutable state...that's a huge win if you want to parallelize your code later on, but it also means that things that are trivial in C++ take a lot of thought and effort in rust, especially if you're thinking in a C++ paradigm (as opposed to coming from a functional language).
> ... from a programmer's perspective, I would love to see a mix of compile-time lifetime analysis, Automatic Reference Counting (like in the lobster programming language), and manual memory management (marked as unsafe) for when extreme performance is needed.
Surprisingly, such a thing is actually possible. We can use borrow-checker-like static analysis layered on top of shared mutability (such as RC). It involves selectively making certain sub-graphs ("regions") immutable, thus getting the borrow checker's zero-cost memory safety, without the architectural restrictions the borrow checker normally imposes.
The core insight is that a borrow checker doesn't need every object to be completely affine, all it needs is some notion of what objects are reachable from what others.
This has recently been prototyped in the research languages from Forty2 ("capsules") [0] and from Milano, Turcotti, Myers [1].
Pony's "iso" [2] has shown that it can work remarkably well in production, and Vale [3] is making it work with single ownership.
In my (biased!) opinion, it's one of the most promising areas in language research. If we can get the speed and safety benefits of Rust while being more ergonomic and intuitive, I'd consider that a big step forward for the field. An exciting time to be in language design!
> After playing with rust, I really like it. However, I generally don't need to write the low level code that justifies the increased complexity of the borrow checker. I would rather just pay the small price for a garbage collector. What language is most like rust, but at a higher level?
I think you've mixed yourself up somewhere here. A garbage collector doesn't protect you from stale references, it just makes sure that it results in a program crash (or exception of some sort) rather than letting your program run off into fantasy lands where pigs fly and everyone casts magic spells. Rust still protects you by turning these errors into compile time errors, saving debugging time.
>The borrow-checker in Rust is kind of silly tool to use in the context of a managed language
I suspect a lot people are drawn to Rust more because of its type system and great tooling. I'd be perfectly happy with a version of Rust that swapped out the borrow checker for a GC, but such a language doesn't exist today, and Rust does exist.
> If you subscribe to this idea, then it sort of follows that Rust's borrow checker may be "in the wrong place". That is, rather than forcing you to write code that is memory safe in a particular statically verifiable way, Rust could have instead enforced memory safety by injecting run-time checks into the code and optimizing them out when it recognizes code that appeases the borrow checker.
That kind of lack of transparency about what in the hell your code is doing at runtime is really inappropriate in a system's language.
It's an interesting idea, and it would be neat to play with in a language that wanted to restrict itself to more business-logic level safety concerns, but it would absolutely come at the cost of not being appropriate for systems-level tasks.
> The static analysis of the borrow checker is not able to figure out when you're headed for trouble via this route.
This route exists purely because you can't statically prove some programs correct. Without it, the compiler would have to reject some correct programs and the language would be more limited.
This is unfair to complain about it when the GC-based alternatives push all of memory management to runtime, so the surface for getting panics is a lot larger. Rust's solution is not perfect but still the best you can get.
> Designing the data structures for something tightly coupled with concurrency, like a window manager or a game, is difficult.
It is difficult in any language. The difference is that in Rust you face that difficulty when trying to design/compile your program, and in other more permissive languages you face it when your customer reports a bug. The cost of fixing problems is lowest when they are detected early, so Rust has a clear edge here.
> I wish there were a simpler language, like Rust (no garbage collector, no runtime), but that was a little more helpful and willing to do implicit things even if they are slightly slower than the most optimised code possible.
The fiddling with borrows isn't about being as optimised as possible, it's about being correct in the absence of garbage collection. Are you sure you need no GC and no runtime? Tasks like the script you describe sound like a perfect fit for something like OCaml.
> The borrow checker does a whole lot more than replacing the GC though so it is very weird to point at it and say that the lack of a GC leads to that.
Well, yes and no. Sure, it helps with data races (but not with race conditions in general), but foremost it is a tool that allows for correct compile-time memory management. Compile-time memory management is only possible in a subset of programs, so rust as well has to use (A)RC at times. This is okay when used sparingly, but atomic increases are very expensive on today’s hardware.
I am familiar with RAII, but that is the exact same thing what Rust enforces at compile time, with the exact same shortcomings, so I don’t see how is it an argument against GC.
Reference counting can cause long pauses as well - as I said, it is the same problem, just looking at it from the other direction. If an object with many many references to plenty other objects die it can take quite a long time to deallocate, there are no free lunch. And then we haven’t even talked about cycles that need a similar background process to tracing GCs, without which it will leak.
While I am all for research into better ways to do RC, please look at the discussion - it is not at all clear that RC would be better, and even theoretically a tracing GC will win.
> If you're fighting the borrow checker in Rust, you'd probably have been fighting segfaults and use-after-free in C / C++.
That is in my ( admittedly limited) experience just not true. There's plenty of things that are perfectly safe that the borrow checker just doesn't understand.
The borrow checker can prove that a subset of things is safe. But the borrow checker being unable to prove something doesn't mean it's not safe.
> I'd hoped that language features like the typestate stuff that used to be in Rust would someday make the work required to use sound analysis tools in production code smaller. I'm not sure if much thought has been given to what kinds of accommodations languages could give to ease static analysis while still being programmer friendly.
Well, since the Rust borrow check is basically a static analysis that's always on and is required to pass for the compiler to emit code, we've put a lot of thought into how to make it as programmer friendly as possible. The final trick that seemed to got it to fall into place was restricting data to only one mutable reference at all times—this was a restriction that's easy enough to understand and can be readily explained through error messages and tutorials. There's still a learning curve, of course, but I think we've got the system down to a reasonable level.
> You still have to fight with borrow checking in Rust as in C++, it’s just not automated.
This is so misleading. Sure you still have to think about borrowing in C++ but in C++ you only have to convince yourself that you got it right. You don't have to convince a dumb child that barely understands variable scopes.
The benefit is of course that the dumb child is 100% reliable so if you can convince him then you are 100% sure you got it right...
But please can we stop pretending that it isn't a massive pain in the arse to convince the borrow checker that your code is ok.
Also as someone else pointed out, if you are only writing single threaded code the mutable borrowing rules are needlessly restrictive.
I like Rust but it really irks me when people pretend it is as easy as C++. It isn't. It is much harder - it's just that when you're done you have (probably) a better result.
> Most C++ programmers do not fear using references or pointers because of invalidation.
Compared to Rust? I doubt that. Like I said, Rust lets you dance near the line.
This is from experience in various large C++ codebases. I'm not saying people use refcounting a lot, I'm saying it gets used more than Rust.
YMMV though, so it does boil down to a matter of different experiences here. We'll probably have to agree to disagree.
> In fact it often has false positives requiring borrow gymnastics because it is too primitive.
Not really. Aside from non-lexical borrows and a couple other nice-to-have things (but not necessary), the borrow checker is pretty precise for what it tries to prove.
One might argue that the guarantees Rust tries to maintain (one writer or multiple readers for a piece of data) are too primitive. I don't think that's true. Doing a context-sensitive/flow-sensitive analysis might lead to more patterns being allowed but it's hard to scope guarantees when the analysis is interprocedural -- stopping at function boundaries makes sense to me.
> The Rust compiler does not tell you when it's impossible to satisfy your problem using borrows.
It sort of does. If you try to introduce borrows and listen to the suggestions the compiler gives you, and eventually end up nowhere, it's probably not possible to do it that way. It's not perfect, but it's good enough. And it's immune to further changes -- you don't need to design your pointer usage so that it's future-proof; design it however you want, and if a future refactoring introduces a possible use-after-free, fix the compile error.
> And your last line is a complete reversal of your previous attitude: Suddenly you are focusing on theory while ignoring that in practice you can not usually break out `unsafe` to solve your borrow problems.
No, it's not, I'm just pointing out the equivalence. My point was that "The things you can prove statically with the borrow checker are a subset of the things that wont trigger UB in C++." is irrelevant for two reasons -- (a) in practice IMO the things Rust makes you feel safe to do is a superset of what C++ lets you feel safe to do, and (b) if we're going to talk about "all possible things C++ theoretically allows you to do", then you should include unsafe -- you were comparing "safe Rust" with "all C++", which is unfair here, since the entities to be compared for what you're theoretically allowed to do are "all Rust" and "all C++". I focused on a different flaw in your argument, so of course the focus changed. I'm not saying you should use unsafe a lot, I'm saying "what C++ lets you do is equivalent to using unsafe a bit more often in Rust". I don't endorse it, but if you consider "C++ lets you do so many things Rust doesn't" to be a plus point of C++ (I don't), then you should at least be comparing the right things and allowing yourself to use unsafe in Rust too.
I fully agree. In the past year I have worked in Rust a lot and while quarrels with the borrow checker have definitely gotten more and more rare, they still happen occasionally. It's not that the borrow checker presents a reason which I think is incorrect or invalid, it's rather that it's sometimes difficult or just tedious to encode the proof that what you're doing is actually fine.
That said, there have been situations in which I was absolutely confident my code would have been fine if it weren't for the meddling borrow checker, but it actually wasn't and it helped me catch bugs.
Fortunately there is a bit of an escape hatch: if you want to, you can just start reference counting and get close to the experience you'd have with a proper GC. For example, there's nothing preventing you from implementing a reference counted linked list and writing
reply