I agree. More work on latency, pre compilation and PackageCompiler is sorely needed. Julia is also behind on the version of LLVM that is being used, and apparently they are regressions in performance with newer versions. The fact that performance is an implementation detail is concerning, and it would be nice to standardize some of the language.
However, I think it is unfair to compare the inclusion of log10 to the work required to get solutions to these harder problems. I guess why not both? ;)
Julia is incredibly fascinating for me as a language to observe, although I haven't got the time/patience to try it out yet. Its language design seems incredibly progressive and yet pragmatic: they've repopularized dynamic dispatch (which makes perfect sense in a mathematics-oriented language), and has a unique mix of dynamic and static typing that seems perfect in paper.
In practice, Julia seems to suffer from the initial architectural decisions that the creators has made when the language was young - the biggest which is its heavy dependence on LLVM. This is a problem since LLVM is mainly used for static compilation of code for languages like C++/Rust/Zig and not with dynamic execution - its included JIT is famous for being unreasonably slow. Even the dev seems to know this but they don't seem to have that much choice - since too much of their codebase is dependent on it. (For example: https://discourse.julialang.org/t/jeff-bezanson-remarks-on-l... - LLVM codegen keeps getting slower, which in turn cancels out all the little optimizations they can do for the language) It's a language that seems so perfect on paper but suffers from its real-world implementation - and therefore captivates me in a special way. Some major architectural overhaul might be a much better move (such as a fallback bytecode interpreter independent from LLVM) - but it's definitely going to be a major effort.
I should also note that the internal APIs are not the problem for Julia. We usually pick those up immediately. Rather, doing validation for a new release on all platforms, reducing, filing, fixing any regressions, etc is what takes the time (in addition to only a few of us knowing LLVM well enough to do so).
No, that has nothing to do with any of the problems julia faces. Julia's LLVM is just regular LLVM with some patches that are usually upstreamed. The Julia team is pretty good at keeping up with upstream LLVM.
I am fully invested in Julia but I have found it's far from being productive for large scale projects. It's hard to encode interfaces and invariants (in practice this is done by documentation instead of code) and code is not very self documenting.
I expect it to get there over the next years though. And it otherwise his a perfect sweet spot for us.
Julia is a dynamic language, I don't think having the rigor of a compiled language (probably the most rigorous out there) is a reasonable expectation. It would be nice to have as an external linter service, probably.
Compelling argument in your link about how performance for others allows for better libraries for everyone. I am still concerned that Julia does not go far enough in breaking the imperative programming mould, but I think it's disingenuous not to give it a serious try in more than trivial exercises. I have to say though that I have learned R, Python and Golang in the past 8 years and all of them are basically imperative (though R's vectorisation-everywhere is impressive - wish it was all just faster); I hope Julia will give me something dramatically more interesting. I say that because the LLVM has ushered in a period of radically easier language development, so we are likely to be spoiled for choice in the next 5 years. I hope Julia has done enough to put itself way out there in in terms of innovation to make the sizeable investment of time for myself and library developers, worthwhile. Altogether however I cannot be anything other than impressed with the dogged and convincing pitch that you and others are making for it, which somewhat lowers the risk of investing time in a dead end. And even if it doesn't work out all hunky, at least I'll know that Python will face serious competition, and that can only be good, even for Python.
Yes, because it's all set for prime time in the next release.
"Julia features optional typing, multiple dispatch, and good performance, achieved using type inference and just-in-time (JIT) compilation (and optional ahead-of-time compilation), implemented using LLVM."
So it'll be updated when v1.9 comes out? Anyways, it's a somewhat interesting thing that Julia still gets 3rd even though it's measuring compilation time.
The REPL-based workflow and the compile time latency just need to go.
Stop defending it. Its a problem. It harms adoption. Fix it and move on and you'll stop hearing endless complaints about it. Sometimes the endless complaints have a point and you need to listen to it.
Coming from a decade-long history with ruby I do think that dynamic, duck-typed languages are inherently flawed, but c'est la vie and Julia is miles better than Matlab. I'm not looking at Julia to be exactly like go or rust, I want automatic differentiation. Don't particularly care about the memory bloat either.
Newbie documentation does need to be a whole lot better. I suspect that would probably get fixed if the compilation and REPL-centric issues got fixed and it wasn't so immediately hostile to your time on day 1.
Package management also needs to get better. Rubygems gets the shape mostly right if you don't look too close at the details, cargo seems to be excellent. Languages created in the past 10 years really should have strong package management from the start and not treat it as an afterthought and not try to reinvent the wheel without understand the successful systems that have come before them.
But really its about the compilation and REPL-centric issues. That should have been a requirement for 1.0, and I can't believe that a language that is nearly 10 years old could have that bad of an initial workflow for users.
I agree. However, I wish more people valued solid foundations over tooling because no tooling is worth building on "good-enough" basis. If that ever was true we would have never had the situation when we desperately seek for alternatives and create numerous solutions each coming with its baggage of gotchas like "yeah, but you need to use this PackageCompiler library to make it faster".
So after reading the release notes about 6 sec. of start-up times and people keep complaining about warm-ups, I am smirking about how Julia is a dynamic language when compiling & executing a script written in a static language other than let's say Scala/Rust/C++ can be comparable or even faster in some situations.
It's really hard to do, because having a command-line centered development experience with Julia is a complete nonstarter until compile time latency is near zero. And that's just really, really hard, and unlikely to happen, at least any time soon.
The reason people have been saying it's gettting better for 3 years is because it has been. PackageCompiler 1.0 was released in 2020 which made it possible to distribute Julia programs as self contained apps, Julia 1.6 released in 2021 added parallel precompilation which made loading a lot faster. Julia 1.8 was released in 2022 which improved precompilation a bunch, and Julia 1.9 will be released in 1 to 2 weeks and makes precompilation cache native code which significantly improves things again.
Deployment is a fundamentally hard problem for dynamically typed languages. Shipping a Julia .so will probably never be as easy as shipping a .jar file in java. However, Julia has gotten a lot more deployable over the past 3 or so years and work on that front continues. Julia 1.10 already has a bunch of compiler speedups that make things a bunch faster than 1.9 (I expect 1.10 to ship late 2023 or early 2024)
As someone who has used Julia for production use, I basically agree with everything you've said.
IMO:
1) Julia rushed to v1.0, but its understandable - it is sort of a catch-22 issue. More developers won't join unless it is stable (v1.0). But it won't get stable unless a lot of developers use it and feedback is heard.
2) I personally found that Julia community is slightly hostile to feedback and negativity. May be it is just me but it has way too much hype-driven-positivity that leads to delusion.
3) A lot happened between v0.2 to v1.0 which IMO should have been carefully and slowly done.
4) Developer experience should have been one of the major focuses. Stack traces should be beautiful and absolutely transparent. Debuggers are clunky - I used the Atom (Juno) debugger and wanted to toss the laptop off the window - just shows how frustrating it was. Compile times and smoothness of dev experience pays dividends and it was largely ignored.
5) I really like official libraries, not SomeBasicFunctionality.jl dependency. I am spoiled by Python.
6) Marketing around speed is misplaced IMO. It lures people like me into fanboys of Julia. There is so much more to a programming language than speed.
I don't want to be overly negative. The language and the original paper is beautiful - and this is all hard stuff. Kudos for the progress made so far.
I've tried julia a handful of times. IMO, the thing slowing adoption is that the usecases where julia feels like the most powerful, optimal choice are too limited. For example
- Slow startup times (e.g., time-to-first-plot) kill it's a appeal for scripting. For a long time, one got told that the "correct" way to use julia was in a notebook. Outside of that, nobody wanted to hear your complaints.
- Garbage collection kills it's appeal for realtime applications.
- The potential for new code paths to trigger JIT compilation presents similar issues for domains that care about latency. Yes, I know there is supposedly static compilation for julia, but as you can read in other comments here, that's still a half baked, brittle feature.
The second two points mean I still have the same two language problem I had with c++ and python. I'm still going to write my robotics algorithms in c++, so julia just becomes a glue language; but there's nothing that makes it more compelling that python for that use. This is especially true when you consider the sub-par tooling. For example, the lsp is written julia itself, so it suffers the same usability problems as TTFP : you won't start getting autocompletions for several minutes after opening a file. It is also insanely memory hungry to the extent that it's basically unusable on a laptop with 8gb of ram (on the other hand, I have no problem with clangd). Similarly, auto-formatting a 40 line file takes 5 seconds. The debugging and stacktrace story is similarly frustrating.
When you take all of this together, julia just doesn't seem worth it outside of very specific uses, e.g., long running large scale simulations where startup time is amortized away and aggregate throughput is more important than P99 latency.
This article is painfully true. Julia is full of gems, but also features so many warts everywhere that I'm not sure whether it will be able to overcome the latter – I hope it will.
I would add to the author's list the subtle differences between REPL and scripts, the weird obsession of Julia for embedding slightly different and out of date versions of its dependencies (BLAS, uv, LLVM, ...) that makes it a nightmare to package, the painfully slow documentation website,
the sometimes surprising function names, and wholeheartedly agree on the failure that is functional programming handling, both in syntax and implementation – which is surprising for a language stemming from MIT.
I'd like to use Julia for audio and image processing. The language and library support is well suited for my applications. However, the current state of the PackageManager is reason for Julia to be a non-starter for me. I looked at the linked release notes with hope that PackageManager improvements would be slated for 1.9. The startup time for standalone executables is unacceptably large and baffling given the computing efforts required to compile and package them. I reached for Julia to improve upon Python and the start up costs far exceed the available system time for gains. The mere existence of the acronym TTFP indicates that the Julia community has neither an appropriate frame of reference nor prioritization of developing Julia into a truly general programming language -- Julia remains far from production ready. Computing generality goes far beyond the habitual use of: input, analysis, plot. Swift and Rust ought to be benchmarks for executable startup. From my perspective, Julia runtimes ought to be able to be constructed and frozen in dynamic shared libraries such that startup can approach that of Swift and Rust. Julia developers perhaps can look to GraalVM for inspiration. A long wait for Julia 2.0, particularly if standalone executable compilation would be addressed, can not possibly help the adoption of Julia from my perspective.
Julia is very good, the only problem is the requirement for patched LLVM (patches they provide are not yet merged in the upstream), which can cause the conflicts with other frameworks if there is no separation.
However, I think it is unfair to compare the inclusion of log10 to the work required to get solutions to these harder problems. I guess why not both? ;)
reply