All these wrappers are there because nixpkgs doesn't have non-awkward ways to deal with the stuff people deal with often. Nix the language is probably the least of the issues, it's maybe a few % of the chaotic Nix ecosystem. Most of the logic (that desperately needs proper documentation) is in nixpkgs and tooling.
There's lots of criticism about the Nix expression language, but it's way overblown IMO. The core language is small. It's essentially just JSON with functions, very similar to Jsonnet. But with Nix, people make too big of a deal out of implementation details like it being lazy, functional, etc. I personally like Nix as a configuration language because it's suitable for complex configurations despite it being small and simple.
There's a case to be made about improving the state of the Nixpkgs APIs though. Stabilizing the APIs and documenting it more thoroughly would dramatically change how people perceive the complexity of Nix. Experienced Nix users grep the Nixpkgs repo to learn about its APIs, but that's far from ideal for newcomers who won't even know where they should look for information.
Still, despite its reputation, Nix lowers the bar for software packaging. Package definitions tend to be more concise, customizable, and reusable than the competition. Nixpkgs contains many useful utilities for packaging and there's an abundance of real world examples contained in a single repo. I also frequently work with RPM, but Nix is much easier in comparison.
100% this. The language is designed for its use case which is packaging and configuration (nothing more or less). It has a learning curve due to being lazy and functional but works great once you get the hang of it. But the documentation of all its functions is so annoying. You have builtins and the nixpkgs functions[1]. There is learning the language, and then learning how to use it. Then there is the entire ecosystem of custom packaging functions that have their own pros/cons [2]. The issue isn’t with the language but the difficulty with trying to make existing tooling work the Nix way. That part is where I agree with the curse of nix. But the effort is worth it because once the packaging is complete it just works (forever).
Why do you think it is? The problem with Nix is not the language, it is mostly the ad hoc, grown over time “standard library” for packaging programs. It is just hard to know what that magic buildXPackage function does, or sometimes even where it is. You basically have to grep through the nixpkgs repo to find these things.
The language is very small and is a great fit for its domain.
I'm a nix fan and have been using nixos for years now, packaging stuff and writing my own nix-based libraries, but I still find the language to be painful.
The problem is, documentation is sparse to non-existant, error messages are poor and often far from the actual problem, and it makes use of concept that lots of people are not familiar with (like lazy evaluation) without ever explaining them, resulting in surprising semantics.
But as far as I can tell, the nix devs are fully aware of these issues - they are affected by it too! The problem is, solving each of those issues is hard, and requires a lot of focused efforts.
I've tried to use Nix on and off for years. I always run into issues where I need to write some package and
(1) I get mired in packaging low level transitive dependencies and things which seem like they should be easy end up being nearly impossible
(2) Basic things like figuring out the argument types for a particular nixpkgs function take an insanely long time. It always involves grepping around the whole repo (nixpkgs is huge for those who don't know) for invocations to find the variable that gets passed into that function and then grepping around the repo for the function that produces that variable and so on until you get to the source type. Things have awful names and documentation is sparse.
I'm a big believer in Nix as an abstract concept, but the execution has been a miserable experience for me to the extent that I can't make sense of people who report such positive experiences. I would think I'm doing something wrong, but so many people have corroborated my experiences and when I bring specific issues to experienced Nix users they also get stuck.
Ultimately, I've reluctantly gone back to dealing with Docker and system package managers because their failure modes are rarer, more predictable, and more easily worked around.
I think the problem is people are assuming that Nix is just one another tool.
In reality Nix is a programming language. It is a relatively simple language, but it is also functional and lazily evalated. And while the language is simple, its stdlib (nixpkgs) is actually puts another abstraction layer on top of it which is quite complex.
I do think though that Nix should do the same thing other programming languages do: identify what the nix stdlib is, move it to a separate repo with its own release schedule and ensure that whatever is added to it/removed from it is documented.
This won't help author of this blog, but it would be a great help for anyone who is willing to go through Nix Pills.
I agree, even with FP and (some) Haskell experience, customising Nix and debugging what's going on is made hard by Nixpkgs being a complex codebase more than anything. However, it is different than the usual large web app. The problem of packaging software is fundamentally easy to break down into tiny pieces, the package definitions themselves. Arch's PKGBUILDs were very easy to understand for me as a novice, much better than RPM SPECs or whatever Debian has, so I think it's not impossible for Nix-equivalent packages to be similarly simple.
The biggest barrier, as I see it, is not actually Nix or Nixpkgs code, but the way NixOS differs from every other Linux distro. Most of my delving into Nixpkgs has been motivated by getting something working that would be trivial in a traditional distro. I agree the Nix language is a red herring, what's really needed is to make work other than "adding a package to Nixpkgs" easier. Actually usable introspection tools that the GP mentioned would be great.
It's been a while since I used NixOS, but I never really had an issue with the language. What was frustrating was in package solutions there were all these special functions which as far as I can tell were undocumented and I couldn't figure out what they could do. So there was sort of this gap in terms of documentation between theoretical understanding and tooling/functional understanding which I struggled to cross.
The language itself isn't as challenging as the overall structure of NixOS/nixpkgs itself. It's a monolithic source-tree of interdependent "derivations". The entire notion of "package" has been thrown halfway out the window. There is logic and reason to the madness, but it's really difficult to find your bearings.
I understand the first complaint but not the second
> The second flaw is that NixOS does not actually provide real isolation.
I thought this would going to be about security, but then it was about having to repackage the world. But Nixpkgs is current the biggest and most up to date package repository in the world (we finally beat AUR). Packaging enough stuff is emphatically not the problem.
As a side benefit, autogenerating more Nix means less handwritten Nix! I do think people give the language itself to much grief when the problem is more the library side with the copious override shenanagins and other stuff.
I don't understand what's wrong with the Nix language. Sure, the space-delimited lists can get a little annoying, but other than that it's minimalistic enough to use a configuration language but complex enough to do more complicated things like overriding derivations. Maybe it's because I learnt Haskell before Nix so the syntax is very familiar to me, whereas I see how it could be a learning curve for other users.
What I find the most annoying is the lack of documentation of some of the nixpkgs functions. Some of the functions provided by import <nixpkgs> {}.lib appear to be the same as the ones builtin to the Nix language and there doesn't seem to be any clear guidance on when to use which version. I've also had to look at the source code to find out the difference between writeTextFile, writeText, writeTextDir, writeScript, and writeScriptBin. [The docs][1] explain writeTextFile, but the only documentation for the rest is ‘Many more commands wrap writeTextFile including writeText, writeTextDir, writeScript, and writeScriptBin. These are convenience functions over writeTextFile.’.
Additionally, it's a bit frustrating for me how all the documentation for nixpkgs — the lib functions, how to make a derivation, specific details for building packages in certain languages, how to contribute to nixpkgs, overriding packages/overlays etc — are in [one gigantic web page][2] that's quite slow to load and even slower to search for things in.
I think people just (understandably) have little patience for new DSLs in general.
Another thing that I'm realizing is awesome about Nix as I'm getting my feet wet with CUE (also an interesting configuration language) is that in a way, the power of Nix makes it easier to learn than it otherwise might be.
So much of what we actually do with Nix leverages functions and libraries that are written in Nixlang, stored in the same monorepo as all of our build recipes. This means that once you get more familiar with Nix-the-language from using it in your own configs, exploring the implementations of the tools you use in building your packages and environment becomes easier and more natural. And Nixpkgs is then an awesome one-stop-shop for countless examples, and even kind of a quick reference for how a bazillion different pieces of software can be configured.
Maybe it's an acquired taste, idk. But I feel like the language is great at being simple when it can and surprisingly flexible when it needs to be. And Nixpkgs becomes a great resource for users, once they learn some of that relatively simple language.
While I really love nix and nixos, I must say that Nixpkgs development is just out of control. There are over 5k issues and almost 4k PRs open. Reviewers are far fewer in number than contributors and there are multiple instances of low-quality packaging. Some features that most beginners want (for example, easily packaging nodejs programs) are still not there. Lack of standard is a heavy issue and there are many ways of doing the same.
There's an element of truth to the "poor documentation" complaint, because there are tutorials etc out there that are out of date, incomplete etc. But the official nix project has excellent documentation.
I think the problem is that nix is just hard to learn. It's really different from most package managers, but people expect to look at a few examples and just pick it up intuitively. It's a whole different paradigm, with a custom programming language that also has a different paradigm. (Though Haskell folks eat it right up... no surprise there.)
It's gonna take time and effort to learn it, and there's just no way around it.
Writing the packaging expressions should not be harder than writing the program, for the average developer. The average developer doesn't know FP, at this point. Nix needs:
1. Approachability for those not indoctrinated in lazy, declarative, functional programming (i.e. Haskell); nope, Nix Pills are not sufficient for the average dev
2. Editor tooling to help guide the writing of expressions (just as anyone learning C# or Rust can use LSP); a better type system for Nix would help here
3. Better documentation for practical things like "Using Nix to manage the dependencies and package a C++ program using Meson", rather than having people piece this together from a bunch of disparate docs
4. A much better CLI UX than `nix-env -qa` and the like (this is ongoing and experimental, but even that broke recently, causing lots of confusion; now it requires `nix --extra-experimental-features nix-command --extra-experimental-features flakes search nixpkgs`)
5. To seriously answer the question: is the Nix language required for the Nix packaging system to exist? Laziness is required, to some degree, but can the next iteration provide an on-ramp which doesn't involve learning a new lang and paradigm? Guix folks sure think so.
I feel like Nix folks have been focused so long on solving the tough problems of declarative, deterministic packaging that they haven't been able to focus on the UX. I also feel like folks for whom Haskell is comfortable may not realize just how absurd it feels to everyone else. Perhaps like the early days of Git.
I do really hope Nix succeeds in this, though; I've been using it, or it's been using me, for several years. [1] and [2] for more info.
Having used Nix a bunch, the system has been hard to use and learn in exactly the same ways that messy legacy codebases I've worked with have been hard—which makes sense since Nixpkgs is an absolutely massive codebase that's evolved over the years with very distributed development. People latch onto the Nix language and the paradigm because that's the most tangible difference but, everything else held equal, Nix + Nixpkgs would be no easier to work with if everything were in Python. (And, if my limited experience is any guide, would be worse with an ad-hoc DSL pretending to be Python à la Starlark!)
This is a pattern I've seen over and over: if a project tries to do something different, the novel aspect gets blamed for issues that would not be surprising in any other context. When faced with a massive awful codebase in Java, well, sometimes legacy code can be a pain, right? It is what it is. People rarely jump to "OOP is a mistake" or "Java is completely unusable". But that's exactly what happens all the time with, say, functional programming.
Pointing to an uncommon choice as the underlying cause of a problem makes for a nice, pat story; the pervasive, diffuse problems inherent to messy legacy code do not. After all, here's a single thing that, presumably we can actually change! Nix being functional is an objective fact while people can—reasonably or not—disagree on how well-designed and well-written Nixpkgs-the-codebase is. Besides, even if we could agree, what could we do about it? We all know how much momentum legacy code has and how hard it is to make anything beyond local improvements. Being a socially convenient story doesn't mean it's a particularly strong explanation, but it does mean that people will believe it even if it isn't.
Yeah, frustration on both sides here. On one hand, I can see how the nix evangelism can get annoying, especially here on HN. On the other hand the nix crowd is seeing a parade of ad hoc, informally-specified, niche implementations of way-less-than-half of nixpkgs. This is an important problem, maybe one of this most important problems the tech world is grappling with right now, and there is already a solution.
The reality is that the docs are fine. The reference material is comprehensive, and there are some very good tutorials and introductory essays to get people started. The problem is that it's difficult. Nix is very, very different from what people already know, and it takes quite a bit of effort to learn how it works. Most people just aren't prepared to do that much work just to install some software.
reply