patchelf is not really widely used for solving reproducible builds issues. It's made for rewriting RPATHs which is essential for NixOS, but not something you would be seeing in other distributions except for when someone need to work around poor upstream decisions.
I really appreciate how easy Nix makes this. It’s relatively straightforward to overlay a package in your configuration to add a patch - meaning I get to apply my own patches and also get updates from upstream - at least until my patch doesn’t apply cleanly.
I run NixOS, but I'm relatively new to Nix. A couple weeks ago I was trying to get an old vanity key generator to build because it wasn't on nixpkgs (vanitygen++) and on a lark I decided to write my own derivation using some examples of similar C++ packages I found and figuring out what nix packages it needed to build. So I did, and the maintainer accepted the PR: https://github.com/10gic/vanitygen-plusplus/commit/7bcee06f3... Note that I am not a C++ developer and only know the basics about how to get C/C++ things to build.
So what does this get us? In theory, this is now trivially-buildable on any distro that has nix installed, or on nixos. I haven't converted it to a flake yet (still learning that) but if I (or someone else) did, it would be deterministically reproducible anywhere on nix.
I agree that there's a learning curve on figuring out the "recipe" for a thing, but once someone does that work, it's basically done forever. Which is a promise that no other system can even approach.
And once you figure out a certain amount of it, this moment will come where you realize you want every system you deal with to be Nixified. Like the guy elsewhere in this thread who rebuilt his own router using NixOS and loves it.
Getting new developers up to speed in your shop's dev environment? Completely trivial to do with Nix. Getting a machine back up after a failure? Reinstall NixOS and reapply the config you were using (which is in source control). Upgraded Gnome and some things you need broke? Or updated your GPU driver and now your screen just shows black? Simple... Roll back in the bootloader to a previously-working config, then undo your changes to the config definition.
Have 10 different projects all with different dependencies and version requirements that would normally collide with each other? Trivial, each project has its own default.nix file and Nix can drop you right into that environment while you're working on those projects.
NixOS (and very honorable mention to Guix) are in fact the only ways to run Linux while keeping your sanity intact!
Yeah, thanks. To add to that, the Nixpkgs "standard library" includes lots of convenient tools for patching like this, and much of the common stuff is done automatically by the standard package function. The #nixos IRC channel is a good place to ask if you run into trouble when packaging something.
Generally, making NixOS packages is really delightful. Even the patching stuff is straightforward and easy to use. Now that I've learned the basics, I routinely make small packages.
With many typical packages the only thing you need to do is specify how to fetch the source, plus the list of build dependencies, and there are built-in functions for fetching from HTTPS or Git.
It works fine for applying the patch, but then you have to manually follow all of the build steps, turn it into a distribution package, etc. With Nix it is easy to edit an upstream package that already knows how to build the software, to include the patch.
I end up just using patchelf since getting the alternatives working is harder. I would prefer if NixOS is going to give up on moving everything into /nix that it just makes symlinks in the standard paths so users don't need to find something to do it for them.
If the derivation has already been written (as GP implies), then compiling something yourself with Nix is essentially just a longer installation process. No downloading tarballs, no keeping track of things in /usr/local, no stow or manually adding things to your path. Nix is essentially a source distribution that pulls common binaries from caches.
Indeed. I started using NixOS and Nix a bit more than six months ago and I was contributing patches within no-time. One can just submit a pull request on GitHub (with the proper format) and a lot of aspects are automated: e.g. the impact of the change (in terms of packages that are effected) is automatically determined and someone with the right privileges can request a build. Some PRs linger around for a longer time, but most of my PRs were merged within the day. Also, I received useful feedback if a PR needed more polish.
What helped me a lot with Nix is that you can easily make your own local derivations. Derivations are usually short [1] and bear little risk with sandboxed builds enabled. Basically, you do a nix-shell -p mypackage, you get dropped in a shell with your derivation, but it does not affect the rest of your system.
I use it, it's quite cool, I did need to patch it to fit my needs, but it makes it easy package anything.
I wish though that it would expose the mechanism as nix derivation instead of CLI command (I eventually figured how to do it, but feels a bit hacky to me)
Nix is an abstraction around `tar -xf && make && make install`. In fact, those commands are even executed by nix, just in a sterile reproducible environment.
What sets nix apart from other package managers is that you are never running `make install` on your root filesystem, but `make` can still dynamically link to libraries (that also aren't installed on the root filesystem) without editing the Makefile directly to find them.
This way, you can't break existing packages, you can trivially roll back changes (because updates are new instances), and you can always start over fresh.
The only problem is that you have to wrap every package in a derivation, then publish that derivation somewhere. Right now, all derivations are tracked in a single git repo (with dozens of branches), all coordinated over GitHub Issues, and referenced by nix itself by an arbitrary (versionless) name in a global namespace in this file: https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/...
That last bit can be avoided by using pinning and flakes, but it's still the default way to use nixpkgs, and documentation doesn't clarify much or offer a better consistent UX paradigm.
How do you maintain derivations for your own software with respect to nixpkgs?
Do you have your own function that composes things and imports <nixpkgs>, along with overrides for things that you find deficient in the derivations found in nixpkgs? Do you include derivations for your own software as part of your fork of nixpkgs that doesn't make its way back upstream?
3. there are very powerful hooks to allow you to easily customize derivations by various means (look at some more complex derivation, like php, which defines dozens of options and there are other mechanisms as well).
1. Brew also has, but not 2 or 3 (it's definitely more customizable than say debian packages, but much less so than nix)
Just to elaborate a bit for those not familiar to Nix (slightly simplified to exclude recent support for content addressing). Nix work with derivations, a derivation is basically a data structure that specifies how a package is built. Derivations are normally not created by hand but using a function (eg. stdenv.mkDerivation).
When you ask Nix to build a package, it hashes a normalized form of derivation data structure. This hash is useful in various ways, but one way it is used [1] is to look up whether the derivation is already in the Nix Store. Because if it is, there is no need to build it. So Nix looks up whether
/nix/<the_derivation_hash>
exists. If it exists, the build is done. If it doesn't exist and you have a binary cache configured (which by default is the binary cache provided by the NixOS project), Nix will look up the derivation hash in the binary cache. If it exists in the binary cache, Nix will download the path to the local Nix store. After that
/nix/<the_derivation_hash>
exists in the store and the build is done (without building anything). Only if that fails, Nix will actually build the derivation.
Now, one of the cool things about Nix is that it is derivations all the way down. So, it's not that just what we traditionally think of as packages is a derivation, but people wrap up all kinds of things as derivations, including configuration, etc. Since derivations are usually generated by functions, there are all kinds of useful functions that make derivations for eg.: single configuration files, scripts, etc.
In the end, building a NixOS system generation is just building a derivation. nixos-rebuild switches to a different generation by just setting a bunch of symlinks to an output path in the store containing that system generation (/nix/<system_config_derivation_hash).
At any rate, when you make a one-line change to a 200-line Nix configuration, Nix does have state to keep track of what it needs to rebuild or not. Nix will just try to build the derivation (and its dependencies), but it hashes the derivations, finds that their output paths are already in the store.
Some might argue that then the store is state. But it's not, at build time you are evaluating a pure function with memoization (the Nix Store).
[1] There is also a package name and version in the store path, but lets keep it simple.
> Nix is not a build tool designed to work with arbitrary or latest development versions of libraries, for example
Well it sort of is and it sort of isn't. The great thing about Nix is how the provided packages remain malleable. You can usually quite easily make a small override to a provided package to make it build from a specific revision of the source you desire, or add a custom patch, and Nix will just build it all for you then & there. Then you can go and rebuild bits of the rest of the distribution that depend on that using your custom version. If you so want.
You almost never want to invoke patchelf by itself but use some wrapper like autoPatchelfHook, FHS, nix-ld, nix-autobahn, etc.
Since 2018 a lot quite a lot happened. Give it another chance.
reply