I would love to see a discussion from somebody who really likes Nix on why it isn't ready for prime time yet/just play devil's advocate aloud on why it isn't the greatest thing since sliced bread.
I feel like I must not be doing anything "serious" enough to need package reproducibility at that high level. I'm aware of things like git checkout/tags, package-lock.json, Cargo.lock, Docker image tags.
What is a real world use case where Nix isn't overkill? I've read toolchains but... nvm (node version manager), rustup. I still Rust on a machine once and I never think about it again.
It feels like the Kubernetes of package management/build if you will complexity wise. "Do we really need this or is it just what everybody else is doing?"
> I would love to see a discussion from somebody who really likes Nix on why it isn't ready for prime time yet/just play devil's advocate aloud on why it isn't the greatest thing since sliced bread.
Top reasons in my mind:
1. Error messages. Even with my >1 year of experience using NixOS full-time, I've encountered errors that I simply _cannot_ fix. This is getting better (recent nix releases let you introspect problems more easily).
2. Documentation gaps. Much of the nix docs are actually pretty good now! But sometimes you run into "missing links" that make it really hard.
> What is a real world use case where Nix isn't overkill? I've read toolchains but... nvm (node version manager), rustup. I still Rust on a machine once and I never think about it again.
For me, nix is unbelievably powerful to construct system images for various virtual machine formats. I'm using the nixos-generators[1] project to construct 8 or 9 image formats from one NixOS configuration. Packer and similar tools are the non-nix analog, but nixos-generators requires essentially adding a singular line to support something like Proxmox as opposed to much more work in some other tool.
I'm also using nix to build all our team's software - which varies from Vue.js to Rust to Django - and fold _all_ those development dependencies into a singular nix `devShell`. This means you can clone our repository, use `nix develop .`, and all engineers use the identical verisons of all software, dependencies, and libraries across several toolchains. It's incredibly convenient. (Imagine that you're a .js developer who needs to make a quick edit to a Rust backend route but doesn't sling Rust all day - you don't need to know how to setup rust at all, the `devShell` provides it).
It's a good question, and a very mature/well-engineered Docker dev environment probably gets you near-parity with an equivalent nix setup. That said, my reasons are:
- Although not _all_ of our projects need nix builds in the end, at least a few do, and acquiring their devshells is essentially zero-effort (you just ask nix for the devshell for the package instead of the package output itself)
- As some other commenters have noted, dealing with large container contexts can get hairy/slow. A devshell just tweaks a few environment variables, which is less of a tangle when working on the various projects (I use direnv, so my emacs session hooks into `flake.nix` and finds everything in-path automatically)
- While you could get a bit-for-bit identical dev environments by pushing a built container image to a registry that all devs pull from, I think most people would write a `Dockerfile` once and let folks build it locally before hopping in, which leaves a small (but extant) possibility that some environments may be subtly different (shifting container tags, ad-hoc apt-get commands, etc). A flake.nix coupled with a flake.lock means all devshells are lock-step identical.
I don't know much about Nix, but I'm planning on reading more about it.
Since you mentioned packer/proxmox/nixos-generators - am I understanding correctly that nix could be used instead of packer to generate a vm image/template for proxmox(or whatever hypervisor)? Is it limited to NixOS or could it create a centos image as an example?
I've used a combination of packer+ansible/chef/salt to create images, but it's always felt a little clunky.
> [...] am I understanding correctly that nix could be used instead of packer to generate a vm image/template for proxmox(or whatever hypervisor)?
That's correct, although as you mention later, it's limited to NixOS. It's an unfortunate constraint given how tremendously powerful it is, but the limitation makes sense. nixos-generators is literally a nix function that accepts a NixOS configuration and operates on the nix value in order to construct the VM image, so it wouldn't work quite the same without nix at the core.
Because nix "knows" about the total definition of the system, nixos-generators can do wonderful things like construct an AMI image or VMWare image without talking to any hypervisor APIs or cloud APIs at all - it knows how to construct the system from the "ground up" and you just end up with a big disk image file after the appropriate `nix build ...` command, no AWS keys or running Proxmox API required.
It's a tradeoff, to be sure. But if the use case fits - which for us, it does quite well - you can be outrageously productive. Adding an entirely new image format takes as long as typing a new line in `flake.nix`, and I leverage a common NixOS module between all the image types to invoke a quick and easy local qemu VM when I'd like to experiment locally.
It's become hard to imagine what managing a "normal" system would be like with the features I get by baking it all into a NixOS configuration.
I love Nix, all of my machines run NixOS, but I think it's 5-10 years out from widespread adoption. Currently the biggest issues have to do with severely lacking documentation, lack of developer tooling (missing things like LSP), very slow iteration cycles, confusing or impossible to understand errors, and even naming (Nix refers to the language, package manager, cli, community, etc).
That said, right now it's still the best tool I've ever used. Nix is great at a few things: reproducible development environments, reproducible builds, and reproducible systems. I introduced Nix at my last job to do most of these things and it helped onboarding dramatically: Nix-based development environments with automatic activation, Nix-based Docker image builds which reuse the work done for the dev environment, and Nix-Darwin configuration to configure systems with one command. It's incredibly powerful and you're able to use as much or as little as you like. So the amount of "overkill" is variable.
Of course, the problem here is that knowledge of Nix itself is necessary to build and maintain these things. Due to the reasons outlined previously, it is quite an investment. I believe it is worth it given the stability, flexibility, and efficiency Nix can provide.
> I think it's 5-10 years out from widespread adoption
To quote Andy Warhol, "They always say that time changes things, but you actually have to change them yourself."
I don't mean you personally! I mean that it's not clear how time is going to fix Nix's documentation.
My guess is that at some point a big company will either clone or copy NixOS and enforce some discipline on the documentation. At that point it'll take over the world.
> I mean that it's not clear how time is going to fix Nix's documentation.
This is my estimate based on the work that has happened over the last two years. There is steady movement in the right direction and the community is organized alongside the NixOS Foundation members to make things happen. Just not at a break-neck pace. Places like Determinate Systems, Tweag, etc are the largest forces currently, but there are plenty of others that are trying to move things along. Over time, that steady movement will land Nix in a place where its pains have been largely dealt with. I don't think we're there yet and, at this pace, won't be for a while.
New to both. In spite of significant interest and weeks of work, I can't even get Guix installed on bare metal on my Mac nor on my older Acer laptop (a converted chromebook) nor on a raspberry pi. New to Nix and have it running with hardly a hitch on all or these.
Reached out to Guix IRC and mailing list with no response (though one helpful HN commenter got back to me -- thank you, and I still need to respond!).
Nix community on the other hand has been extremely helpful on their discourse and has been remarkably welcoming to PRs and contributions in spite of me being a noob.
Will take another look at Guix in a year, it certainly seems to get many things "more right" IMO, but I can't validate that opinion if I can't get it installed.
There's no port of the GNU C Library for macOS, and there's no free toolchain that would let us build binaries for macOS.
Possible routes are:
- virtualization (use a GNU system on top of macOS)
- arbitrarily cut the graph and graft macOS-built binaries on top of a huge binary blob specific to macOS
So you're saying that it is not a language, package manager, cli and community, but it is a language, a package manager and a cli (and I guess we can assume that talking about the "Nix community" makes sense).
There is the nix language and the nix program which is a package manager and cli at the same time.
There might also be the running joke in German that nix literally means nothing but there is no etc. nix does not generally refer to anything else.
Sandro is right that the language is called Nix and the CLI is called Nix. Though, part of the trouble is that Nix handles multiple things (eg. Dev environments) and is often referred to separately as the Nix Package Manager. Realistically, people do talk about "Nix" as one of many of these things which confuses a lot of people. Even the name NixOS is confusing for those unfamiliar with Nix.
It is ready for prime time if you are an advanced linux user already.
It is the greatest thing since sliced bread if you where using Arch or Gentoo before.
Do you have more than two machines you want to keep in sync? Do you don't want to track custom patches for software? Do you want to make upgrades painless? Do you want to be forced to at least have documentation as code? Do you need to manage a multi user environment and want to know what is configured?
With NixOS you don't need to think about re-installs. They are painless and an order of magnitude easier to do. You know you will get the same configuration back.
Kubernetes is complicated and a good choice for a big team with lots of containers, but rather pointless for a single machine. NixOS already works great for a single machine and scales well for more machines but not as well as k8s. If you are building 100 different machines on hydra, you start to run into scaling problems pretty fast.
So what are actual bad points:
- you must work with the nix, not against it. If you want to continue your current workflow and don't change anything, nix will be a big obstacle for you.
- you need to invest some time and start climb the steeper learning curve.
- documentation could and should be better, especially general guides that enable and teach you how to solve your problems yourself
- smaller and bigger rough edges in places that are usually very different to classical distros
I think for a professional / team setting, its learning curve is sufficiently steep enough that it'd be risky to adopt with only one or two team members familiar with Nix.
Writing Nix code for your own packages varies in difficulty from 'trivial' to 'demands you know a lot about Nix, the compiler, Linux, and your software package'.
> What is a real world use case where Nix isn't overkill? ...
I'd rephrase this as "nix is 2nd best at everything".
So, yeah.. to setup tools you could use asdf; provision a system you could use ansible; to build an image you could use packer; etc. -- But, if you've already paid the (steep) learning curve for nix, it's going to be a better choice when doing anything that involves packages.
I'd put emphasis on "declarative" more than I'd emphasise "reproducible", though.
> It feels like the Kubernetes of package management/build if you will complexity wise.
This is an apt comparison.
However, there Kubernetes alternatives are often simpler with 80% of the power.
With Nix, there's nothing that's simpler that handles as many use cases.
This is a 100% fair point and I do agree that people can be overzealous with pushing Nix. For many projects, Nix is indeed overkill, and it's worth pointing out that some tools, like pnpm, have taken important cues from Nix. In many cases good enough really _is_ good enough and saying "just install npm using Homebrew or whatever you prefer" is totally fine. But when good enough isn't good enough and teammates and would-be contributors burn an hour installing dependencies just to push a small fix, that's where Nix can really shine. Another sweet spot is larger orgs where knowing that you can run `nix develop` in any repo and get everything you need is a legit superpower.
Personally, I think because its feature-set is currently very much "scattered".
Nix is in the transition to use 'flakes': a new concept that gives a bit more structure and allows easier reuse of Nix packages, NixOS modules and more. In addition it includes a standardized 'lock' file. Lock files are quite useful (or even essential) for reproducibility.
However, it is in experimental phase for more than 4 years now. It is behind a configuration flag, so it isn't obvious how to use it. A division in community, projects and documentation happens this way.
Because it is still considered an experimental feature, flakes and its surrounding CLI tools aren't mentioned in the official docs.
Even though it is experimental, flakes is considered an essential part of Nix by a large portion of the community.
This makes those people look for and create their own solutions:
It makes sense that these are created. I'm still hoping Nix flakes will become the default and UX can be iterated upon. But it doesn't make the future of Nix all bright and beautiful atm.
> It is behind a configuration flag, so it isn't obvious how to use it.
It is: the error message tells you exactly how to activate it.
> A division in community, projects and documentation happens this way.
No, not really. flakes are not fundamentally different, they just replace channels, a very small component, with something new and more powerful.
You can convert any NixOS config in minutes if you just import the configuration.nix, add --impure and add a nixpkgs input.
> its surrounding CLI tools aren't mentioned in the official docs.
> This makes those people look for and create their own solutions:
Also not entirely true. Before flakes there where several projects like niv which are now more and more in maintenance mode and got replaced by flakes.
There was a need for a feature like flakes in the community for a long time and it it slowly forming shape.
> This results in multiple documentation sites:
Yes, like for any other big ecosystem. There will grow community maintained wiki's, opinionated websites, guides and resources. Is that necessarily a bad thing? We can't prevent people from creating them and not everything fits into the official guide especially if it is opinionated.
> Multiple wrapping tools for development environments:
Same thing. nix-shell is pretty basic and does not have every bell and whistle and people in the community build on top of that, create wrappers and new tools and try to build the bells and whistles. Similar things for example happened in the debian community with for example aptitude and synaptics.
> I'm still hoping Nix flakes will become the default
It will and the tools you listed above won't go away. Flakes do not attempt to replace them and what they try to achieve but to support them.
I think you get that in most posts advocating for nix: its hard to spinup on as a concept/system.
At least for me personally, I've always tried to add that disclaimer, and very few nix-accolades recommend nix _wholesale, no questions asked_ to a random viewer. Almost everyone I know who is all-in on nix recommends nix with caveats, and only after doing some vetting of who you are recommending it to.
That said, even though it is hard... the pain is worth it in the end. Once you get over the hump, its a refreshing ecosystem to rely on overall. One of the main reasons I say that is because it is uniquely positioned to hit both the throwaway script and the production system. You can use it to do all of the following:
- Install things to the system temporarily
- Write scripts that are copy-pasteable between machines
- Setup monorepos with multiple languages targeted
- Run "bare metal" docker-like systems directly on the host
- Build bare-minimum docker containers
- Configure your host OS in a reproducible way
- Build prod servers in a reproducible way
- Safely upgrade/rollback a computer
- Share computer configuration in a simple data-driven manner
Some things are much harder in practice than in theory. I recently had to work with a JavaScript project that doesn’t get a lot of maintenance love. It requires a JavaScript version that is no longer part of Nixpkgs. Now try figuring out how to build just that version and also make Yarn use that version. It’s not incredibly hard but the documentation is not obvious to find.
Then go on a very very long walk while your Darwin machine is now compiling node.
Contrast that with something like Volta which just works and takes minutes to setup.
Or take anything in Node with a postInstall script that tries to download binaries.
Even Haskell that ostensibly has good support in Nix has a lot of flaws unless you use some of the third party Haskell tooling such as Haskell.nix
Quite a few Haskell packages are marked as broken and there’s only a single version of every Haskell package in the Nixpkgs Haskell packages. Meaning you will quickly run into lots of errors because of version bound problems.
Then there’s matters such a what if you install a formatter and a linter through Nix as is often suggested in those super neat tutorials on how to quickly set up a dev environment. But now you’re using a version of these tools that’s decoupled from the rest of the project.
You have similar but yet different problems on other distros, too.
For example if you want to use a very old node version then you either find a prerecompilation for your distro or also compile it yourself. The precompilation might be linked against an older glibc version or compiling does not work with your distros because your icu version is incompatible and then you are in the same rabbit hole.
volta "just works" because it was specifically designed to achieve one defined purpose run on a few platforms and is well tested on those. nix is general and cannot achieve everything. Somewhere maintainers need to cut a line otherwise things get completely out of hand.
Network is restricted in the sandbox because otherwise reproducibility would be severely reduced and we could not guarantee that the outputs match what we expect them to be. Also downloaded binaries would usually need patching anyway.
Also you are not supposed to take everything from nixpkgs. Like on any other distro if you are developing something the ecosystems own tools are the most well equipped and have all the versions. The versions in nixpkgs are the versions we need for other packages and NixOS, not every possible version you could need to develop a project with specific requirements necessarily.
PS: volta did not just work on my NixOS machine because it assumes that relative to glibc there will be a lib directory where it can place things.
> Then go on a very very long walk while your Darwin machine is now compiling node.
Yes, if you fall off the happy-path of cached public hydra builds you can easily go from "a few seconds" to "lol maybe today?". Cross-compilation especially is likely to burn you bad in this regard.
> I would love to see a discussion from somebody who really likes Nix on why it isn't ready for prime time yet
I have several friends who use Nix and generally recommend it.
I do believe I think enough like them to like Nix for the same reasons (declarative, reproducible).
I've asked them to play devil's advocate, and I generally get the downsides, but...
There is one downside that I feel they can't tell me, but they end up telling me indirectly:
Nix is very complicated and takes a long time to learn. It's the Haskell of package managers.
I love Haskell, so it's not even unreasonable that I'll like Nix. But I won't like the learning curve. I spent a decade becoming productive in Haskell, and a year becoming productive in Rust. Will I like for my operating system to have crutches for months as I reconstruct my neural pathways to perform basic navigation?
This is why I think a tutorial like this is great: A gentle introduction, and a lot of courage to spend time, is all you need.
> It feels like the Kubernetes of package management/build if you will complexity wise.
In terms of learning curve, it might not be as bad as Kubernetes.
Kubernetes has a lot of complexity because of its distributed model.
But in terms of learning a new operating system with its own paradigms, it may seem similarly impairing to learn. (You'll lose the same abilities to run things until you figure out what's going on.)
> "Do we really need this or is it just what everybody else is doing?"
> would love to see a discussion from somebody who really likes Nix on why it isn't ready for prime time yet
The daemon needs to be run as root root in a standard multi-user install which rules out HPC-like envs where Nix would actually shine. There are workarounds but none of them official.
Tooling and documentation are on the same level as for a random esoteric programming language. I love Nix but have been hesitant so far to push it at work despite immense productivity gains.
Similarly to HPC, Nix looks like a perfect match for CI/CD pipelines on the surface. In reality the only easy way to get it done is to use additional non-foss software [1].
> There are workarounds but none of them official.
But does it need to be official? Wouldn't that at the same time bloat nix even more? Isn't it enough if core people of the community for example build https://github.com/nix-community/nix-user-chroot ?
> In reality the only easy way to get it done is to use additional non-foss software
That is not true at all. You can use a nix docker image, a 5 line nix file and 2 commands to do a very basic CI pipeline almost anywhere.
When looking at the up-front time investment and long-term effects, it’s (counterintuitively) crucial that it be official.
If you’re lucky unofficial things eventually end up being official and work fine, but much more often unofficial things die on the vine. If you’re really unlucky the whole branch rots and the system you built on that branch ends up isolated on the ground.
Apologies for stretching the metaphor, it’s been a long day.
> What is a real world use case where Nix isn't overkill?
When you need to tie together code in two or more different languages. For example if you need to call an Node executable in your rust program. In that case neither npm nor rustup is enough since you need something to tie the two together. Nix is excellent for that in my experience.
Fragmented documentation. Flakes are still "experimental" and their interface subject to change, but they're also widely acknowledged to be the better way to use Nix. There aren't that many Flakes-first docs, and many existing documents describe things that don't really work with Flakes, so the already incomplete & fragmented docs are even more incomplete & fragmented than it seems at first!
AFAICT, there's no way to use the Flakes-based tools (`nix develop` and such) in the shebang of a script, so you have to stick to legacy Nix when making a shell script or Makefile or Python script that uses Nix to set up the environment. That means no lockfile, which is annoying.
Troubleshooting errors can be a horrible exercise in frustration.
Making derivations for particularly complicated build processes can be a massive pain in the ass.
Nice, but that appears to need the (non-portable) `-S` option to `env`. POSIX only requires a single argument string to pass to the utility invoked (though the documentation has it as [argument...] the OPERANDS section uses the singular "a string" form)[1]. Further arguments may be ignored (it's implementation dependent). With `nix-shell` your shebang can be `#! /usr/bin/env nix-shell` and subsequent arguments can be on the following lines, and `nix-shell` itself knows to parse that. Busybox's `env` doesn't support `-S`, and Busybox is the standard shell environment in Alpine (for a common use).
I feel like I must not be doing anything "serious" enough to need package reproducibility at that high level. I'm aware of things like git checkout/tags, package-lock.json, Cargo.lock, Docker image tags.
What is a real world use case where Nix isn't overkill? I've read toolchains but... nvm (node version manager), rustup. I still Rust on a machine once and I never think about it again.
It feels like the Kubernetes of package management/build if you will complexity wise. "Do we really need this or is it just what everybody else is doing?"