NixOS and Back Again

Linux is great. I love Linux. One of the aspects I love the most is customization and the chance to really bend the whole system setup to exactly what I want. Now when I say I love "Linux" more specifics are needed.

There are a ton of distributions out there, and for much of my time in the linux world (1997-ish until now), many of them have been basically shipping their own spin on the same thing. The major differences really have come down to package manager, release philosophy, defaults, and theming. When you install most distributions the main learning curve is absorbing the defaults and deciphering the package manager syntax.

Nix(OS) Is Different

Disclaimer: For the sake of this little write up, I'm not going to distinguish too much between nix the package management system and NixOS the operating system/distribution. I'm also going to rephrase a lot of nomenclature and descriptions in a way I feel is more accessible to those who are not already familiar with nix.

NixOS works on a declarative configuration file that defines your entire system (well, mostly). Changes to this file and a subsequent rebuild trigger packages being added and removed from the system, settings changed, etc. The resulting system is a snapshot known as a generation. This is actually a really killer feature, since every generation can be switched between pretty much at will. What this means is that making some wild changes to your system setup are virtually guaranteed to not bork it beyond recovery.

Caveat: A lot of user specific settings are better served being configured through "Home Manager", a user-land system that relies on the nix packager and syntax, but only affects the user home directory. Home Manager is well integrated with the nix system, meaning everything can be packaged together in one configuration file (or set of files) for simplicity or organization.

Nix Is Very Cool

Nix is incredibly powerful and useful, not only in theory or principal, but in practice. It means one configuration file can be pulled on to any number of systems and in minutes (literally) you have identical setups with all your tools configured exactly the same.

The integration in configuration of applications is actually astonishing. The amount of work that has been invested in nix and home manager is a triumph of open source. Using a nix language server you can get configuration completion and type checking for tools like zsh and neovim (and gazillions more). In my experience, this makes configuration options really discoverable, and resources like mynixos are incredibly helpful.

Home Manager with nixvim is my single favorite neovim package manager I have ever used. (I'll cover more on that in a later post.)

Nix Is Very Hard

I'm going to point out some friction points I have encountered while running NixOS across several systems over 6 or so months. Of course, this is all just my experience and Is not meant to be overly negative toward nix or NixOS.

Syntax

I don't think any discussion about nix or nixos can be complete without discussing the syntax. This is a simple file, and perhaps it is not a great example of complexity, but it does demonstrate some basic syntax:

{ inputs, ... }:
{
  imports = [
    ./hardware-configuration.nix
    ../../common.nix
    inputs.home-manager.nixosModules.default
  ];
  networking.hostName = "nixos";
}

The real highlight is the functional purity of inputs and outputs. If you aren't familiar with functional programming principals at the start, getting syntax and functionality correct will be a steep climb. I think most experienced "Linux People" are used to doing things "the Linux way", mostly involving scripting languages or familiar configuration files. Nix, in large part, is a departure from this.

Docs Are... Where are the docs?

While there are some really good documentation resources, there are also enormous gaps. The modus operandi of the community is to scour github (or whatever) for examples of other configurations that "get close" to what you are trying to do, and go from there. This suddenly feels like the old joke that no one has ever written a Makefile from scratch, except the very first one.

While, generally speaking, this seems to work out pretty well, I have found myself completely unable to figure out how to do something with very few resources to fall back on.

There Is No "Right Way"

In true Linux/Unix fashion, there isn't just one way to do anything. There are these things called flakes, you should use them! But also, you might not need them. They are really powerful and let you extend the capabilities of nix. However, if you are sticking to the happy path on software and settings, it is another layer of complexity. You can have home manager or nix download and package things, clone from git, fetch with http or ftp, etc. Which is the right method is sometimes mystifying and trial-and-error is your only solution.

Developing Is Great, But Also Harder

I write code all day long, and nix has some incredible tools to set up and maintain multiple development environments that can be used for different projects. There are dev shells, nix environments, and the aforementioned flakes. Which of these is the right tool for the job takes some experience to understand.

For example, if you are trying to compile some code which needs the openssl development libraries, you cannot install these on the base nixos, by design. This has to do with reproducibility and the generations I mentioned earlier, versioning all of that would be quite difficult. What this means is that we need a development environment to provide the necessary files to get this done. There are a couple of ways to do this:

With nix-env:

nix-env -p openssl

will drop you right into a shell in your current directory that now has the openssl development files available. If they were not present before, they will be downloaded and stored.

You can also use flakes:

{
  description = "Dev Shells";

  inputs = {
    nixpkgs.url = "nixpkgs/nixos-23.11";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = nixpkgs.legacyPackages.${system};
      in
      {
        devShells.linux = pkgs.mkShell {
          packages = with pkgs; [
            openssl
          ];
        };
      });
}

Isn't that clear as day? Now flakes allow you to do all kinds of neat things, like you can build the entire dev environment in the flake.nix file in your repository, and then when someone clones they can get your exact toolchain versions so you are guaranteed things will work.

Where this comes off the rails a bit for me is when I need to do something kind of bonkers... like cross compiling from linux to macos, there is not a built in toolchain solution in nixos for that, so I would end up using cross-rs like I do everywhere else, somewhat missing out on some nix-y goodness. When I start reaching for other tooling outside the nix ecosystem it starts to feel a little more hacky and I start wonder if I'm doing it "wrong".

What generally happens is that 50 times a day I completely forget to enter a dev shell and open neovim to do some work and suddenly LSP features are not working because there is no openssl on the base system and Cargo decided it could not compile and process the project.

Is this a nix problem? Absolutely not. This is a me problem.

And That's The Issue

Nix is awesome, all the principals are amazing and it has even got me out of some binds. But... and this is an important but... It introduces a lot of cognitive overhead when you are trying to add things to your workflow. It is such a unique departure from all I know about Linux-ing that everything feels fresh and new mixed with scary and uncertain.

Sometimes I don't even know where to start looking for answers, and instead of being able to solve the problem with man pages or applying some acquired past knowledge, I'm just starting from zero. And that, to me, feels like too much work.

All of these feelings are my feelings and don't speak to some objective truth about Nix or NixOS. It just simply means that at this moment in time I don't have the brain power to embrace this new way of doing everything.

So What's Linux Nerd To Do?

Well my first plan was to just jump to another distro and continue using Home Manager to port my configurations around. However, HM still requires a working Nix installation and I immediately ran into all kinds of local conflicts when I attempted to set it up on a Void machine. Neovim was freaking out and things were just total chaos. In an unrelated endeavour, I had home manager running on my Macbook and something caused that installation to break within 24 hours of my other issues and it all just felt like a sign.

I decided to spend an evening or two ripping my configurations out of home manager and going back to the old dotfiles repo. I also started writing some setup and install scripts for my common used distros (Void, Tumbleweed, and Debian) to recreate the Nix+HM experience.

Is it the same? Not at all. Is it messy and dirty and gross? Of course. Is it Wonderful in its own Linux way? You bet.

And All This Means...?

Nothing, really. I gave NixOS a lot of run and it just didn't seem like the right fit for me at this point in my life. I have too many other responsibilities and Nix felt like I was going back to school. That being said, I do firmly believe that some sort of system like nix is probably the Linux of the future. I am not at all certain if that Future Linux is nix in the current form. With projects like Fedora Silverblue and OpenSuse MicroOS, the idea of atomic, reproducible, immutable, etc etc is starting to take hold. All of these projects have differences, but at their core it feels like we are all searching for a way to configure our systems in a sane and efficient manner. Seeing which of these ultimately wins the hearts and minds of users should be fascinating.

There is so much amazing work that has gone into the nix ecosystem that I am sure it will thrive for a long time. I continue to have my eye on the nix world and track what is happening, but over here, I'm just rocking a traditional Linux environment.