Mounting the same /nix directory on multiple machines

Question

I want to consistently install software using the nix-package manager on multiple openSUSE machines (of different versions), of the same architecture. I am not root on any of the systems, but would want to convince our sysadmin to install nix in multi-user mode on all machines using a network mount.

• Would it be possible to mount the same /nix directory on all machines and run nix in multi-user mode on all those machines?

• Would a nix-env -i interfere with other machines?

• Would a nix-env -i xxx install xxx in the user profiles of all machines or just on the machine, where the command was executed? How about the ones installed by root?

• Does garbage-collection on one machine take into account software installed on other machines?

Answer 1

1. It is possible to mount a Nix store on multiple machines to share binaries between many hosts with similar architectures. If /nix is available on all those machines, then each of them can use the installed packages in pretty much the same way they would be used if they'd be installed locally.

   Now, having multiple machines write to the same Nix store at the same time is only going to work reliably if your underlying network filesystem supports network-wide file locking properly. This may sound harmless, but in my experience most network file systems actually don't support file locking properly and my guess is that if you'd attempt this feat then you'll run into trouble occasionally in the form of dead-locks and/or an inconsistent store.

2. nix-env -i — and all other nix-xxx command is general — synchronize access to all resources in /nix/store and /nix/var, so multiple running operations do not interfere with each other (assuming that the file system provides reliable synchronization). If a single user runs nix-env -i on two machines at the same time, then he'll run into a race condition, obviously, because one of the two commands is going to cancel out the effect of the other. That phenomenon happens the same way when you run two nix-env -i commands simultaneously on a single machine, too, though, so it's not a problem that's specific to a shared store.

3. nix-env -i modifies the user environment on all hosts simultaneously. The user profile ~foo/.nix-profile is just a symlink into the shared store at /nix/var/nix/profiles/per-user/foo/profile, so changes to that profile made on one machine will be visible on all other machines, too. root is just like any other user in that regard.

4. Yes, nix-collect-garbage will work fine on any of the machines that share the store. Since the user profiles are shared, too, the complete usage / dependency graph is visible to the tool and no store path will ever be garbage collected that's still referenced by a user profile.

   A different matter, however, are temporary environments like those created for nix-shell or by nix-build. Nix records the existence of these environments by symlinking from /nix/var/nix/gcroots to the appropriate path on the local hard disk, i.e. the place where the temporary environment lives. Such an environment is considered dead if that symlink becomes stale, i.e. if the "real path" of the Nix environment vanishes. Now, if a user enters a nix-shell environment on machine A, then there'll be a symlink created in /nix/var/nix/gcroots that points to, say /run/user/1000/nix-shell-environment-1. A garbage collector running on machine B, however, won't find that path and will consider the environment dead, therefore deleting the symlink from /nix/var/nix/gcroots and the underlying store paths from /nix/store. If that happens, then the active nix-shell environment running on A will suddenly stop working.