What is the purpose of nix-instantiate? What is a store-derivation?

Question

In the manual it is written :

The command nix-instantiate generates store derivations from (high-level) Nix expressions.

But what are store derivations ?

The manual says the following about store derivations :

A description of a build action. The result of a derivation is a store object. Derivations are typically specified in Nix expressions using the derivation primitive. These are translated into low-level store derivations (implicitly by nix-env and nix-build, or explicitly by nix-instantiate)

This is a little bit difficult to understand for a nix-newbee and I found nothing more enlightening about nix-instantiate and store derivations by googling. I also asked on #nixos, I got no answer, yet.

Could someone please explain on a simple example what a store derivation is, what is it used for ?

Why one would generate store derivations using nix-instantiate? Could you give a super simple, easy to understand example ?

Answer 1

What is nix-instantiate good for ?

The command nix-instantiate sole purpose is to evaluate Nix expressions. The primary purpose of the Nix languages is to generate derivations.

What is a store-derivation?

A derivation (see example) is a computer-friendly representation of the build recipes used for building (realize) packages. They are files with the extension .drv which are listed in the store directory, usually /nix/store.

These build recipes are understood by the Nix daemon, and are used to ensure that all the dependencies are built before, and stored in the pre-computed paths. Once all dependencies are successfully compiled, then the Nix daemon can look for substitute, or realize the derivation locally. All the detailed explanation is available in Eelco Dolstra PhD Thesis.

These files are created each time the nix-instantiate command evaluates the derivation function of the Nix language, unless the --eval command line option is provided.

Why one would generate store derivations using nix-instantiate ?

If you are interested in the build output, you should prefer nix-build, which is equivalent to:

$ nix-store -r $(nix-instantiate '<nixpkgs>' -A hello)

In some cases you are not interested in the build results, but in looking at the compilation time dependencies. For example, if you want you to investigate the build time dependencies of the hello package. Then using the nix-store command as follow, you can request all the dependencies of the build recipe:

$ nix-store --tree -q $(nix-instantiate '<nixpkgs>' -A hello)

Answer 2

^{All quotes are from Eelco Dolstra's PhD thesis.}

Store derivations

A store derivation is a Nix expression with all variability removed and translated into an alternative format. This intermediate representation "describes a single, static, constant build action" that can be built into software components.

^{"Nix expressions usually translate to a graph of store derivations."}

To put it differently,

*------------------------------------------------------*
|                                                      |
| NIX EXPRESSION == function                           |
|                                                      |
| ( Describes how to build a component. That is, how ) |
| ( to  compose  its  input parameters, which can be ) |
| ( other components as well.                        ) |
|                                                      |
| STORE DERIVATION == function application             |
|                                                      |
| ( Result of a Nix expression called with concrete arguments. ) |
| ( Corollary: a single Nix  expression  can produce ) |
| ( different derivations depending on the inputs.   ) |
|                                                      |
*------------------------------------------------------*

For context:

^{Image taken from section "2.4 Store derivations".}

---

The thesis describes a Nix expression as a "family of build actions", in contrast to a derivation that is "exactly one build action".

                              ARG_1, ..., ARG_N

                        | ---(aaa, ...)---> DERIVATION_1
        NIX EXPRESSION  | ---(bbb, ...)---> DERIVATION_2
                        |       :
           function(    |       :
             param_1,   |       :
             ...,       |       :
             param_N    |       :
             )          |       :
                        | ---(zzz, ...)---> DERIVATION_N

The derivations above could be producing the same application but would build it with different configuration options for example. (See APT packages vim-nox, vim-gtk, vim-gtk3, vim-tiny, etc.)

Why is it called "derivation"?

Its name comes from "2.2 Nix expressions":

The result of the function [i.e., Nix expression] is a derivation. This is Nix-speak for a component build action, which derives the component from its inputs.

Why are "store derivations" needed?

Section "2.4 Store derivations" has all the details, but here's the gist:

Nix expressions are not built directly; rather, they are translated to the more primitive language of store derivations, which encode single component build actions. This is analogous to the way that compilers generally do the bulk of their work on simpler intermediate representations of the code being compiled, rather than on a fullblown language with all its complexities.

Format of store derivations

From section "5.4. Translating Nix expressions to store derivations":

The abstract syntax of store derivations is shown in Figure 5.5 in a Haskell-like [135] syntax (see Section 1.7). The store derivation example shown in Figure 2.13 is a value of this data type.

Figure 5.5.: Abstract syntax of store derivations

 data StoreDrv = StoreDrv {    
   output : Path,              
   outputHash : String,        
   outputHashAlgo : String,    
   inputDrvs : [Path],         
   inputSrcs : [Path],         
   system : String,            
   builder : Path,             
   args : [String],            
   envVars : [(String,String)] 
 }                             

Example

For example, the Nix expression to build the Hello package in Figure 2.6,

Figure 2.6

{stdenv, fetchurl, perl}:

stdenv.mkDerivation {
  name = "hello-2.1.1";
  builder = ./builder.sh;
  src = fetchurl {
    url = http://ftp.gnu.org/pub/gnu/hello/hello-2.1.1.tar.gz;
    md5 = "70c9ccf9fac07f762c24f2df2290784d";
  };
  inherit perl;
}

will result in an intermediate representation of something similar to in Figure 2.13:

Figure 2.13 Store derivation

{ output = "/nix/store/bwacc7a5c5n3...-hello-2.1.1" 25
, inputDrvs = { 26
    "/nix/store/7mwh9alhscz7...-bash-3.0.drv",
    "/nix/store/fi8m2vldnrxq...-hello-2.1.1.tar.gz.drv",
    "/nix/store/khllx1q519r3...-stdenv-linux.drv",
    "/nix/store/mjdfbi6dcyz7...-perl-5.8.6.drv" 27 }
  }
, inputSrcs = {"/nix/store/d74lr8jfsvdh...-builder.sh"} 28
, system = "i686-linux" 29
, builder = "/nix/store/3nca8lmpr8gg...-bash-3.0/bin/sh" 30
, args = ["-e","/nix/store/d74lr8jfsvdh...-builder.sh"] 31
, envVars = { 32
    ("builder","/nix/store/3nca8lmpr8gg...-bash-3.0/bin/sh"),
    ("name","hello-2.1.1"),
    ("out","/nix/store/bwacc7a5c5n3...-hello-2.1.1"),
    ("perl","/nix/store/h87pfv8klr4p...-perl-5.8.6"), 33
    ("src","/nix/store/h6gq0lmj9lkg...-hello-2.1.1.tar.gz"),
    ("stdenv","/nix/store/hhxbaln5n11c...-stdenv-linux"),
    ("system","i686-linux"),
    ("gtk","/store/8yzprq56x5fa...-gtk+-2.6.6"),
  }
}

The abstract syntax of store derivations is shown in Figure 5.5 in a Haskell-like [135] syntax (see Section 1.7). The store derivation example shown in Figure 2.13 is a value of this data type.

Figure 5.5.: Abstract syntax of store derivations

 data StoreDrv = StoreDrv {    
   output : Path,              
   outputHash : String,        
   outputHashAlgo : String,    
   inputDrvs : [Path],         
   inputSrcs : [Path],         
   system : String,            
   builder : Path,             
   args : [String],            
   envVars : [(String,String)] 
 }