Why do rapply and lapply handle NULL differently?

Question

I'm aware that NULL values in lists can sometimes trip people up. I'm curious why in a specific instance lapply and rapply seem to treat NULL values differently.

l <- list(a = 1, c = NULL, d = 3)

lapply(l,is.null)
$a
[1] FALSE

$c
[1] TRUE

$d
[1] FALSE

So far so good. How about if we do the exact same thing with rapply?

rapply(l, is.null, how = "replace")
$a
[1] FALSE

$c
list()

$d
[1] FALSE

This example is very simple and non-recursive, but you see the same behavior in rapply with nested lists.

My question is why? If, as advertised in ?rapply, it is a 'recursive version of lapply', why do they behave so differently in this case?

Answer 1

I think you answered your own question: because it's recursive.

You don't often see this, but NULL can actually be used to indicate an empty sequence, because it is the empty pairlist (similar to how () in Scheme terminates a list. Internally, R is very scheme like).

So, rapply recurses into the empty list, but doesn't bother turning it back into a pairlist when it's done; you get a regular empty list.

Actually, rapply and lapply don't really treat NULL that differently:

> lapply(NULL, identity)
list()

And you can see in the R source code (memory.c) that this is exactly how pairlists are meant to work:

SEXP allocList(int n)
{
    int i;
    SEXP result;
    result = R_NilValue;
    for (i = 0; i < n; i++)
        result = CONS(R_NilValue, result);
    return result;
}