unlist nested lists without losing Nulls

Question

  dat=list(list(id = "T"), list(id = "T"), 
structure(list(NULL), names = NA_character_), list(id = "T"), 
list(id = "T"), list(id = "T"))

      unlist(dat)

I treid but did not work

    dat[sapply(dat, is.null)] <- NA

Answer 1

You should use the answer by ThomasIsCoding with a list as above, where you do not have NULL at the top level and do not want to recursively unlist.

However, if either of these conditions are not satisfied then you can traverse the list, setting NULL to NA.

tidyverse approach

I like to use purrr::modify_tree() for operations like this, passing a function to apply to every leaf.

dat |> 
    purrr::modify_tree(
        leaf = \(x) if(is.null(x)) NA else x,
        post = unlist
    )
#   id   id <NA>   id   id   id 
#  "T"  "T"   NA  "T"  "T"  "T" 

The post parameter is a function that is applied on the way "up", i.e. after the leaves are transformed.

base R approach

It feels as if you ought to be able to use base::rapply() to recursively iterate. Unfortunately, it ignores NULL elements. We can instead write a recursive function:

replace_null  <- function(x, repl = NA, is_node = is.list) {
    if(is_node(x)) return(lapply(x, replace_null))
    if(is.null(x)) repl else x # or in R 4.4:   x %||% repl
}
unlist(replace_null(dat))
#   id   id <NA>   id   id   id 
#  "T"  "T"   NA  "T"  "T"  "T" 

The is_node parameter determines whether to recurse further, in the same way as the argument with the same name in purrr::modify_tree().

The null coalescing operator %||%

Incidentally if you're using the current development version of R (to be released as 4.4) you can use the null coalescing operator.

L %||% R newly in base is an expressive idiom for the phrases if(!is.null(L)) L else R or if(is.null(L)) R else L.

Here is an example of the difference of this approach when one of the top-level items is NULL.

l  <- list(1, NULL)
unlist(l)
# [1] 1

dat |> 
    purrr::modify_tree(
        leaf = \(x) x %||% NA,
        post = unlist
    )
# [1]  1 NA

Answer 2

You can configure recursive = FALSE in unlist, e.g.,

> unlist(dat, recursive = FALSE)
$id
[1] "T"

$id
[1] "T"

$<NA>
NULL

$id
[1] "T"

$id
[1] "T"

$id
[1] "T"