Converting a list of lists into a data.frame in R

Question

I am attempting to convert a list of lists to a data.frame. I realize this question has been asked multiple times, but I cannot find an earlier answer that works in my case.

Here are a couple of earlier posts:

How to flatten a list of lists?

R list of lists to data.frame

By far the best answer I have seen is by Benjamin Christoffersen at the second link above, but in my case I only have one value per sublist, I have missing observations, and my lists have names, which I wish to keep.

Here is my example data set:

AA <- list(my.col1 =    1, my.col2 =    4, my.col3 = NULL, my.col4 = NULL)
BB <- list(my.col1 = NULL, my.col2 = NULL, my.col3 = NULL, my.col4 = NULL)
CC <- list(my.col1 =   13, my.col2 =    8, my.col3 =    2, my.col4 =   10)
DD <- list(my.col1 = NULL, my.col2 = NULL, my.col3 =   -5, my.col4 =    7)

my.stuff <- list(AA, BB, CC, DD)
names(my.stuff) <- c("AA", "BB", "CC", "DD")
my.stuff

Here is the desired data.frame:

desired.object <- read.table(text = '
     my.var   my.col1 my.col2 my.col3 my.col4
        AA       1       4    NULL    NULL
        BB    NULL    NULL    NULL    NULL
        CC      13       8       2      10
        DD    NULL    NULL      -5       7', 
stringsAsFactors = FALSE, header = TRUE, na.strings = "NULL")
desired.object
#  my.var my.col1 my.col2 my.col3 my.col4
#1     AA       1       4      NA      NA
#2     BB      NA      NA      NA      NA
#3     CC      13       8       2      10
#4     DD      NA      NA      -5       7

I can get output that looks similar, but it is not at all in the format I want:

my.stuff2    <- do.call(rbind, my.stuff)
my.stuff2
#    my.col1 my.col2 my.col3 my.col4
# AA 1       4       NULL    NULL   
# BB NULL    NULL    NULL    NULL   
# CC 13      8       2       10     
# DD NULL    NULL    -5      7

Sorry if this problem has already been answered.

Answer 1

What about this?

do <- as.data.frame(do.call(rbind, lapply(my.stuff, as.vector)))
do <- cbind(my.var=rownames(do), do)
do[do == "NULL"] <- NA

Result

> do
   my.var my.col1 my.col2 my.col3 my.col4
AA     AA       1       4      NA      NA
BB     BB      NA      NA      NA      NA
CC     CC      13       8       2      10
DD     DD      NA      NA      -5       7

Edit:

If we don't want lists as column objects as @akrun reasonably suggests, we could do it this way:

u <- as.character(unlist(my.stuff, recursive=FALSE))
u[u == "NULL"] <- NA
do <- matrix(as.integer(u), nrow=4, byrow=TRUE, 
             dimnames=list(NULL, names(my.stuff[[1]])))
do <- data.frame(my.var=names(my.stuff), do, stringsAsFactors=FALSE)

Test:

> all.equal(str(do), str(desired.object))
'data.frame':   4 obs. of  5 variables:
 $ my.var : chr  "AA" "BB" "CC" "DD"
 $ my.col1: int  1 NA 13 NA
 $ my.col2: int  4 NA 8 NA
 $ my.col3: int  NA NA 2 -5
 $ my.col4: int  NA NA 10 7
'data.frame':   4 obs. of  5 variables:
 $ my.var : chr  "AA" "BB" "CC" "DD"
 $ my.col1: int  1 NA 13 NA
 $ my.col2: int  4 NA 8 NA
 $ my.col3: int  NA NA 2 -5
 $ my.col4: int  NA NA 10 7
[1] TRUE

Answer 2

We can use a recursive map

library(tidyverse)
map_df(my.stuff, ~ map_df(.x,  ~ 
                      replace(.x, is.null(.x), NA)), .id = "my.var")  
# A tibble: 4 x 5
#  my.var my.col1 my.col2 my.col3 my.col4
#  <chr>    <dbl>   <dbl>   <dbl>   <dbl>
#1 AA           1       4      NA      NA
#2 BB          NA      NA      NA      NA
#3 CC          13       8       2      10
#4 DD          NA      NA      -5       7