R: Converting a nested list with empty elements to data.frame (from json)

Question

I have imported a json file like this one:

library(rjson)
json_str <- '[{"id": 1, "code": 7909, "text": [{"col1": "a", "col2": "some text"}], "date": "2015-12-01"}, {"id": 2, "code": 7651, "text": [], "date": "2015-12-01"}, {"id": 3, "code": 4768, "text": [{"col1": "aaa", "col2": "Blah, blah"}, {"col1": "bbb", "col2": "Blah, blah, blah"}], "date": "2015-12-01"}]'
my.list <- fromJSON(json_str)
str(my.list)

Needless to say the real file is much longer.

As a result I get a nested list of 3 elements where each element is a list of 4, and then, the element $text is a list of variable length from nothing to any number of elements, in my case, usually no more than 3.

After some research I have found several answers about converting a list to data.frame, for example here and here. However, none of them work when one or more of the nested lists in '$text` is empty.

do.call(rbind, lapply(my.list, data.frame, stringsAsFactors=FALSE))

library(data.table)
rbindlist(my.list, fill=TRUE)

Both return an error.

I would like to either convert the list in $text to several columns of the data.frame or just one (pasting the content).

Another option would be to be able to skip some elements (say $text) and convert the rest of the list, then in a separate line convert those elements (say $text) to a different data.frame. I think I could somehow relate one data.frame to the other.

Can anyone give me any idea on how to do this. Thanks

Answer 1

By the sounds of it, something like the following should work:

do.call(rbind.data.frame, lapply(my.list, function(x) {
    x[["text"]] <- toString(unlist(x[["text"]]))
    x
}))
##    id code                                   text       date
## 2   1 7909                           a, some text 2015-12-01
## 21  2 7651                                        2015-12-01
## 3   3 4768 aaa, Blah, blah, bbb, Blah, blah, blah 2015-12-01

This follows your idea of pasting the values together (here using toString) to form a single column in the data.frame.