Issue with complete.cases: invalid 'type' (list) of argument

Question

I'm getting the following error:

Error in complete.cases(dt) : invalid 'type' (list) of argument

Never had this message before when using complete.cases on a data frame.

A call to class(dt) returns data.frame, so no problem there.

dt is relatively big -- 800,000 obs of 90 variables.

Similar operations on other data frames there are no problems.

Anyone knows what could be the problem?

Answer 1

I also encountered the same issue. As @hrbrmstr correctly pointed out, the data.frame has list objects. In my case it was a data.frame of lists.
I converted the data.frame of lists into an actual data frame using the following command:

DF <- data.frame(matrix(unlist(DF), nrow=nrow(DF)),stringsAsFactors=FALSE)

Using complete.cases on this worked out.

Answer 2

I had the same issue and @hrbrmstr's comment on the original question was a great help. Posting the answer with code so that anyone who stumbled upon this question may benefit from it.

The dataset in question, x, has one or more variable of type list, leading to the error message:

Error in complete.cases(x) : invalid 'type' (list) of argument

Usually, you would get a quick preview of the variable types from a data frame using str(). However, in my case I have a data frame of 2,431 variables (long and wide dataframe) so using str() is of limited use there. Instead, I use the handy sapply() code to get a table of all classes present in my data frame:

table(as.character(sapply(x, class)))
# output:
c("ordered", "factor")      character         list         logical          numeric 
             1                   69            1            2225              136 

Notice there is a variable of type list in our data frame.

---

Solution

In the following code snippet, we identify any variables that are of the list type and remove it from x. We use table() again to verify that our data frame now no longer contain any list variables:

is_list <- sapply(x, is.list)
x <- x[, !is_list]
table(as.character(sapply(x, class)))

Proceed to apply complete.cases():

x <- x[complete.cases(x), ]