Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Glmer taking days to compute and never ending

Tags:

r

lme4

I've seen some previous similar complains around Stack about this but couldn't get to a solution that would work for me. Also, I do need everything in my model to remain, due to theoretical implications. I've looked into Rfast package as well (thank you to one of the creators who has been very kind to respond over email to my queries). However, it doesn't serve the purpose in my case, as I need both random slopes and random intercepts, as well as the correlation between the two.

So, I've got the model as:

acc_glmer <- glmer(Accuracy ~ Language * Switching + 
                        (Language|Subject) + (Switching|Subject) + 
                        (Language|Item) + (Switching|Item),
                      family = "binomial", data)

Does anyone have suggestions of how I can speed up the process or what could be going wrong? Even if it takes hours, I'd be happy because it will still end at some point. Currently it runs for more than a day before I kill the process and RStudio usually blocks, so I have to restart it (no error message or warning). As an additional info, I'm running it on data with 27,000 observations (no NAs for these variables).

Any input would be much appreciated, thank you.

like image 299
Orestes_Fox Avatar asked Dec 09 '25 18:12

Orestes_Fox


1 Answers

Thank you everyone for the very helpful feedback! I'll elaborate for future troubleshooting of other people what I tried and what worked or didn't for my data:

  1. So, I tried first adding verbose=2 but it seems to do nothing. And it would still run forever (at least 16 hours), without displaying anything.
  2. At the next step, I tried setting control = glmerControl(optimizer = "Nelder_Mead"), and found it didn't converge for my data, with the message:
Warning message:
In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge with max|grad| = 0.130366 (tol = 0.002, component 1)
  1. When I added control = glmerControl(optimizer = "Nelder_Mead", calc.derivs=FALSE) it did forcefully "converge" but next solution worked better I think.
  2. I tried the same with the other glmer option control = glmerControl(optimizer = "bobyqa"). Model converged.
  3. According to what I read, bobyqa uses nAGQ=0 by default (correct me if wrong), so I tried the following settings too: control = glmerControl(optimizer = "Nelder_Mead"), nAGQ=0. Model converged.
  4. I didn't use mustart or estart because I don't think I understand well what kind of values make sense to add there for my case, unfortunately. So I can't comment on whether they worked or not for me.
  5. Finally, when I run allFit to the model, with nAGQ=0, the summary I get is that they're all OK, and the likelihood is very similar between all (all are at -8101.988 and Nelder-Mead is -8102.555 ).

Thank you again everyone! I think after reading the suggested articles I understand a bit better my problem. Feel free to comment and correct me for any of the above points.

like image 150
Orestes_Fox Avatar answered Dec 11 '25 11:12

Orestes_Fox



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!