Get a clean summary of nlme.lme() or lme4.lmer() in RPy

Question

I am interfacing with the nlme and lme4 R functions via RPy, and I would like to get access to an output summary from my python console.

I run the following code:

test1=nlme.lme(r.formula('Pupil~CoI*Time'), random=r.formula('~1|ID'),data=dfr)
test2=nlme.lme(r.formula('Pupil~CoI*measurement'),random=r.formula('~1|ID'),data=dfr)
test1_sum= r.summary(test1)
test2_sum= r.summary(test2)
print test1_sum
print test2_sum

for nlme, and this for lme4:

test1=lme4.lmer(r.formula('Pupil~CoI*Time+(1|ID)'),data=dfr)
test2=lme4.lmer(r.formula('Pupil~CoI*measurement+(1|ID)'),data=dfr)
test1_sum= r.summary(test1)
test2_sum= r.summary(test2)
print test1_sum
print test2_sum

To get a code snippet with data and explicit imports, please refer to this IPython notebook.

In all cases, I get a huge amount of print output which includes a horridly long section looking like:

Data: structure(list(CoI = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L,  1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,  1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,  1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,  1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L,  1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L ......

I would like to get a summary more along the lines of:

Random effects:
 Formula: ~1 | ID
        (Intercept)  Residual
StdDev:   0.2201214 0.1199874

Fixed effects: Pupil ~ CoI * measurement 
                         Value  Std.Error   DF   t-value p-value
(Intercept)          1.2068660 0.06369911 5769 18.946357       0
CoIhard             -0.0394413 0.00629117 5769 -6.269306       0
measurement         -0.0002743 0.00003207 5769 -8.554287       0
CoIhard:measurement  0.0005227 0.00004536 5769 11.524511       0
 Correlation: 
                    (Intr) CoIhrd msrmnt
CoIhard             -0.049              
measurement         -0.060  0.612       
CoIhard:measurement  0.043 -0.865 -0.707

Standardized Within-Group Residuals:
        Min          Q1         Med          Q3         Max 
-9.86773055 -0.37638950  0.02085029  0.43203795  4.97364143 

Number of Observations: 5784
Number of Groups: 12 

(which is included in what I get, but only comes thousands of entries after the above) How do I achieve that?

Answer 1

The right way is to use the .rx2() method, a number of different ways of using it:

In [43]:

print test2_sum.names
Unable to unlink tempfile c:\docume~1\x60t\locals~1\temp\tmpnhw4n4
 [1] "methTitle"    "objClass"     "devcomp"      "isLmer"       "useScale"    

 [6] "logLik"       "family"       "link"         "ngrps"        "coefficients"

[11] "sigma"        "vcov"         "varcor"       "AICtab"       "call"        

[16] "residuals"   

In [44]:

print test2_sum.rx2('vcov') # to access R type print out
Unable to unlink tempfile c:\docume~1\x60t\locals~1\temp\tmpebn3f1
4 x 4 Matrix of class "dpoMatrix"

                      (Intercept)     CoIhard  measurement CoIhard:measurement

(Intercept)         93253.4275120 -80.6588422 -0.503069702         0.503069702

CoIhard               -80.6588422 161.3176844  0.503069702        -1.006139404

measurement            -0.5030697   0.5030697  0.004192248        -0.004192248

CoIhard:measurement     0.5030697  -1.0061394 -0.004192248         0.008384495

In [45]:

print test2_sum.rx2('varcor') # to access R type print out
Unable to unlink tempfile c:\docume~1\x60t\locals~1\temp\tmpcad6ld
 Groups   Name        Std.Dev.

 ID       (Intercept) 1057.39 

 Residual              242.24 

In [46]:

list(test2_sum.rx2('varcor')) # to get the values
Out[46]:
[<Matrix - Python:0x0782CEB8 / R:0x0E97FB28>
[1118073.223847]]
In [47]:

list(test2_sum.rx2('varcor')[0]) # to get the values
Out[47]:
[1118073.2238471208]

You will get rid of most of the things by skipping the calls and residuals, try:

for i, v in enumerate(list(test2_sum.names)):
    if v not in ['call', 'residuals']:
        print '%s========================================================='%i, v
        print test2_sum.rx2(v)

Additional Edit:

I think the best approach to access the tTable of lme4 result (using rpy2), would be to convert it to a pandas DataFrame:

In [73]:

print com.convert_robj(test2_sum.rx2('tTable'))
                           Value   Std.Error    DF    t-value       p-value
(Intercept)          2480.515542  305.374210  5769   8.122872  5.521357e-16
CoIhard               -90.840336   12.701090  5769  -7.152169  9.602962e-13
measurement            -0.288709    0.064748  5769  -4.458998  8.390496e-06
CoIhard:measurement     1.049136    0.091567  5769  11.457595  4.546122e-30

[4 rows x 5 columns]

The print output does not matches exactly with R print out, but it is very easy to get it done:

In [87]:

print test2_sum.rx2('tTable').__str__().replace('\r\n\r\n', '\n')

                           Value    Std.Error   DF   t-value      p-value
(Intercept)         2480.5155423 305.37420990 5769  8.122872 5.521357e-16
CoIhard              -90.8403359  12.70108989 5769 -7.152169 9.602962e-13
measurement           -0.2887093   0.06474757 5769 -4.458998 8.390496e-06
CoIhard:measurement    1.0491363   0.09156689 5769 11.457595 4.546122e-30