How to capture RCurl verbose output

Question

I have the following request

library(RCurl)
res=getURL("http://www.google.com/search?hl=en&lr=&ie=ISO-8859-1&q=RCurl&btnG=Search",
           .opts=list(verbose = TRUE)
           )

and would like to capture the verbose output of the call (i.e., what is printed in red in the R console). I thought that the output lines are messages and are therefore printed to stderr(). The following works for messages

sink(textConnection("test","w"),type="message")
message("test message")
sink(stderr(),type="message")
test
#[1] "test message"

but not if I replace message("test message") by the RCurl request res=getURL(.....) as given above. Obviously, RCurl's output is not printed to stderr(). It's also not printed to stdout().

So, how do I capture the output?

Bonus question: Is sink(stderr(),type="message") the correct way of setting the connection back to R's default value?

Thank you for your help!

Answer 1

You need to use the debugGatherer function:

d <- debugGatherer()
x <- getURL("http://www.google.com/search?hl=en&lr=&ie=ISO-8859-1&q=RCurl&btnG=Search",
    debugfunction = d$update, verbose = TRUE)

You can then pull out the verbose content using:

d$value()

But, I imagine you just want the two following elements:

> cat(d$value()['text'])
About to connect() to www.google.com port 80 (#0)
  Trying 173.194.112.176... connected
Connected to www.google.com (173.194.112.176) port 80 (#0)
Connection #0 to host www.google.com left intact
Closing connection #0

> cat(d$value()['headerIn'])
HTTP/1.1 200 OK

Date: Thu, 14 Nov 2013 19:54:18 GMT

Expires: -1

Cache-Control: private, max-age=0

Content-Type: text/html; charset=ISO-8859-1

Set-Cookie: PREF=ID=783ad15e124023b0:FF=0:TM=1384458858:LM=1384458858:S=GuYBk1a3SfTJBIjh; expires=Sat, 14-Nov-2015 19:54:18 GMT; path=/; domain=.google.com

Set-Cookie: NID=67=sNsGhMCgjGZFtILEodYKCjxsi0Yio3oSA4xHakDGVHQKxG-fJlY05AlYlJf4Wwcto2HY2uP5Zt2iWxA4Dt0KUWxq14J-F-KvJ38zoBhWBWNxm6Ju0Oupl8gj41USR0PB; expires=Fri, 16-May-2014 19:54:18 GMT; path=/; domain=.google.com; HttpOnly

P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."

Server: gws

X-XSS-Protection: 1; mode=block

X-Frame-Options: SAMEORIGIN

Transfer-Encoding: chunked