How do I read a web page in Racket?

Question

All of the information I can find online is about writing web servers, but there seems to be very little about functions useful for web clients. Ideally, I would like the function to look something like this:

(website "http://www.google.com")

And return a string containing the entire web page, but I would be happy with anything that works.

Answer 1

Here's a simple program that looks like it does what you want:

#lang racket

(require net/url)

(port->bytes
 (get-pure-port (string->url "http://www.google.com")))

If you're like me, you probably also want to parse it into an s-expression. Neil Van Dyke's neil/html-parsing does this:

#lang racket

(require (planet neil/html-parsing:2:0)
         net/url)

(html->xexp
 (get-pure-port (string->url "http://www.google.com")))

Note that since this program refers to a planet package, running this program for the first time will download and install the htmlprag package. Building the documentation could take quite a while. That's an one-time cost, though, and running the program again shouldn't take more than a few seconds.