Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using rvest, is it possible to click a tab that activates a div and reveals new content for scraping

I'm new to rvest and I'm trying to determine if its possible to use rvest to click a tab that activates a div so that data can be scraped. I've been reading the rvest documentation on cran and have not read anything that talks about clicking links, buttons or tabs.

The site that I'm interested in scraping is: touch.tvg.com

From the home page I'd like to click the race button (Again, how do you clik buttons in rvest)

Next, I'd like to select an upcoming race. This should redirect me to the selected race's url. For example: The 10th race at Hoosier

Once one the race page, I'd like to click the Pool tab and scrape the pool information.

I've attached some screen shots below. Any suggestion, guidance is appreciated.

From Home Page - Click Races Button Home Page - Click Races

From Races Page - Click an upcoming Race Races Page - Click an upcoming race

From The Particular Race Page - Click The Pool Tab From the Race Page - Click the pool tab

With the Pool data (Div) visible - Scrape the Pool Data. With Pool data visible - Scrape Pool Data

like image 307
Mutuelinvestor Avatar asked Jul 14 '16 01:07

Mutuelinvestor


People also ask

How do you scrape an Rvest?

In general, web scraping in R (or in any other language) boils down to the following three steps: Get the HTML for the web page that you want to scrape. Decide what part of the page you want to read and find out what HTML/CSS you need to select it. Select the HTML and analyze it in the way you need.

What is the purpose of Rvest package in R?

Overview. rvest helps you scrape (or harvest) data from web pages. It is designed to work with magrittr to make it easy to express common web scraping tasks, inspired by libraries like beautiful soup and RoboBrowser.

What is Rvest?

rvest is new package that makes it easy to scrape (or harvest) data from html web pages, inspired by libraries like beautiful soup. It is designed to work with magrittr so that you can express complex operations as elegant pipelines composed of simple, easily understood pieces.


1 Answers

RSelenium seems to offer all the functionality needed to harvest the data of interest. The best results might be achieved by combining the strength of rselenium with those of rvest.

Thank to everyone for their comments.

like image 187
Mutuelinvestor Avatar answered Nov 04 '22 01:11

Mutuelinvestor