library("rvest")
url <- "myurl.com"
tables<- url %>%
read_html() %>%
html_nodes(xpath='//*[@id="pageContainer"]/table[1]') %>%
html_table(fill = T)
tables[[1]]
html content of the cell is like this
<td><a href="http://somelink.com" target="_blank">Click Here</a></td>
But in scraped html I only get ,
Click Here
You could handle this by editing rvest::html_table
with trace
.
Example of existing behaviour:
library(rvest)
x <- "https://en.wikipedia.org/wiki/Academy_Award_for_Best_Picture" %>%
read_html() %>%
html_nodes("#mw-content-text > table:nth-child(55)")
html_table(x)
#[[1]]
# Film Production company(s) Producer(s)
#1 The Great Ziegfeld Metro-Goldwyn-Mayer Hunt Stromberg
#2 Anthony Adverse Warner Bros. Henry Blanke
#3 Dodsworth Goldwyn, United Artists Samuel Goldwyn and Merritt Hulbert
#4 Libeled Lady Metro-Goldwyn-Mayer Lawrence Weingarten
#5 Mr. Deeds Goes to Town Columbia Frank Capra
#6 Romeo and Juliet Metro-Goldwyn-Mayer Irving Thalberg
#7 San Francisco Metro-Goldwyn-Mayer John Emerson and Bernard H. Hyman
#8 The Story of Louis Pasteur Warner Bros. Henry Blanke
#9 A Tale of Two Cities Metro-Goldwyn-Mayer David O. Selznick
#10 Three Smart Girls Universal Joe Pasternak and Charles R. Rogers
html_table
essentially extracts the cells of the html table and runs html_text
on them. All we need to do is replace that by extracting the <a>
tag from each cell and running html_attr(., "href")
instead.
trace(rvest:::html_table.xml_node, quote({
values <- lapply(lapply(cells, html_node, "a"), html_attr, name = "href")
values[[1]] <- html_text(cells[[1]])
}), at = 14)
New behaviour:
html_table(x)
#Tracing html_table.xml_node(X[[i]], ...) step 14
#[[1]]
# Film Production company(s) Producer(s)
#1 /wiki/The_Great_Ziegfeld NA /wiki/Hunt_Stromberg
#2 /wiki/Anthony_Adverse NA /wiki/Henry_Blanke
#3 /wiki/Dodsworth_(film) NA /wiki/Samuel_Goldwyn
#4 /wiki/Libeled_Lady NA /wiki/Lawrence_Weingarten
#5 /wiki/Mr._Deeds_Goes_to_Town NA /wiki/Frank_Capra
#6 /wiki/Romeo_and_Juliet_(1936_film) NA /wiki/Irving_Thalberg
#7 /wiki/San_Francisco_(1936_film) NA /wiki/John_Emerson_(filmmaker)
#8 /wiki/The_Story_of_Louis_Pasteur NA /wiki/Henry_Blanke
#9 /wiki/A_Tale_of_Two_Cities_(1935_film) NA /wiki/David_O._Selznick
#10 /wiki/Three_Smart_Girls NA /wiki/Joe_Pasternak
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With