How to get the link inside html_table using rvest?

Question

library("rvest")
url <- "myurl.com"
tables<- url %>%
      read_html() %>%
      html_nodes(xpath='//*[@id="pageContainer"]/table[1]') %>%
      html_table(fill = T)
tables[[1]]

html content of the cell is like this

<td><a href="http://somelink.com" target="_blank">Click Here</a></td>

But in scraped html I only get ,

Click Here

Answer 1

You could handle this by editing rvest::html_table with trace.

Example of existing behaviour:

library(rvest)
x <- "https://en.wikipedia.org/wiki/Academy_Award_for_Best_Picture" %>% 
  read_html() %>% 
  html_nodes("#mw-content-text > table:nth-child(55)")

html_table(x)
#[[1]]
#                         Film   Production company(s)                         Producer(s)
#1          The Great Ziegfeld     Metro-Goldwyn-Mayer                      Hunt Stromberg
#2             Anthony Adverse            Warner Bros.                        Henry Blanke
#3                   Dodsworth Goldwyn, United Artists  Samuel Goldwyn and Merritt Hulbert
#4                Libeled Lady     Metro-Goldwyn-Mayer                 Lawrence Weingarten
#5      Mr. Deeds Goes to Town                Columbia                         Frank Capra
#6            Romeo and Juliet     Metro-Goldwyn-Mayer                     Irving Thalberg
#7               San Francisco     Metro-Goldwyn-Mayer   John Emerson and Bernard H. Hyman
#8  The Story of Louis Pasteur            Warner Bros.                        Henry Blanke
#9        A Tale of Two Cities     Metro-Goldwyn-Mayer                   David O. Selznick
#10          Three Smart Girls               Universal Joe Pasternak and Charles R. Rogers

html_table essentially extracts the cells of the html table and runs html_text on them. All we need to do is replace that by extracting the <a> tag from each cell and running html_attr(., "href") instead.

trace(rvest:::html_table.xml_node, quote({ 
  values      <- lapply(lapply(cells, html_node, "a"), html_attr, name = "href")
  values[[1]] <- html_text(cells[[1]])
}), at = 14)

New behaviour:

html_table(x)
#Tracing html_table.xml_node(X[[i]], ...) step 14 
#[[1]]
#                                     Film Production company(s)                    Producer(s)
#1                /wiki/The_Great_Ziegfeld                    NA           /wiki/Hunt_Stromberg
#2                   /wiki/Anthony_Adverse                    NA             /wiki/Henry_Blanke
#3                  /wiki/Dodsworth_(film)                    NA           /wiki/Samuel_Goldwyn
#4                      /wiki/Libeled_Lady                    NA      /wiki/Lawrence_Weingarten
#5            /wiki/Mr._Deeds_Goes_to_Town                    NA              /wiki/Frank_Capra
#6      /wiki/Romeo_and_Juliet_(1936_film)                    NA          /wiki/Irving_Thalberg
#7         /wiki/San_Francisco_(1936_film)                    NA /wiki/John_Emerson_(filmmaker)
#8        /wiki/The_Story_of_Louis_Pasteur                    NA             /wiki/Henry_Blanke
#9  /wiki/A_Tale_of_Two_Cities_(1935_film)                    NA        /wiki/David_O._Selznick
#10                /wiki/Three_Smart_Girls                    NA            /wiki/Joe_Pasternak