WebScraping in R: extract names from `href` tags

Question

This is my code:

library(rvest)
library(XML)
library(xml2)
url_imb <- 'https://www.imdb.com/search/title/?count=100&release_date=2016,2016&title_type=feature'
web_page<-read_html(url_imb)

I want to extract all Directors names related to adv_li_dr_0tags.

This is what I did: CSS SELECTOR:

directors_0<-html_text(html_nodes(web_page,"p a"))

XPATH SELECTOR:

directors_0<-html_attr(html_nodes(web_page,xpath='//p[@class=""]//a'),"href")

It is incomplete of course. But can you help me? How to extract elemnts related to a tag in href.

Answer 1

Is this what you want?

library(rvest)
library(XML)
library(xml2)
url_imb <- 'https://www.imdb.com/search/title/?count=100&release_date=2016,2016&title_type=feature'
directors <- read_html(url_imb) %>% 
  html_nodes(xpath = "//p[contains(text(),'Director')]/a[contains(@href, '_dr')]") %>% 
  html_text()

Answer 2

I would consider using a css attribute = value selector with contains operator to specify the href attribute must contain the substring adv_li_dr_ . Note I have dropped the 0 on the assumption you want all directors. If you want only the first director for each film then put the 0 in on the end. Note this should be faster and less fragile than xpath.

library(rvest)
library(magrittr)

url_imb <- 'https://www.imdb.com/search/title/?count=100&release_date=2016,2016&title_type=feature'
directors <-read_html(url_imb) %>% html_nodes('[href*=adv_li_dr_]')%>%html_text()

---

Reading:

1. Attribute selectors.