How to get the list of all Yahoo Finance mutual funds in R?

Question

I want to get the list of all the mutual funds that are available through Yahoo Finance into R. There is a stockSymbols function in the TTR package, but it does not seem to get the mutual funds.

Thanks,

Answer 1

I do not think Yahoo provide a list of all mutual funds they have data for (similarly, they do not provide a list of the stocks they cover). You could download the list from the website you mention in the comments, loop through all the funds, retrieve the corresponding "Profile" page from Yahoo, and extract the information you need -- the "Category" field seems to be the closest thing to the "sector and industry" you want.

# Read the list of funds
# I assume the file was downloaded manually from 
#   http://www.eoddata.com/Data/symbollist.aspx?e=USMF
# This requires registration (free).
d <- read.delim( "USMF.txt", stringsAsFactors = FALSE )

# Retrieve the profile page, for each of the funds.
# It takes 1 second for each, and there are 24,000 of them:
# this may take more than 6 hours.
library(RCurl)
library(stringr)
d$Category <- ""
for( i in seq_len(nrow(d)) ) {
  try({
    url <- paste0("http://uk.finance.yahoo.com/q/pr?s=", d$Symbol[i])
    cat( url, " " )
    profile <- getURL(url)
    row  <- str_extract(profile, "Category.*?</tr>")
    cell <- str_extract(row,     "<td.*</td>"      )
    d$Category[i] <- str_replace_all( cell, "<.*?>", "" )
    cat( d$Category[i], "\n" )
  })
}
head(d)