R Authentication error when looping search_tweets function (rtweet package) over vector of Twitter handles

Question

I've created a loop that runs through a vector of Twitter handles, and collects tweets from them using the search_tweets function from the rtweet package.

Downloading the latest version of rtweet

## install devtools package if it's not already
if (!requireNamespace("devtools", quietly = TRUE)) {
install.packages("devtools")
}

## install dev version of rtweet from github
devtools::install_github("mkearney/rtweet")

library(rtweet)

Creating token

## autheticate via web browser
token=create_token(
app = "My_app",
consumer_key = "My Consumer Key",
consumer_secret = 
"My Secret Code",set_renv = FALSE)

Here are my my Twitter handles, stored in a vector

twitter_handles=c("@realDonaldTrump","@HillaryClinton","@MittRomney")

Then I loop through these handles, and store results of each handle as a unique dataframe

#Loop through the twitter handles & store the results as individual dataframes
for(handle in twitter_handles) {

  result <- search_tweets(handle, n = 3500 , include_rts = FALSE,retryonratelimit = TRUE)

if(length(result) != 0){

  result$`Twitter Handle` <- handle
  result$Source <- "Search"

  df_name <- paste(tolower(substring(handle, 2)),"_search")

  if(exists(df_name)) {
    assign(df_name, unique(rbind(get(df_name), result)))
  } else {
    assign(df_name, result)
  }
  }

  }

However when I do this, I get an error message

Warning: 32 - Could not authenticate you. Error in vector("list", ntimes) : invalid 'length' argument

However I don't think this is an authentication problem, because when I try with a random keyword/hashtag, I get results

data <- search_tweets("#rstats", n = 10, include_rts = FALSE,token = token)

My loop was working fine, but recently it has started throwing errors. Any ideas on why this is happening, and whether there is a fix?

Your help is highly appreciated!

Answer 1

On the first error:

"Warning: 32 - Could not authenticate you. Error in vector("list", ntimes) : invalid 'length' argument"

in general, you'll encounter this error if you are using an older version of rtweet.

Why?

When Twitter updates their API's, they sometimes change the structure of the API GET requests. rtweet has to reformat their requests each time this happens, requiring you to use the latest version of rtweet to maintain connectivity to the Twitter API. Interestingly, some API calls will still succeed as those calls to the Twitter API are unchanged.

The error you encountered is referenced @ TwitterCommunity.com.

Obtaining the latest version of rtweet

To obtain the latest version of rtweet you can use the devtools package (after installing it).

## install devtools package if it's not already
if (!requireNamespace("devtools", quietly = TRUE)) {
  install.packages("devtools")
}

## install dev version of rtweet from github
devtools::install_github("mkearney/rtweet")

## load rtweet package
library(rtweet)

Related Errors

Error in vector("list", n.times) : invalid 'length' argument In addition: Warning message: rate limit exceeded.

A good place to look for tracked errors is the Github package tracking list on github for the rtweet package.

• https://github.com/mkearney/rtweet/issues/109

Token Security

This is an aside comment but my sense is you may also want to share your complete code without your API keys. You can do this in R by using ~/.Reviron.

# Reload .Renviron
# Do this to capture any edits to Environment variables
readRenviron("~/.Renviron")

# Generate a token
token <- create_token(
  app = "rtweet_51672443_test_application",
  consumer_key = Sys.getenv("RTWEET_CONSUMER_KEY"),
  consumer_secret = Sys.getenv("RTWEET_CONSUMER_SECRET_KEY")
) 

where .Renviron contains:

RTWEET_CONSUMER_KEY="<Insert Consumer Key obtained from Titter>"
RTWEET_CONSUMER_SECRET_KEY="<Insert Consumer Secret Key obtained from Titter>"

I hope the above helps point you in the right direction.