Get last Twitter mention from API with Tweepy avoiding rate limit

Question

I used to have some good working python that did auto-replies on the Tweepy stream listener, but due to changes in the Twitter API in August it no longer works.

I am re-building it by getting my most recent mention every 10 seconds (ideally it'd be less as I want to do near-instant replies), and checking if it was in the last ten seconds... if it was then the script assumes it's a new tweet and replies.

from tweepy import OAuthHandler
from tweepy import API
from datetime import datetime, time, timedelta

consumer_key = ''
consumer_secret = ''
access_token = ''
access_token_secret = ''
account_screen_name = ''
account_user_id = '897579556009332736'

auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
twitterApi = API(auth)

mentions = twitterApi.mentions_timeline(count=1)
now = datetime.now()

for mention in mentions:
    if now < (mention.created_at + timedelta(hours=1) + timedelta(seconds=10)):
        print "there's a mention in the last 10 seconds"
        # do magic reply stuff here!
    else:
        print "do nothing, no recent tweets/the last mention was more than 10 seconds ago so it isn't new"

This could work on a loop every 12 seconds; but any less and it hits the rate limit (i.e. this method above at 10 secs will hit the rate limit eventually)... so, is there a better way of just retrieving the most recent mention in order to do a reply based on the mention? I feel like I am probably doing it in a very inefficient way (e.g. this method actually gets the last 20 mentions!!) and the API probably has a better method that I could do more often without hitting rate limits?

Answer 1

Answer

Using tweepy: no

Using other methods: yes , but not for free

Explanation

First thing to understand is that twitter does allow for real-time streaming of tweets through their API. The next thing is that it is possible to use their API to track tweets containing some @TrackedUsername to keep track of mentions live. However, while the first feature is free to use through a number of libraries and wrapper APIs, the second feature is not. It is one of their features locked behind a monthly subscription to either their premium or enterprise api.

What I do know is that people do not use tweepy as distributed to utilize these services. Maybe someone out there has modified it for personal use compatible with such features, but if so I haven't found it. What people use instead is the Python Twitter API.

Speculation

As I have never used their premium services myself it would be worth reading into what I'm about to say next, but from what I understand authentication protocol for their pay-to-use services is slightly different which is what causes problems with tweepy.

Back to Facts

Essentially what this means is that searching for @TrackedUsername will return nothing at all, and searching for TrackedUsername will not return tweets containing @TrackedUsername or #TrackedUsername (unless it also contains just TrackedUsername but then you're tracking text instead of mentions). Basically because twitter wanted to get paid for this service.

If you do choose to pay for this service then the python implementation most commonly used is Python Twitter API as mentioned above. These features will allow you to track tweets that contain mentions in real-time with a higher rate limit and support for searching farther back into past tweets.