Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Query regarding pagination in tweepy (get_followers) of a particular twitter user

I am fairly new to tweepy and pagination using the cursor class. I have been trying to user the cursor class to get all the followers of a particular twitter user but I keep getting the error where it says "tweepy.error.TweepError: This method does not perform pagination" Hence I would really appreciate any help if someone could please help me achieve this task of obtaining all the followers of a particular twitter user with pagination, using tweepy. The code I have so far is as follows:

import tweepy

consumer_key='xyz'
consumer_secret='xyz'

access_token='abc'
access_token_secret='def'

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)

auth.set_access_token(access_token, access_token_secret)

api = tweepy.API(auth)


user = api.get_user('somehandle')
print user.name

followers = tweepy.Cursor(user.followers)
temp=[]
for user in followers.items():
    temp.append(user)
    print temp
#the following part works fine but that is without pagination so I will be able to retrieve at #most 100 followers
aDict =  user.followers()
for friend in aDict:
    friendDict = friend.__getstate__()
    print friendDict['screen_name']
like image 276
anonuser0428 Avatar asked Jan 10 '13 18:01

anonuser0428


1 Answers

There is a handy method called followers_ids. It returns up to 5000 followers (twitter api limit) ids for the given screen_name (or id, user_id or cursor).

Then, you can paginate these results manually in python and call lookup_users for every chunk. As long as lookup_users can handle only 100 user ids at a time (twitter api limit), it's pretty logical to set chunk size to 100.

Here's the code (pagination part was taken from here):

import itertools
import tweepy


def paginate(iterable, page_size):
    while True:
        i1, i2 = itertools.tee(iterable)
        iterable, page = (itertools.islice(i1, page_size, None),
                list(itertools.islice(i2, page_size)))
        if len(page) == 0:
            break
        yield page


auth = tweepy.OAuthHandler(<consumer_key>, <consumer_secret>)
auth.set_access_token(<key>, <secret>)

api = tweepy.API(auth)

followers = api.followers_ids(screen_name='gvanrossum')

for page in paginate(followers, 100):
    results = api.lookup_users(user_ids=page)
    for result in results:
        print result.screen_name

Hope that helps.

like image 166
alecxe Avatar answered Oct 11 '22 03:10

alecxe