Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Tweepy - Exclude Retweets

Tags:

Ultimate goal is to use the tweepy api search to focus on topics (i.e docker) and to EXCLUDE retweets. I have looked at other threads that mention excluding retweets but they were completely applicable. I have tried to incorporate what I've learned into the code below but I believe the "if not" piece of code is in the wrong place. Any help is greatly appreciated.

#!/usr/bin/python
import tweepy
import csv #Import csv
import os

# Consumer keys and access tokens, used for OAuth
consumer_key = 'MINE'
consumer_secret = 'MINE'
access_token = 'MINE'
access_token_secret = 'MINE'

# OAuth process, using the keys and tokens
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)


api = tweepy.API(auth)
# Open/Create a file to append data
csvFile = open('docker1.csv', 'a')
#Use csv Writer
csvWriter = csv.writer(csvFile)


ids = set()
for tweet in tweepy.Cursor(api.search, 
                    q="docker", 
                    Since="2016-08-09", 
                    #until="2014-02-15", 
                    lang="en").items(5000000):
if not tweet['retweeted'] and 'RT @' not in tweet['text']:
    #Write a row to the csv file/ I use encode utf-8
    csvWriter.writerow([tweet.created_at, tweet.text.encode('utf-8'), tweet.favorite_count, tweet.retweet_count, tweet.id, tweet.user.screen_name])
    #print "...%s tweets downloaded so far" % (len(tweet.id))
    ids.add(tweet.id) # add new id
    print ("number of unique ids seen so far: {}",format(len(ids)))
csvFile.close()

Error Message

like image 973
hansolo Avatar asked Aug 10 '16 11:08

hansolo


People also ask

How do you delete retweets from Tweepy?

The API. unretweet() method of the API class in Tweepy module is used to un-retweet a retweeted tweet.

Is Tweepy case sensitive?

Keep in mind that the filters are case-sensitive too.

How many tweets can Tweepy extract?

Tweepy provides the convenient Cursor interface to iterate through different types of objects. Twitter allows a maximum of 3200 tweets for extraction.

What does Tweepy return?

Returns full Tweet objects for up to 100 Tweets per request, specified by the id parameter.


1 Answers

Filtering at API level:

q='your_search -filter:retweets'

read more on this here.

Dumb way is to filter in code

So tweet is an object not a JSON or dict, you should not access it like tweet['retweeted'] and tweet['text']

Instead use this line :

if not tweet.retweeted:

Or for your use case :

if (not tweet.retweeted) and ('RT @' not in tweet.text):
like image 146
harshil9968 Avatar answered Oct 10 '22 09:10

harshil9968