I am trying to download some tweets for research purpose the code worked very well until a few days ago.
Error Message
> Traceback (most recent call last):
>
> File "<ipython-input-1-10547a7aea4c>", line 1, in <module>
> runfile('F:/twitter_locations_n_scraper/02_twitterscrapping.py', wdir='F:/twitter_locations_n_scraper')
>
> File
> "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py",
> line 668, in runfile
> execfile(filename, namespace)
>
> File
> "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py",
> line 108, in execfile
> exec(compile(f.read(), filename, 'exec'), namespace)
>
> File "F:/twitter_locations_n_scraper/02_twitterscrapping.py", line
> 70, in <module>
> for item in cursor.items(1000): # Remove the limit to 1000
>
> File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\cursor.py",
> line 49, in __next__
> return self.next()
>
> File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\cursor.py",
> line 197, in next
> self.current_page = self.page_iterator.next()
>
> File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\cursor.py",
> line 108, in next
> data = self.method(max_id=self.max_id, parser=RawParser(), *self.args, **self.kargs)
>
> File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\binder.py",
> line 250, in _call
> return method.execute()
>
> File "C:\ProgramData\Anaconda3\lib\site-packages\tweepy\binder.py",
> line 234, in execute
> raise TweepError(error_msg, resp, api_code=api_error_code)
>
> TweepError: Twitter error response: status code = 400
I did try the supposed solution
api.update_status('Test')
But this does not work for me, I did change the keys and still I am facing the same error, Reinstalled the Tweepy Package twice and yet the error does not go.
Appreciate the help in solving the error, Thanks in advance.
UPDATE: Code that I am getting error
import tweepy
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
import pandas as pd
import json
import csv
import sys
import time
path="F:\\twitter_locations_n_scraper\\"
ckey = '*****'
csecret = '******'
atoken = '*****'
asecret = '*****'
def toDataFrame(tweets):
# COnvert to data frame
DataSet = pd.DataFrame()
DataSet['tweetID'] = [tweet.id for tweet in tweets]
DataSet['tweetText'] = [tweet.text.encode('utf-8') for tweet in tweets]
DataSet['tweetRetweetCt'] = [tweet.retweet_count for tweet in tweets]
DataSet['tweetFavoriteCt'] = [tweet.favorite_count for tweet in tweets]
DataSet['tweetSource'] = [tweet.source for tweet in tweets]
DataSet['tweetCreated'] = [tweet.created_at for tweet in tweets]
DataSet['userID'] = [tweet.user.id for tweet in tweets]
DataSet['userScreen'] = [tweet.user.screen_name for tweet in tweets]
DataSet['userName'] = [tweet.user.name for tweet in tweets]
DataSet['userCreateDt'] = [tweet.user.created_at for tweet in tweets]
DataSet['userDesc'] = [tweet.user.description for tweet in tweets]
DataSet['userFollowerCt'] = [tweet.user.followers_count for tweet in tweets]
DataSet['userFriendsCt'] = [tweet.user.friends_count for tweet in tweets]
DataSet['userLocation'] = [tweet.user.location for tweet in tweets]
DataSet['userTimezone'] = [tweet.user.time_zone for tweet in tweets]
DataSet['Coordinates'] = [tweet.coordinates for tweet in tweets]
DataSet['GeoEnabled'] = [tweet.user.geo_enabled for tweet in tweets]
DataSet['Language'] = [tweet.user.lang for tweet in tweets]
tweets_place= []
#users_retweeted = []
for tweet in tweets:
if tweet.place:
tweets_place.append(tweet.place.full_name)
else:
tweets_place.append('null')
DataSet['TweetPlace'] = [i for i in tweets_place]
#DataSet['UserWhoRetweeted'] = [i for i in users_retweeted]
return DataSet
OAUTH_KEYS = {'consumer_key':ckey, 'consumer_secret':csecret,'access_token_key':atoken, 'access_token_secret':asecret}
#auth = tweepy.OAuthHandler(OAUTH_KEYS['consumer_key'], OAUTH_KEYS['consumer_secret'])
auth = tweepy.AppAuthHandler('******', '*******')
api = tweepy.API(auth, wait_on_rate_limit=True,wait_on_rate_limit_notify=True)
if (not api):
print ("Can't Authenticate")
sys.exit(-1)
else:
df=pd.read_csv(path+"00_all_location_India_full.csv")
print ("Scraping data now") # Enter lat and long and radius in Kms
for i in range(0,len(df)):
x="%s,%s,30km"%(df['latitude'][i],df['longitude'][i]) #,q='motox4', since='2018-08-16',until='2018-08-18',
cursor = tweepy.Cursor(api.search,since='2018-08-23',until='2018-08-24',lang='en',count=1000)
results=[]
print (i)
for item in cursor.items(1000): # Remove the limit to 1000
results.append(item)
DataSet = toDataFrame(results)
DataSet['latitude']=df['latitude'][i]
DataSet['longitude']=df['longitude'][i]
DataSet['radius']=30
del DataSet['Coordinates']
DataSet.to_csv(path+'cities_aug23.csv',encoding='utf-8',index=False,mode="a")
This is interesting question , actually , the way you wrote your question have lead me to write this answer ,
you have mention that it was working but not any more , so this could happen in my opinion for 2 reason ...
browser have depreciated some function within the browser which prevents the script to work . maybe you should try to preventDefault() function on page load and see if any error appears. try also changing your internet connection , or restart your modem Also make sure your loading this from https website not http .
or these path's which you listed on your code , probably there is no permission access to read it , i have big doubts about this path , however you check all path's and try to close anything that might block connections to these path's .
C:\ProgramData\Anaconda3\lib\
because it require windows authentication to access it . try minimize your UAC permission from control panel . you may also check the security tab by right clicking on the folder and selecting properties .
so , revert everything back as it is and check the above . please let me know the result as i can edit it accordingly to the results of your replay
hope my post helps you .
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With