Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Not able to get Country of a Tweet - Twython API

I am using the following code to collect Tweets pertaining to a certain topic but in all the tweets that I have extracted the 'places' attribute is None. Am I doing something wrong? Also, the code is meant to extract existing tweets and I do not need streaming api solution and not looking for this solution of streaming API : https://www.quora.com/How-can-I-get-a-stream-of-tweets-from-a-particular-country-using-Twitter-API

api =   Twython(consumer_key, consumer_secret, access_key, access_secret)

tweets                          =   []
MAX_ATTEMPTS                    =   200
COUNT_OF_TWEETS_TO_BE_FETCHED   =   10000
in_max_id = sys.argv[1]
next_max_id = ''
for i in range(0,MAX_ATTEMPTS):

    if(COUNT_OF_TWEETS_TO_BE_FETCHED < len(tweets)):
        break # we got 500 tweets... !!

    #----------------------------------------------------------------#
    # STEP 1: Query Twitter
    # STEP 2: Save the returned tweets
    # STEP 3: Get the next max_id
    #----------------------------------------------------------------#

    # STEP 1: Query Twitter
    if(0 == i):
        # Query twitter for data. 
        results    = api.search(q="#something",count='100',lang='en',max_id=in_max_id,include_entities='true',geo= True)
    else:
        # After the first call we should have max_id from result of previous call. Pass it in query.
        results    = api.search(q="#something",include_entities='true',max_id=next_max_id,lang='en',geo= True)

    # STEP 2: Save the returned tweets
    for result in results['statuses']:

        temp = ""
        tweet_text = result['text']
        temp += tweet_text.encode('utf-8') + " "
        hashtags = result['entities']['hashtags']
        for i in hashtags:
            temp += i['text'].encode('utf-8') + " " 
        print result
        #temp += i["place"]["country"] + "\n"
        #output_file.write(temp)




    # STEP 3: Get the next max_id
    try:
        # Parse the data returned to get max_id to be passed in consequent call.
        next_results_url_params    = results['search_metadata']['next_results']
        next_max_id        = next_results_url_params.split('max_id=')[1].split('&')[0]
    except:
        # No more next pages
        break
like image 944
silent_dev Avatar asked Dec 12 '15 13:12

silent_dev


People also ask

Can I use Twitter API without authentication?

This means that the only requests you can make to a Twitter API must not require an authenticated user. With application-only authentication, you can perform actions such as: Pull user timelines. Access friends and followers of any account.

How far back can you go with Twitter API?

30-Day Search API provides data from the previous 30 days. Full-Archive Search API provides complete and instant access to the full corpus of Twitter data dating all the way back to the first Tweet in March 2006.

What does the Twitter API allow access to?

Our API platform provides broad access to public Twitter data that users have chosen to share with the world. We also support APIs that allow users to manage their own non-public Twitter information (e.g., Direct Messages) and provide this information to developers whom they have authorized to do so.


2 Answers

The short answer is, No, you are doing nothing wrong. The reason why all place tags are empty is because statistically they are very unlikely to contain data. Only about 1% of all tweets have data in their place tag. This is because users rarely tweet their location. Location is off by default.

Download 100 or more tweets and you probably will find place tag data.

like image 166
Jonas Avatar answered Sep 30 '22 08:09

Jonas


If place field is a MUST for all the tweet that you app will process, then you can limit your search over a place to make sure all the result will definitely have it.

You can doing so by setting geocode (latitude,longitude,radius[km/mi]) parameter, to limit your search within an area.

An example such request via Twython is:

geocode = '25.032341,55.385557,100mi'
api.search(q="#something",count='100',lang='en',include_entities='true',geocode=geocode)
like image 36
R‌‌‌.. Avatar answered Sep 30 '22 07:09

R‌‌‌..