Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Tweepy StreamListener extended mode

My current very simple code is supposed to use the twitter streaming API for simply printing all tweets under a certain keyword into console. However I can't figure out how to get the full text of the tweet instead of the truncated version. The program also elsewhere utilizes

    new_tweets = api.user_timeline(screen_name=username, count=200, tweet_mode='extended')

in which the last parameter specifies exactly that.

This is my current code:

class LiveTweetListener(tweepy.StreamListener):
    def on_status(self, status):
        try:
            print(status.full_text)
        except AttributeError:
            print(status.text)

    def on_error(self, status_code):
        if status_code == 420:
            print("420")
            return False

def start_stream(track):
    Listener = LiveTweetListener()
    stream = tweepy.Stream(auth=api.auth, listener=LiveTweetListener())
    stream.filter(track=track, async=True)

Passing tweet_mode='extended' with stream = tweepy.Stream doesn't seem to work

like image 236
Ladislav Louka Avatar asked Jul 31 '17 08:07

Ladislav Louka


People also ask

What is Tweet mode extended?

When using extended mode, the text attribute of Status objects returned by tweepy. API methods is replaced by a full_text attribute, which contains the entire untruncated text of the Tweet. The truncated attribute of the Status object will be False , and the entities attribute will contain all entities.

What is Tweepy StreamListener?

In Tweepy, an instance of tweepy. Stream establishes a streaming session and routes messages to StreamListener instance. The on_data method of a stream listener receives all messages and calls functions according to the message type.

How can I get more than 100 tweets on Tweepy?

If you need more than 100 Tweets, you have to use the paginator method and specify the limit i.e. the total number of Tweets that you want. Replace limit=1000 with the maximum number of tweets you want. Replace the limit=1000 with the maximum number of tweets you want (gist).

What is Tweepy rate limit?

But keep in mind that Twitter levies a rate limit on the number of requests made to the Twitter API. To be precise, 900 requests/15 minutes are allowed; Twitter feeds anything above that an error.


1 Answers

There's a bit more logic required for getting this to work. First, add tweet_mode back to the Stream, like this:

stream = tweepy.Stream(auth=api.auth, listener=LiveTweetListener(), tweet_mode='extended')

Next, change your on_status to something like this:

def on_status(self, status):
    try:
        if hasattr(status, 'retweeted_status') and hasattr(status.retweeted_status, 'extended_tweet'):
            print('retweeted: ' + status.retweeted_status.extended_tweet['full_text'])
        if hasattr(status, 'extended_tweet'):
            print('extended_tweet: ' + status.extended_tweet['full_text'])
        else:
            print('text: ' + status.text)
    except AttributeError:
        print('attribute error: ' + status.text)

Here's why this works:

  1. If the original tweet was retweeted, status.retweeted_tweet.extended_tweet.full_text holds the text.
  2. If this is an extended mode tweet, you can find the text in status.extended_tweet.full_tweet.
  3. Otherwise, this is a classic tweet and you can find the text in status.text.

While full_text doesn't appear as an extended_tweet attribute, it is part of the JSON payload, when available, which is why you can still access it via ['full_text'].

like image 69
Joe Mayo Avatar answered Oct 06 '22 20:10

Joe Mayo