How to track URL's using Twitter Streaming API

Question

I want to track links using streaming API. Ideally I would like to define "http://twitter.com" and get all statuses that contain this url, both shortened (for example http://bit.ly/reXnK3 ), and full (http://twitter.com/dev/api/help.html). Any way I could achieve this? Currently it seems that the "track" method doesn't support URLs. Or am I missing something?

p.s. I know about Search API and about BackType, but I would prefer to use Streaming API, since it's real-time, and doesn't have rate limitations.

Answer 1

The track parameter is in the statuses/filter, and the result of your query should definitely include links, if links are in the Tweet. This is an example of what you get from the stream. I think your problem is that the streaming API doesn't let you track URLs as such, but will let you track keywords, or users by user id. Is the URL you're trying to track associated with some sort of keyword? Can you parse the URL and use - say - both domain name and specific URI as separate keywords (example: in the case of http://twitter.com/dev/api/help.html, the following keywords may help: twitter.com,dev,api)? What you get with this strategy has a lot of "noise" in it, but you can store the results in your database and then query for the complete URL you're trying to track. The documentation about the "track" method is as follows:

The text of the Tweet and some entity fields are considered for matches. Specifically, the text attribute of the Tweet, expanded_url and display_url for links and media, text for hashtags, and screen_name for user mentions are checked for matches.

I have never tried the streaming API, and feel more comfortable with the search API, but both of them should definitely give similar results. Try to enlarge your comma-separated list of keywords, add the URL you're trying to track as two keywords (one for the domain name, one for the specific URI), and you should get something close to what you're trying to achieve. At least, that's what I would do if I had this problem, based on what Twitter says in the documentation.