Get video from tweet using Twitter API

Question

I am trying to use the Twitter API to import a video from a given tweet. However, when I use the statuses/show endpoint, it doesn't return any extended entity for the video as it would an image, but instead returns a url entity linking to some video container embed with a video player containing an obscure link to the video.

Here is an example:

I am trying to import the tweet at https://twitter.com/NHL/status/633987786018717696

Using the Twitter API's statuses/show endpoint and the tweet id, I get this response:

{
  "created_at": "Wed Aug 19 13:04:01 +0000 2015",
  "id": 633987786018717700,
  "id_str": "633987786018717696",
  "text": "The offseason has us missing all of our fans, even the wacky ones... especially the wacky ones. #IsItOctoberYet?\nhttps://t.co/v4UGDQpa61",
  "source": "<a href="http://twitter.com" rel="nofollow">Twitter Web Client</a>",
  "truncated": false,
  "in_reply_to_status_id": null,
  "in_reply_to_status_id_str": null,
  "in_reply_to_user_id": null,
  "in_reply_to_user_id_str": null,
  "in_reply_to_screen_name": null,
  "user":  {
    "id": 50004938,
    "id_str": "50004938",
    "name": "NHL",
    "screen_name": "NHL",
    "location": "30 cities across U.S. & Canada",
    "description": "The official source of everything you need and want to know from the National Hockey League. Read before tweeting us: http://t.co/JlyVXSpqMn",
    "url": "http://t.co/VI8RlwuVr9",
    "entities":  {
      "url":  {
        "urls":  [
           {
            "url": "http://t.co/VI8RlwuVr9",
            "expanded_url": "http://www.NHL.com",
            "display_url": "NHL.com",
            "indices":  [
              0,
              22
            ]
          }
        ]
      },
      "description":  {
        "urls":  [
           {
            "url": "http://t.co/JlyVXSpqMn",
            "expanded_url": "http://nhl.com/socialmediapolicy",
            "display_url": "nhl.com/socialmediapol…",
            "indices":  [
              118,
              140
            ]
          }
        ]
      }
    },
    "protected": false,
    "followers_count": 4130811,
    "friends_count": 2646,
    "listed_count": 18479,
    "created_at": "Tue Jun 23 15:24:18 +0000 2009",
    "favourites_count": 909,
    "utc_offset": -14400,
    "time_zone": "Eastern Time (US & Canada)",
    "geo_enabled": true,
    "verified": true,
    "statuses_count": 87436,
    "lang": "en",
    "contributors_enabled": false,
    "is_translator": false,
    "is_translation_enabled": true,
    "profile_background_color": "000000",
    "profile_background_image_url": "http://pbs.twimg.com/profile_background_images/378800000139631457/fd-xWa9G.jpeg",
    "profile_background_image_url_https": "https://pbs.twimg.com/profile_background_images/378800000139631457/fd-xWa9G.jpeg",
    "profile_background_tile": false,
    "profile_image_url": "http://pbs.twimg.com/profile_images/534776558238437376/yxrm83O7_normal.jpeg",
    "profile_image_url_https": "https://pbs.twimg.com/profile_images/534776558238437376/yxrm83O7_normal.jpeg",
    "profile_banner_url": "https://pbs.twimg.com/profile_banners/50004938/1435502670",
    "profile_link_color": "040CDE",
    "profile_sidebar_border_color": "FFFFFF",
    "profile_sidebar_fill_color": "2E2E2E",
    "profile_text_color": "0F5A80",
    "profile_use_background_image": true,
    "has_extended_profile": false,
    "default_profile": false,
    "default_profile_image": false,
    "following": true,
    "follow_request_sent": false,
    "notifications": false
  },
  "geo": null,
  "coordinates": null,
  "place": null,
  "contributors": null,
  "is_quote_status": false,
  "retweet_count": 865,
  "favorite_count": 1342,
  "entities":  {
    "hashtags":  [
       {
        "text": "IsItOctoberYet",
        "indices":  [
          96,
          111
        ]
      }
    ],
    "symbols":  [],
    "user_mentions":  [],
    "urls":  [
       {
        "url": "https://t.co/v4UGDQpa61",
        "expanded_url": "https://amp.twimg.com/v/2a0210d1-4d39-4665-a749-ea34f8efef08",
        "display_url": "amp.twimg.com/v/2a0210d1-4d3…",
        "indices":  [
          113,
          136
        ]
      }
    ]
  },
      "favorited": false,
      "retweeted": false,
      "possibly_sensitive": false,
      "possibly_sensitive_appealable": false,
      "lang": "en"
    }

Upon following the URL, the source of the video tag is https://amp.twimg.com/amplify-web-player/prod/source.html?vmap_url=https%3A%2F%2Famp.twimg.com%2Fprod%2Fmultibr_v_1%2Fvmap%2F2015%2F08%2F20%2F13%2F609fc2af-1d06-4894-80be-1c231f97557a%2Fa69baa90-58de-4d1d-b2dc-2c3ef1ab9b35.vmap&duration=91.958&image_src=https%3A%2F%2Famp.twimg.com%2Fprod%2Fdefault%2F2015%2F08%2F20%2F13%2Fe8f0b317-ba48-4cec-bf2c-da4598e2b46b_poster-67227.jpg&content_id=609fc2af-1d06-4894-80be-1c231f97557a&page=amplify_card

How do I extract this video file from the tweet if they do not supply an external_entity for it?

Answer 1

The Twitter API has now changed and the videos are stored in the extended_entities object. There could be multiple sources depending on bitrate. This is how to retrieve the one with the highest bitrate:

var bitrate = 0;
var hq_video_url;
for (var j=0; j<tweet.extended_entities.media[0].video_info.variants.length; j++) {
    if (tweet.extended_entities.media[0].video_info.variants[j].bitrate) {
        if (tweet.extended_entities.media[0].video_info.variants[j].bitrate > bitrate) {
            bitrate = tweet.extended_entities.media[0].video_info.variants[j].bitrate;
            hq_video_url = tweet.extended_entities.media[0].video_info.variants[j].url;
        }
    }
}

Answer 2

Workaround for GIFs here!

In Twitter API V2, it is not possible to fetch GIF and video URLs currently. I know it is too silly. But one workaround is to fetch the preview image of the content and construct the media URL by hand.

Let's say we want to get GIF URL of the following tweet via V2 API: https://twitter.com/FloodSocial/status/870042717589340160

When we fetch the tweet with the following URL https://api.twitter.com/2/tweets/870042717589340160?tweet.fields=attachments,author_id,created_at,entities,id,text&media.fields=preview_image_url,url&expansions=attachments.media_keys (with your bearer token of course), you will see that the response includes a preview_image_url with https://pbs.twimg.com/tweet_video_thumb/DBMDLy_U0AAqUWP.jpg

So here we can extract the DBMDLy_U0AAqUWP part from the URL, and construct the real GIF URL manually where it should be https://video.twimg.com/tweet_video/DBMDLy_U0AAqUWP.mp4

There you go. Only you need to write the extractor function.

Answer 3

When using statuses/show endpoint , add this option tweet_mode:'extended' to get the extended_entities object. This path in the response will have video urls : extended_entities.media[0].video_info.variants Example:

[
  {
    content_type: 'application/x-mpegURL',
    url: 'https://video.twimg.com/ext_tw_video/1358226.........'
  },
  {
    bitrate: 832000,
    content_type: 'video/mp4',
    url: 'https://video.twimg.com/ext_tw_video/1358226.........'
  },
  {
    bitrate: 256000,
    content_type: 'video/mp4',
    url: 'https://video.twimg.com/ext_tw_video/1358226.........'
  }
]