Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

HTML5 MediaSource works with some mp4 files and not with others (same codecs)

I was playing around with MediaSource API. The code is taken directly from Mozilla's example page: https://developer.mozilla.org/en-US/docs/Web/API/MediaSource/endOfStream

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8"/>
  </head>
  <body>
    <video controls></video>
    <script>
      var video = document.querySelector('video');

      var assetURL = 'frag_bunny.mp4';
      // Need to be specific for Blink regarding codecs
      // ./mp4info frag_bunny.mp4 | grep Codec
      var mimeCodec = 'video/mp4; codecs="avc1.42E01E, mp4a.40.2"';

      if ('MediaSource' in window && MediaSource.isTypeSupported(mimeCodec)) {
        var mediaSource = new MediaSource;
        //console.log(mediaSource.readyState); // closed
        video.src = URL.createObjectURL(mediaSource);
        mediaSource.addEventListener('sourceopen', sourceOpen);
      } else {
        console.error('Unsupported MIME type or codec: ', mimeCodec);
      }

      function sourceOpen (_) {
        //console.log(this.readyState); // open
        var mediaSource = this;
        var sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);
        fetchAB(assetURL, function (buf) {
          sourceBuffer.addEventListener('updateend', function (_) {
            mediaSource.endOfStream();
            video.play();
            //console.log(mediaSource.readyState); // ended
          });
          sourceBuffer.appendBuffer(buf);
        });
      };

      function fetchAB (url, cb) {
        console.log(url);
        var xhr = new XMLHttpRequest;
        xhr.open('get', url);
        xhr.responseType = 'arraybuffer';
        xhr.onload = function () {
          cb(xhr.response);
        };
        xhr.send();
      };
    </script>
  </body>
</html>

This works fine unless I use another video. I have a short test mp4 which - according to bento4's mp4info - is encoded with the same codecs as frag_bunny.mp4. It crashes at mediaSource.endOfStream(); because of

Uncaught DOMException: Failed to execute 'endOfStream' on 'MediaSource': The MediaSource's readyState is not 'open'.
    at SourceBuffer.<anonymous>

I assume that video file has to meet some specific criteria in order to work with that but I don't know them.

Can someone point me in the right direction?

Some more details about the files

frag_bunny mp4info

File:
  major brand:      mp42
  minor version:    1
  compatible brand: mp42
  compatible brand: avc1
  compatible brand: iso5

Movie:
  duration:   60095 ms
  time scale: 1000
  fragments:  yes

Found 4 Tracks
Track 1:
  flags:        7 ENABLED IN-MOVIE IN-PREVIEW
  id:           1
  type:         Audio
  duration: 60095 ms
  language: eng
  media:
    sample count: 0
    timescale:    22050
    duration:     0 (media timescale units)
    duration:     0 (ms)
    bitrate (computed): 65.075 Kbps
  Sample Description 0
    Coding:      mp4a (MPEG-4 Audio)
    Stream Type: Audio
    Object Type: MPEG-4 Audio
    Max Bitrate: 0
    Avg Bitrate: 64000
    Buffer Size: 6144
    Codecs String: mp4a.40.2
    MPEG-4 Audio Object Type: 2 (AAC Low Complexity)
    MPEG-4 Audio Decoder Config:
      Sampling Frequency: 22050
      Channels: 2
    Sample Rate: 22050
    Sample Size: 16
    Channels:    2
Track 2:
  flags:        7 ENABLED IN-MOVIE IN-PREVIEW
  id:           2
  type:         Video
  duration: 60095 ms
  language: eng
  media:
    sample count: 0
    timescale:    600
    duration:     0 (media timescale units)
    duration:     0 (ms)
    bitrate (computed): 612.178 Kbps
  display width:  640.000000
  display height: 360.000000
  Sample Description 0
    Coding:      avc1 (H.264)
    Width:       640
    Height:      360
    Depth:       24
    AVC Profile:          66 (Baseline)
    AVC Profile Compat:   e0
    AVC Level:            30
    AVC NALU Length Size: 4
    AVC SPS: [2742e01ea9181405ff2e00d418041adb0ad7bdf010]
    AVC PPS: [28de09c8]
    Codecs String: avc1.42E01E
Track 3:
  flags:        7 ENABLED IN-MOVIE IN-PREVIEW
  id:           3
  type:         Hint
  duration: 60095 ms
  language: eng
  media:
    sample count: 0
    timescale:    90000
    duration:     0 (media timescale units)
    duration:     0 (ms)
    bitrate (computed): 45.859 Kbps
  Sample Description 0
    Coding:      rtp  (RTP Hints)
Track 4:
  flags:        7 ENABLED IN-MOVIE IN-PREVIEW
  id:           4
  type:         Hint
  duration: 60095 ms
  language: eng
  media:
    sample count: 0
    timescale:    22050
    duration:     0 (media timescale units)
    duration:     0 (ms)
    bitrate (computed): 5.510 Kbps
  Sample Description 0
    Coding:      rtp  (RTP Hints)

my video's mp4info

File:
  major brand:      mp42
  minor version:    0
  compatible brand: mp42
  compatible brand: isom
  compatible brand: avc1

Movie:
  duration:   5568 ms
  time scale: 90000
  fragments:  no

Found 2 Tracks
Track 1:
  flags:        1 ENABLED
  id:           1
  type:         Video
  duration: 5533 ms
  language: und
  media:
    sample count: 166
    timescale:    90000
    duration:     498000 (media timescale units)
    duration:     5533 (ms)
    bitrate (computed): 465.670 Kbps
  display width:  560.000000
  display height: 320.000000
  frame rate (computed): 30.000
  Sample Description 0
    Coding:      avc1 (H.264)
    Width:       560
    Height:      320
    Depth:       24
    AVC Profile:          66 (Baseline)
    AVC Profile Compat:   c0
    AVC Level:            30
    AVC NALU Length Size: 4
    AVC SPS: [6742c01e9e218118534d40404050000003001000000303c8f162ee]
    AVC PPS: [68ce06cb20]
    Codecs String: avc1.42C01E
Track 2:
  flags:        3 ENABLED IN-MOVIE
  id:           2
  type:         Audio
  duration: 5568 ms
  language: eng
  media:
    sample count: 261
    timescale:    48000
    duration:     267264 (media timescale units)
    duration:     5568 (ms)
    bitrate (computed): 83.050 Kbps
  Sample Description 0
    Coding:      mp4a (MPEG-4 Audio)
    Stream Type: Audio
    Object Type: MPEG-4 Audio
    Max Bitrate: 91632
    Avg Bitrate: 83051
    Buffer Size: 280
    Codecs String: mp4a.40.2
    MPEG-4 Audio Object Type: 2 (AAC Low Complexity)
    MPEG-4 Audio Decoder Config:
      Sampling Frequency: 48000
      Channels: 1
    Sample Rate: 48000
    Sample Size: 16
    Channels:    1

Chrome version: 55.0.2883.95 (64-bit)

like image 488
Xander Avatar asked Mar 09 '23 20:03

Xander


1 Answers

t works fine if mp4 is fragmented. You can do that using bento4's mp4fragment tool which you can find here: https://www.bento4.com/documentation/mp4fragment/

like image 155
Xander Avatar answered Apr 30 '23 01:04

Xander