sintel.mp4 doesn't work in Safari 9.0.3 about videostream

[jhiesey]

So it looks like there are a few issues. The main one is fixed by jhiesey/mp4-box-encoding@acfbcd7; however, I still need to avoid using SourceBuffer.abort() due to https://bugs.webkit.org/show_bug.cgi?id=165342

[jhiesey]

@PaTiToMaSteR Media Source is only supported on iPadOS 13; not regular iOS 13 or any older iOS. See https://stackoverflow.com/questions/52999825/why-exactly-doesnt-html5-media-source-video-work-on-ios

This issue was about Mac OS Safari; I think it's working fine at this point. If not, feel free to reopen.

[jhiesey]

Yeah I know this is broken. Safari is really picky about the videos it accepts. This will take a bit of work to understand what's wrong.

[feross]

@jhiesey I know you're busy -- but do you think you'll have some time to eventually get around to this? Maybe we could raise a bounty for it from some webtorrent users, since I know there are quite a few people doing video+web seeds with webtorrent these days.

[feross]

Also, if you want to get together and work on it together sometime, I'm down!

[hacknlove]

There is something I can do to help?

[jakefb]

I am going to spend some time trying to solve this problem. Now that iOS supports WebRTC it would be great to get videos streaming.

If anyone is keen to help let me know.

[jhiesey]

@jakefb Unfortunately iOS still doesn't support Media Source Extensions, so this module isn't going to work there. Hopefully Apple will eventually add support. Maybe iOS 12?

[diegocr]

Hi there, did anyone had any success getting videostream to work on Safari? (@jakefb? /cc @jhiesey)

I have tried to debug this on my own and here are my findings:

• The initialization segments are appended to the SourceBuffer without any apparent issue, no error throw from doing so, at:

  videostream/videostream.js

  Line 56 in c730a19

  mediaSource.write(trackData.init, function (err) {

• Subsequent appendBuffer attempts does fail with InvalidStateError, triggered from these steps:

  • The <video> element gets waiting signaled.

  • MP4Remuxer.seek() procedure is initialed through pump()

    videostream/videostream.js

    Line 78 in c730a19

    var muxed = self._muxer.seek(self._elem.currentTime, !self._tracks)

  • MediaSourceStream._write() is reached to append a new segment to the SourceBuffer, this one went fine apparently, however by listening to the error event on the SourceBuffer we can catch that the append actually failed, despite that behind a try/catch where nothing is caught at this point.

    https://github.com/feross/mediasource/blob/d8d351a45f8783a53031c3fa1af83f01437e78de/index.js#L181-L189

  • As per the w3c specification at https://w3c.github.io/media-source/#sourcebuffer-segment-parser-loop the browser (Safari) seems to follow the steps indicated in point 6.

    If the append state equals PARSING_MEDIA_SEGMENT, then run the following steps:
    If the first initialization segment received flag is false, then run the append error algorithm and abort this algorithm.

    Being such append error algorithm what's exactly happening at this point under Safari.

  • The above occurs silently to the videostream/mediasource logic, so the next appendBuffer attempt does fail with InvalidStateError because the MediaSource state is closed as result of the endOfStream() implicitly invoked as part of the before mentioned append error algorithm, and the SourceBuffer(s) we're attempting to append buffers to have been detached from it.

So, the million question is, why does the appendBuffer fails once the initialization segments were apparently sent successfully... or weren't?

Worth noting that mp4box.js does work fine under Safari, so i suspect it's either how the mediasource module was made to work, or perhaps what is mentioned in #20, dunno -- thoughts?

If someone could bring some light/guidance on this it will be quite awesome/appreciated, I'm eager to help on getting this working under Safari, thanks in advance!

[jhiesey]

Thanks for taking a look! I know it's broken in Safari (and Edge too), I just haven't gotten around to debugging the issue. Unfortunately, as you discovered, debugging this stuff is a major pain, especially since Safari seems to be particularly picky about what it accepts. Since you seem interested in this I can try to take a look this week or next.

Interesting that mp4box.js does work. My guess based on your debugging is there's something about the initialization segment videostream is generating that Safari doesn't like.

I don't think this is related to #20, since that's about supporting fragmented input files.

[diegocr]

Interestingly, Edge does work fine here, i think the main difference with your vanilla example is that i'm not using MultiStream, but not sure if that's really related... what issue is throwing at you?

Alright, if you could look into the Safari issue that will be awesome, if you need any help don't hesitate to ping me, meanwhile i'm gonna check how feasible is adding support for webm/mkv, at first glance i think using ts-ebml might be possible, just a little concerned on how seeking would work with them.

[victorlewis]

If you look at the mediasource object returned here:

videostream/videostream.js

Line 46 in c730a19

var mediaSource = self._elemWrapper.createWriteStream(trackData.mime)

The data in Safari is different from Chrome/other browsers. In Safari I'm getting Duration:NaN instead of the correct one, readyState:"closed" instead of "open", segment:0 instead of 2 etc.

This spreads into the track object, in which the video element returns Error4:MEDIA_ERR_SRC_NOT_SUPPORTED in Safari along with the same NaN duration.

---

Additionally, I've retested Videostream-1.2.3 and it's compatible with Safari, but only if the video is muxed using the mp42 brand. Playback is occasionally stopped with an error in "shouldAppend", and the page must be refreshed for playback to be restored.

[feross]

Wohoo, I redeployed https://webtorrent.io and it (sorta) works in Safari now!! 🌟

[diegocr]

Thank you very much @jhiesey!

Amazing just swapping some array member made it work :)

[jhiesey]

So I did a little poking around and it seems that SourceBuffer.abort() actually works, at least the way I'm using it. The remaining stalls are due to something else. Taking a look.

[feross]

@jhiesey Nice, let me know if you need my help for anything

[feross]

Is this issue fully resolved now? Playback in Safari seems quite reliable in my experience.

[PaTiToMaSteR]

Video streaming in Safari, iPad and iOS doesn't work. Even in webtorrent.io. The video plays when is fully buffered/downloaded. Also we need to consider that not only autoplay: true in the ops it's enough, as WebKit published time ago that the way to go in iOS devices is use: playsinline attribute.

Any advances in this regard?

Also worth to mention I'm not downloading the whole content of the webtorrent as has multiple mp4 with different qualities.

The code I'm using is as follows:
var opts = { autoplay: true, muted: false } file.appendTo("#video_container", opts, function (err, elem) { console.log($('#video_container').html()); })

In iOS using Chrome (chrome://inspect) prints this:
<video autoplay="" controls="" src="blob:https....lalala">

Hope this helps @feross