Mobile Safari: Audio + cache manifest

Question

I'm having a small web app, which plays really short sound bits on the click of several buttons. It explicitly targets mobile Safari on iOS (iPad).

After reading here and elsewhere about the several "shortcomings" of HTML5 audio in this context on mobile Safari and trying a few "hacks" and tricks, I'm stuck with a situation where Safari seems simply (for the lack of a better word) broken:

I can play sound A (it takes a long time for it to start — I'm assuming it's downlading [again]?) on the click of button A. After that, clicking on button B will immediately play the sound A again. Same for button C. In some cases it will play a different sound, sometimes even the right one. But mostly sound A. The format in use was .aiff, is now .m4a .

After writing a few tiny versions myself, I decided to go with the Buzz library to handle the sound loading/playing/etc..

Funnily enough, their demo includes a game, which does pretty much exactly what I need and triggers the same faulty behavior. I even ended up in a situation where any audio player in mobile Safari in any tab would play a certain sound out of the Buzz demo game (!).

I was hoping a cache manifest might help overcome Apples preloading limitations and force the app to play the sound right after hitting the button in offline mode. But after confirming that the whole app had been cached, I can't play/hear any sound in offline mode.

Has anyone managed to get something like this to work somehow? (— Having seen how Apple handles certain things, I' don't expect much response, though… )

---

Update 1:

The example in this answer causes the same effect: How to synthesize audio using HTML5/Javascript on iPad

---

Update 2:

Updating iOS (and so Safari) seems to resolve the audio bug. The cache manifest doesn't seem to effect audio files, though. These files are just not available at all.

After removing the cache manifest the app works okay, but adding it to the "home screen" and reloading it prevents the audio from playing as well.

Answer 1

I wish I could tell you there's a magic formula to get all your html5 media to work perfectly, but there isn't. Mobile support for HTML5 audio/video is pretty poor right now; much further behind than its desktop predecesors. To make matters worse, each of the different platforms handle it differently and most of them only support it in semi-recent versions.

However, there are some tricks you can employ to get media files to work correctly in mobile Safari. In order to explain them, you'll need to understand some of its shortcomings.

---

1) You can't load multiple audio / video files

Its been my experience that the browser will only load one file at a time. If you play one file, go and play another, and then come back and it'll just load that file all over again. And, although I didn't try it myself, I don't believe a cache manifest will help you here.

What I had to do is combine all my audio files into one large audio file. Then, depending on which audio track was requested, I'd move the play position to the appropriate starting position and play that track. Then, I'd use a setInterval to examine the playback every few milliseconds to determine if the current play position hit the end of the track. Once it did, I paused it. Pluis, I gave myself a few seconds (2-3) between each track, just in case the phone's CPU was under a lot of load and checked the feed a little too late.

2) You can't auto-play audio / video files

Apple built into their HTML5 media tag technology the limitation that these tracks would only load and play in response to a user click event. That way, developers couldn't auto-play annoying tracks when you went to their websites.

When I was using audio/video tags, I was trying to build a rich media advertisement. So I hooked my audio/tracks loading to the banner click event, when you click a banner and expand the advertisement.

What I'd suggest you should do is have a small lightbox popup come up, asking the user if they want to turn on/off sound. You can attach your load functions to the click event of that popup box, regardless if the user turns the sound on or off.

Personally, I didn't have much luck using the load() function. I'd run that function to load the audio and then click play and it would just load it again. It could have always been that I just didn't do it right, so feel free to prove me wrong and get that working. What did is I told the track to play, so that way it would start loading, and then I'd use a setInterval to see if the current play time incremented just a few milliseconds. Once I noticed that it started to play the track, I'd immediately pause it.

3) Audio/Video tags are only supported in iOS 4.0 and higher

There's no trick to getting around this. Its just the facts.

---

Here's a few sites that helped when when I was developing with audio/video tags:

http://www.w3.org/wiki/HTML/Elements/video
http://dev.w3.org/html5/spec-author-view/video.html#media-elements

Good luck!