synchronize and highlight HTML text to audio

Question

If necessary I can explain in more detail but essentially what I need to do is effect CSS changes to HTML text in sync with an audio track - i.e., highlighting words/phrases in sync with the audio playback. I also need to control the audio playback by clicking on the text. I have good HTML/CSS chops, but I’m not as strong with raw js, so I’m hoping there’s a jQuery approach. I’m hoping someone can steer me in the best direction.

Many thanks in advance,

svs

Answer 1

For ease of use, I recommend a combination of jQuery and Popcorn.js for anything where you want to integrate media with HTML, and visa versa. See this jsfiddle post for an example.

For the record, should jsfiddle ever disappear, here's the code:

HTML

<audio id="greeting" src="https://dl.dropboxusercontent.com/u/17154625/greeting.ogg" controls></audio>

<div id="text">
   <span id="w1" class="word" data-start="1.0">Hello</span>,
   and <span id="w2" class="word" data-start="2.0">welcome</span>
   to Stack <span id="w3" class="word" data-start="3.0">Overflow</span>.
   Thank you for asking your question.
</div>​

CSS

.word {
   color: red;
}
.word:hover, .word.selected {
    color: blue;
    cursor: pointer;
}​

JS

var pop = Popcorn("#greeting");

var wordTimes = {
    "w1": { start: 1, end: 1.5 },
    "w2": { start: 1.9, end: 2.5 },
    "w3": { start: 3, end: 4 }
};

$.each(wordTimes, function(id, time) {
     pop.footnote({
        start: time.start,
        end: time.end,
        text: '',
        target: id,
        effect: "applyclass",
        applyclass: "selected"
    });
});

pop.play();

$('.word').click(function() {
    var audio = $('#greeting');
    audio[0].currentTime = parseFloat($(this).data('start'), 10);
    audio[0].play();
});​