Text to speech: highlighting word by word for iphone

Question

I am doing text to speech on a UITextView by using flite-1.4-iphone. While reading the text, I want to automatically highlight the text word by word.

1. How can I synchronize the voice to the text highlighting while reading?
2. How can I highlight the text on the text view?

Here's my current code:

-(IBAction)btnClick:(id)sender
{
    [indicator startAnimating]; 
    textToSpeech = [[TextToSpeech alloc] init]; 
    [textToSpeech setVoice:@"cmu_us_awb"];
    [textToSpeech speakText:txtview.text];
    if ([txtview.text isEqualToString:@""]) 
    {
        [textToSpeech stopTalking];
        [self animate];
    }
}

Answer 1

Flite does not have a way to tie in to see what word is being spoken when. What it does is uses the text to generate an audio file and just plays it. You could make a custom class to handle the information being passed to the TextToSpeech. The class would separate the string into separate words and then pass them to flite.

If you look in fliteTTS.m, in the speakText: method, you can see that it creates a wav file, and then has the an AVPlayer play the wav file. what you could do, is change that method so that instead of playing the file, it will return the URL to the wav file to your custom class (Which could save them in an array).

Then have the custom class play the sounds in order, and each time you play the next clip, highlight the new section of text.

So instead of speakText:

-(NSString *)urlForSpeech:(NSString *)text
 {
    NSMutableString *cleanString;
    cleanString = [NSMutableString stringWithString:@""];
    if([text length] > 1)
    {
            int x = 0;
            while (x < [text length])
            {
                    unichar ch = [text characterAtIndex:x];
                    [cleanString appendFormat:@"%c", ch];
                    x++;
            }
    }
    if(cleanString == nil)
    {       // string is empty
            cleanString = [NSMutableString stringWithString:@""];
    }
    sound = flite_text_to_wave([cleanString UTF8String], voice);

    NSArray *filePaths = NSSearchPathForDirectoriesInDomains (NSDocumentDirectory, NSUserDomainMask, YES);
    NSString *recordingDirectory = [filePaths objectAtIndex: 0];
    // Pick a file name
    NSString *tempFilePath = [NSString stringWithFormat: @"%@/%s", recordingDirectory, "temp.wav"];
    // save wave to disk
    char *path;     
    path = (char*)[tempFilePath UTF8String];
    cst_wave_save_riff(sound, path);

    return tempFilePath;      
}

To highlight the text, as an AVPlayer plays through the files, use:

[textView select:self];
textView.selectedRange = aSelectedRange; 

Where aSelectedRange is the range of the string you want highlighted.

I am not familiar with AVPlayer, so I can't really help you set that up, but there are some very good samples on apple's developer site. This is what you should look at: link

Just don't forget to delete the audio files when you are done with them.