Frequency Modulation Synthesis Algorithm

Question

Based on what I read, I've made an algorithm for FM sound synthesis. I'm not sure if I did it right. When creating a software synth instrument a function is used to generate an oscillator and a modulator can be used to module the frequency of this oscillator. I don't know if FM synthesis is supposed to only work for modulating sine waves?

The algorithm takes the instruments wave function and the modulator index and ratio for the frequency modulator. For each note it takes the frequency and stores the phase value for the carrier and modulator oscillators. The modulator always uses a sine wave.

This is the algorithm in pseudocode:

function ProduceSample(instrument, notes_playing)
    for each note in notes_playing
        if note.isPlaying()
            # Calculate signal
            if instrument.FMIndex != 0 # Apply FM
                FMFrequency = note.frequency*instrument.FMRatio; # FM frequency is factor of note frequency.
                note.FMPhase = note.FMPhase + FMFrequency / kGraphSampleRate # Phase of modulator.
                frequencyDeviation = sin(note.FMPhase * PI)*instrument.FMIndex*FMFrequency # Frequency deviation. Max deviation is a factor of the FM frequency. Modulation is done by a sine wave. 
                note.phase = note.phase + (note.frequency + frequencyDeviation) / kGraphSampleRate # Adjust phase with deviation
                # Reset the phase value to prevent the float from overflowing
                if note.FMPhase >= 1
                    note.FMPhase = note.FMPhase - 1
                end if
            else # No FM applied
                note.phase = note.phase + note.frequency / kGraphSampleRate # Adjust phase without deviation
            end if
            # Calculate the next sample
            signal = signal + instrument.waveFunction(note.phase,instrument.waveParameter)*note.amplitude
            # Reset the phase value to prevent the float from overflowing
            if note.phase >= 1
                note.phase = note.phase - 1
            end if
        end if
    end loop
    return signal
end function 

So if the note's frequency is at 100Hz, the FMRatio is set at 0.5 and the FMIndex is 0.1 it should produce frequencies going between 95Hz and 105Hz in a 50Hz cycle. Is this the correct way of doing it. My tests show that it doesn't always sound right, especially when modulating saw and square waves. Is it OK to modulate saw and square waves like this or is it for sine waves only?

This is the implementation in C and CoreAudio:

static OSStatus renderInput(void *inRefCon, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inBusNumber, UInt32 inNumberFrames, AudioBufferList *ioData){
    AudioSynthesiser * audioController = (AudioSynthesiser *)inRefCon;
    // Get a pointer to the dataBuffer of the AudioBufferList
    AudioSampleType * outA = (AudioSampleType *) ioData->mBuffers[0].mData;
    if(!audioController->playing){
        for (UInt32 i = 0; i < inNumberFrames; ++i){
            outA[i] = (SInt16)0;
        }
        return noErr;
    }
    Track * track = &audioController->tracks[inBusNumber];
    SynthInstrument * instrument = (SynthInstrument *)track;
    float frequency_deviation;
    float FMFrequency;
    // Loop through the callback buffer, generating samples
    for (UInt32 i = 0; i < inNumberFrames; ++i){
        float signal = 0;
        for (int x = 0; x < 10; x++) {
            Note * note = track->notes_playing[x];
            if(note){
                //Envelope code removed
                //Calculate signal
                if (instrument->FMIndex) { //Apply FM
                    FMFrequency = note->frequency*instrument->FMRatio; //FM frequency is factor of note frequency.
                    note->FMPhase += FMFrequency / kGraphSampleRate; //Phase of modulator.
                    frequency_deviation = sinf(note->FMPhase * M_PI)*instrument->FMIndex*FMFrequency; //Frequency deviation. Max deviation is a factor of the FM frequency. Modulation is done by a sine wave. 
                    note->phase += (note->frequency + frequency_deviation) / kGraphSampleRate; //Adjust phase with deviation
                    // Reset the phase value to prevent the float from overflowing
                    if (note->FMPhase >= 1){
                        note->FMPhase--;
                    }
                }else{
                    note->phase += note->frequency/ kGraphSampleRate; //Adjust phase without deviation
                }
                // Calculate the next sample
                signal += instrument->wave_function(note->phase,instrument->wave_parameter)*track->note_amplitude[x];
                // Reset the phase value to prevent the float from overflowing
                if (note->phase >= 1){
                    note->phase--;
                }
            } //Else nothing added
        }
        if(signal > 1.0){
            signal = 1;
        }else if(signal < -1.0){
            signal = -1.0;
        }
        audioController->wave[audioController->wave_last] = signal;
        if (audioController->wave_last == 499) {
            audioController->wave_last = 0;
        }else{
            audioController->wave_last++;
        }
        outA[i] = (SInt16)(signal * 32767.0f);
    }
    return noErr;
}

Answers are very much appreciated.

Answer 1

Redeye:

To answer your main question, yes it's absolutely fine to modulate waveforms other than sine waves. In fact, that's what FM is best at. Modulating sine waves gives a very boring sounding output, but when you input more complex waveforms with the same modulation, you get much more interesting results.

This is at best an oversimplification and possibly totally false. Modulating sine waves with sine waves is perfectly capable of creating a wide range of complex and not "boring" sounds.

In contrast, complex waveforms multiply the number of resulting sidebands massively and make predictable results much more hard to achieve. Most documentation about FM - which is actually the almost-equivalent PHASE modulation (PM) in many common cases including "the" "FM" by Yamaha - concerns sine waves only.

FYI (in case you don't already know), the most famous FM synth is probably the Yamaha DX7 which was revolutionary in its day (and also one of the first ever synths with MIDI).

The other thing to mention is that FM synthesis was the start of the digital age so the waveforms were generated digitally and hence used more sophisticated waveforms than sine/square/triangle waves to create the interesting sounds."

This is totally false without a doubt. The DX7 and many early FM - in reality, PM - synths by Yamaha offered ONLY sine waves, and yet, as I indicated above, they are still capable of many, many non-"boring" sounds. No "more sophisticated waveforms" were involved.

Only later did Yamaha add other waveforms, and their utility is somewhat questionable when compared to the predictability of the sidebands created by sine waves, as I stated above.

This might be what you need to do to get a better sound - rather than just generate a sine wave to modulate, use complex waveforms."

Or just use sine waves with good arrangements and combinations of parameters (ratio, index, etc.)

The fact that FM/PM with sine waves does not immediately produce studio-quality - or maybe just analogue-like - results for many users does not indicate whatsoever that it is incapable of doing so.

Answer 2

In the end I decided to use phase modulation. I found out many synthesisers use phase modulation even when they are labeled with FM.

It was simple to implement:

signal += wave_function(note_phase * note_frequency / sample_rate + fm_index * sin(note_phase * fm_frequency * pi / sample_rate))*note_amplitude