Creating music visualizer [closed]

Question

So how does someone create a music visualizer? I've looked on Google but I haven't really found anything that talks about the actual programming; mostly just links to plug-ins or visualizing applications.

I use iTunes but I realize that I need Xcode to program for that (I'm currently deployed in Iraq and can't download that large of a file). So right now I'm just interested in learning "the theory" behind it, like processing the frequencies and whatever else is required.

Answer 1

As a visualizer plays a song file, it reads the audio data in very short time slices (usually less than 20 milliseconds). The visualizer does a Fourier transform on each slice, extracting the frequency components, and updates the visual display using the frequency information.

How the visual display is updated in response to the frequency info is up to the programmer. Generally, the graphics methods have to be extremely fast and lightweight in order to update the visuals in time with the music (and not bog down the PC). In the early days (and still), visualizers often modified the color palette in Windows directly to achieve some pretty cool effects.

One characteristic of frequency-component-based visualizers is that they don't often seem to respond to the "beats" of music (like percussion hits, for example) very well. More interesting and responsive visualizers can be written that combine the frequency-domain information with an awareness of "spikes" in the audio that often correspond to percussion hits.

Answer 2

For creating BeatHarness ( http://www.beatharness.com ) I've 'simply' used an FFT to get the audiospectrum, then use some filtering and edge / onset-detectors.

About the Fast Fourier Transform : http://en.wikipedia.org/wiki/Fast_Fourier_transform

If you're accustomed to math you might want to read Paul Bourke's page : http://local.wasp.uwa.edu.au/~pbourke/miscellaneous/dft/

(Paul Bourke is a name you want to google anyway, he has a lot of information about topics you either want to know right now or probably in the next 2 years ;))

If you want to read about beat/tempo-detection google for Masataka Goto, he's written some interesting papers about it.

Edit:

His homepage : http://staff.aist.go.jp/m.goto/ Interesting read : http://staff.aist.go.jp/m.goto/PROJ/bts.html

Once you have some values for for example bass, midtones, treble and volume(left and right), it's all up to your imagination what to do with them. Display a picture, multiply the size by the bass for example - you'll get a picture that'll zoom in on the beat, etc.