FFT analysis Part 1
The Fourier Transform has been known as one of the most profound mathematical concepts of all times. I neither understand properly nor have the patience to even try to explain it, I can however recommend you to read this link the next time your on the bus or similar, it’s a pretty cool read!
Basically, for music, we can use it to separate the signal into frequency bins vs amplitude strengths. As we increase in ‘samples’, the bins holds fewer and fewer frequencies (1024 would say 43.1hz per bin) “The 1st bin contains the DC component, the 2nd contains the amplitude at 1/2048 of the channel’s sample rate, followed by the amplitude at 2/2048, 3/2048, etc.” [1] So a song with 44.1khz sampled with 2048 samples would have 21.53hz (44100/2048) bins….
I want to use this information to synchronize visuals. The most important parts will always be manually set, but if we can control minor effects like blur pulsation, noise, flashes and etc with perfectly sync; it would add a great deal to the whole production. But it ain’t easy. I’ve been using Bass (http://www.un4seen.com/) to play music for ages, it’s a great lib. It happily provides me with a function called BASS_ChannelGetData to which I can ask for my sample size and then returns the frequency bins. The bins contains data from the currently decoded data in the channel, if less than defined in BASS_CONFIG_BUFFER. During run-time BASS decodes the music, the sample data is accumulated (I think) to a buffer, to which BASS_ChannelGetData replies to. The problem is this; I want a fixed size on this buffer (no real problem, BASS_SetConfig) and still be able to move back and forth in time (or even export videos at 20-30-60fps) and getting the same results. If I request a video at a lower frequency, I get lower amplitudes…..I’ve to be able to expect similar results.
This video below is created by sampling Ponas – Medvind with a 4096 sample size at 60fps. I have no idea what the actual FFT data represents, but it looks nice enough.
Due to this problems, I decided to just sample the FFT data at a fixed sampling frequency and save it – to be able to both export videos at different frequencies but most importantly edit at realtime without any issues. Here is the first video from that attempt: Undertakers – Gates of Hell with a sample size of 2048 at 60 fps:
It is a quick fix for sure, and there are some unanswered questions. Buuuut; I don’t care too much at this point. Let’s try to make this look better for next week!
References:
[1] Bass Documentation 2.4, BASS_ChannelGetData