FFT explanation for non math guys

[emily]

Well I also want to read more about FFT. It helps to keep in mind that Fast Fourier Transform is simply an optimization for the real thing: the Discrete Fourier Transform (DFT). If it's difficult to find material in English, imagine how difficult it is to find good material in Greek!

[jasonp]

Lately I've found this free book a fascinating read on FFTs, since I've become interested in number-theoretic transforms and so need to re-apply all of the theory it describes but in domains without complex numbers. The web site has other works by Burrus as well, that complement topics that the book glosses over a little.

It's really surprising how difficult it is to get the original papers mentioned in the bibliography. Most are from the 1980s, only Burrus' notes are more recent that I could find. Either you find a copy of Nussbaumer's book in a university library or you pay the IEEE $10 a page for a PDF. At that rate my collection of hobby papers and books would have cost me $100,000.

[legendarymudkip]

Originally Posted by ewmayer

Since all the output digits happen to be less than 10 and nonnegative, we don't need to do any carry step

Is the carry step like with the GS method? So say if you had (48,52,52,52), would you then get the result 52+520+5200+48000=53772? If not, how else would you do it?

[ewmayer]

Originally Posted by legendarymudkip

Is the carry step like with the GS method? So say if you had (48,52,52,52), would you then get the result 52+520+5200+48000=53772? If not, how else would you do it?

Assuming digit significance (power of 10) decreases from left to right, yes. But in practice we would start with the 0-index output term, the rightmost 52 in your notation, and work our way upward (leftward) like so, with the left-column index indicating power of 10, and /= denoting integer (truncating) divde and %= indicating (nonnegative-output) modulo:

0: 52, /= 5 (carry), %= 2;
1: 5+52, /= 5 (carry), %= 7;
2: 5+52, /= 5 (carry), %= 7;
3: 5+48, /= 5 (carry), %= 3;
4: 5+0, no carry, hence done.

Now *really* in practice we would use a modulo based on nearest-int rounding of the DIV result, yielding a balanced-digit representation of the result, 53772, which has much better numerical properties as far as the next FFT-squaring (assuming one occurs) is concerned. Same as above, but everytime the %= result is > 5 (i.e. half the base) we subtract 10 from the current digit and increment the carry by 1 to account for the -10:

0: 52, /= 5 (carry), %= 2;
1: 5+52, /= 6 (carry), %= -3;
2: 6+52, /= 6 (carry), %= -2;
3: 6+48, /= 5 (carry), %= 4;
4: 5+0, no carry, hence done.

Check the result: 2 + 10*( -3 + 10*( -2 + 10*( 4 + 10*(5)))) = 53772, as expected.

In the case where the "coin lands on its edge" (%= 5 in base-10) you can either use an IEEE-compliant round-to-nearest-even (if your hardware can do that efficiently), or just take whichever direction your preferred NINT emulation (e.g. NINT(x) = (x + c) - c, where the "magic constant" c = 0.75*2^b and b = #bits of the mantissa in your floating point type) - which of the 2 is not crucial, in my experience with these types of computations.

[CadeBrown]

How exactly does the FFT reduce to order NlogN ? It seems as though you are still doing a NxN matrix multiplied by a scalar, which seems to be at least N^2 Is there something most programs do to cut it down?

[ewmayer]

Originally Posted by CadeBrown

How exactly does the FFT reduce to order NlogN ? It seems as though you are still doing a NxN matrix multiplied by a scalar, which seems to be at least N^2 Is there something most programs do to cut it down?

The core aspect of the FFT is that no matrix multiplies are done ... consider the length-4 vector Fourier transform example I give in post #5, in matrix-multiply form. See the obvious redundant operations? Adding a few () and a few rearrangements of the inputs to make things more obvious, our 4 DFT outputs are

out0 = (a+c) + (b+d)
out1 = (a-c)+i*(b-d)
out2 = (a+c) - (b+d)
out3 = (a-c)-i*(b-d)

So instead of a naive matrix multiply, we first compute the following intermediates - these are the famous radix-2 "butterflies":

y0 = (a+c)
y1 = (a-c)
y2 = (b+d)
y3 = (b-d) ,

which we can do in-place, overwriting the original inputs. (Though there are intricacies such as bit-reversal reordering and schemes for avoiding the need for it involved in the deployment of an in-place FFT scheme.)

Then combine these to obtain the outputs:

out0 = y0 + y2
out1 = y1+i*y3
out2 = y0 - y2
out3 = y1-i*y3 .

Thus radix-4 needs 2 passes through the data, each pass doing just linear work, i.e. O(n). For length n = 2^k we need k = lg(n) (lg = base-2 logarithm) such passes, each of O(n) cost, hence O(n log n) overall. For n not a power of 2 things are bit more involved, but one can still prove the O(n log n) property, just with a higher implied constant of proportionality. The procedure can be done recursively - it is perhaps most naturally explained that way - or not. Any decent FFT reference has more details, though I found such small worked examples very useful way back when I was first learning this stuff, and especially working out the mechanics of non-power-of-2-length FFTs, which relatively few references cover in any useful fashion.

[tServo]

FWIW, here is a Youtube video I found posted Autumn 2018 in which a guy multiplies 41 * 37 by hand with paper and pencil using FFT!
How cool-y retro.
It's 7 minutes long and has just the basics.
Enjoy:
https://www.youtube.com/watch?v=YDhsLhTK3Bs

[benbradley]

I could write up something describing how to do the transform done by the machine in this link - it doesn't use complex numbers, because it doesn't do a full Fourier Transform. What it does is called a discrete cosine transform. If you also do a discrete sine transform and combine the results as complex numbers, you've got the whole discrete fourier transform. The absolute values of these complex numbers are the amplitudes in each frequency bin, and the phases are of course the phase in each bin.

I see the Fast Fourier Transform as a separate entity, It's essentially a mathematical trick to calculate the DFT with fewer calculations (something like N log N instead of N^2). I do NOT quite understand how the FFT works despite having written FFT code, following the butterfly diagrams and such. One thing common with the FFT is the precalculated "twiddle factors." These values are repeatedly calculated (though perhaps not obviously so) as part of calculating the DFT.

You can get the guy's PDF book from this page and see the videos - the first video just goes over the book, the next four show the operation of the machine:
https://engineerguy.com/fourier/

[chalsall]

Originally Posted by benbradley

I could write up something describing how to do the transform done by the machine in this link - it doesn't use complex numbers, because it doesn't do a full Fourier Transform. What it does is called a discrete cosine transform.

That would be welcomed. Remember that DCT is rather important for JPEG and even early MPEG.

Wavelet encoding will be left for a later class...