Simplest way to remove something from the FPU stack

Question

I've been having some trouble lately with FPU stack overflows. I managed to track it back to a buggy library function that pushes a garbage value onto the FPU stack every time it's called and never cleans it up.

Fortunately, this is easily reproducible and I know exactly what conditions cause it. I can drop a block of inline ASM into the routine that calls this routine to pop the top value back off the FPU stack... except I don't quite know what to write. My ASM-fu is fair to middlin', but not that strong.

So what's the simplest way to get rid of the top value on the FPU stack in x86 assembly, assuming it's garbage data and I don't care about the value?

Answer 1

If you know how much you need to adjust the stack by, you can use fincstp. You also want to ffree the registers that you increment over.

However, probably the simplest solution is to use one of the popping data transfer operations like fstp. Normally you would store the result into an area of memory for later use, something like:

mem_area: defs 10         ; ten bytes for 80 bits
          fstp mem_area   ; pop it

But, if you know you just want to throw away the value, you can use st(0) itself as the destination, saving the memory requirement:

fstp st(0)

See here for a detailed guide on the instructions (particularly this bit).