FSTENV? Can barely find any info about this instruction

Question

I've got this instruction in a program:

FSTENV (28-BYTE) PTR SS:[ESP-1C]

What does it do?

Which registers does it use and updates?

Thank you!

Answer 1

Just for the sake of completeness, here's the memory layout that FSTENV/FNSTENV instructions produce. It is the same in x86-64 "Long 64-bit Mode", and in x86 "32 bit compatibility mode" (unless you prepend it with the prefix 66h in x86-64.)

Quoting this unwieldy Intel document, this is the layout:

(Btw, the image above should've been named "long mode" as well.)

So if we run an actual test in a long 64-bit mode:

and break right after FNSTENV instruction with the following context state:

the layout of memory it returns is as follows:

Which, I'm not gonna hide, is very bizarre. (Anything from a chopped off RIP register to strange looking opcode.) But it is still supported, for legacy reasons, I guess.

The good news is that there are not many uses left today for these archaic x87 FPU instructions when coding the modern x64 code. In that case you should definitely stick with the XMM0 through XMM15 registers and their relevant instructions for your floating point needs.

Answer 2

Jerry Coffins Answer is correct.
In case you are wondering about the (28-BYTE) PTR SS:[ESP-1C]:
This is the effective address where the FP environment is to be stored, it designates the 28-byte version of the the command and points to 28 (0x1c) bytes below the stack pointer in the stack segment.
I just add the official description from Intel, which I found using a search engine.

Description

Saves the current FPU operating environment at the memory location specified with the destination operand, and then masks all floating-point exceptions. The FPU operating environment consists of the FPU control word, status word, tag word, instruction pointer, data pointer, and last opcode. Figures 7-13 through 7-16 in the IA-32 Intel® Architecture Software Developer's Manual, Volume 1, show the layout in memory of the stored environment, depending on the operating mode of the processor (protected or real) and the current operand-size attribute (16-bit or 32-bit). In virtual-8086 mode, the real mode layouts are used.

The FSTENV instruction checks for and handles any pending unmasked floating-point exceptions before storing the FPU environment; the FNSTENV instruction does not. The saved image reflects the state of the FPU after all floating-point instructions preceding the FSTENV/FNSTENV instruction in the instruction stream have been executed.

These instructions are often used by exception handlers because they provide access to the FPU instruction and data pointers. The environment is typically saved in the stack. Masking all exceptions after saving the environment prevents floating-point exceptions from interrupting the exception handler. Intel® Architecture Compatibility

When operating a Pentium® or Intel486™ processor in MS-DOS* operating system compatibility mode, it is possible (under unusual circumstances) for an FNSTENV instruction to be interrupted prior to being executed to handle a pending FPU exception. See the section titled "No-Wait FPU Instructions Can Get FPU Interrupt in Window" in Appendix D of the IA-32 Intel® Architecture Software Developer's Manual, Volume 1, for a description of these circumstances. An FNSTENV instruction cannot be interrupted in this way on a Pentium Pro processor.

Operation

DEST[FPUControlWord) <- FPUControlWord;

DEST[FPUStatusWord) <- FPUStatusWord;

DEST[FPUTagWord) <- FPUTagWord;

DEST[FPUDataPointer) <- FPUDataPointer;

DEST[FPUInstructionPointer) <- FPUInstructionPointer;

DEST[FPULastInstructionOpcode) <- FPULastInstructionOpcode;

FPU Flags Affected

The C0, C1, C2, and C3 are undefined.

Floating-Point Exceptions

None.

Protected Mode Exceptions

GP(0) - If the destination is located in a nonwritable segment. If a memory operand effective address is outside the CS, DS, ES, FS, or GS segment limit. If the DS, ES, FS, or GS register is used to access memory and it contains a null segment selector.

SS(0) - If a memory operand effective address is outside the SS segment limit.

NM - EM or TS in CR0 is set.

PF(fault-code) - If a page fault occurs.

AC(0) - If alignment checking is enabled and an unaligned memory reference is made while the current privilege level is 3. Real-Address Mode Exceptions

GP - If a memory operand effective address is outside the CS, DS, ES, FS, or GS segment limit.

SS - If a memory operand effective address is outside the SS segment limit.

NM - EM or TS in CR0 is set. Virtual-8086 Mode Exceptions

GP(0) - If a memory operand effective address is outside the CS, DS, ES, FS, or GS segment limit.

SS(0) - If a memory operand effective address is outside the SS segment limit.

NM - EM or TS in CR0 is set.

PF(fault-code) - If a page fault occurs.

AC(0) - If alignment checking is enabled and an unaligned memory reference is made.