Dumping registers in stack for conservative stack scanning

Question

I'm writing a non-invasive conservative GC in C and I'm having some concerns about the correctness of its stack scanning phase.

Specifically, with no compiler optimizations enabled, it works fine because every local variable (which points to an object) is "predictably" allocated on the stack. In -O3, the GC misses some valid references, which I believe is due to the fact that the compiler opts to use registers (instead of the stack, which the GC scans) for some variables and function argument passing and the GC isn't (yet) programmed to handle that. (If you suspect that this still shouldn't be happening and that I'm misreading the source of the problem, please let me know.)

Apart from some rudimentary requirements (having to use a GC_malloc instead of malloc for GC objects, not pointing to the GC heap from the non-GC heap and, of course, not explicitly calling free), the GC shouldn't have any more requirements from the client code or the compiler. Therefore, requiring any additional special patterns from the client code is not an option. Similarly, forcing programs that use this GC to be compiled with special compiler flags (that suppress stack optimizations) should be a last resort option, at best. With these two out of the way, here's the build-up to my question.

I'm trying to find a way to make the GC handle the -O3 case (with the stack optimizations) seamlessly. Here's my train of thoughts (assumptions, more precisely):

1. No matter how far GCC (or any valid compiler) goes with the optimizations, it wouldn't go so far that it would compromise the correctness of the program.
2. If [1] holds, I believe that (at least in practical implementations of C) if a reachable (allocated as local at any call level) variable is going to remain accessible at all, it has to be either on the stack or in the current register set, but not really anywhere else.
3. If [2] holds, it should mean that simply dumping all the registers to the stack when the GC cycle starts would guarantee that a subsequent stack scanning would now find the previously missed references as well.
4. Additionally, if [1] and [2] hold, then if the compiler lets any variables be overwritten (in the case of register-based ones, mostly), it means that it has performed some level of code analysis and it has proved that that variable isn't used anywhere else in that function. So, in that sense, if we don't see a variable in the stack or in the register dump, even though it's still in scope (theoretically), this shouldn't be a cause for alarm (for me) because the compiler has simply sort of helped out the GC by getting rid of a dead reference.

Question #1: Are all 4 of my assumptions correct?

Question #2: What is the most portable way to force a register dump? I've found some sources claiming that a simple setjmp call will have this effect. Is this correct?

Answer 1

Q1. Yes, I believe all four of your statements hold true (at least if we disregard compiler bugs!)

Q2. setjmp will save SOME registers, but not necessarily all registers. However, it should be suffcient for your purposes, since any register NOT saved by setjmp should be saved saved on the stack anyways.

I guess your scheme may got awry if someone stores something that LOOKS like an address in a buffer somewhere, even if it isn't.

You also have to bear in mind that sometimes people do "interesting" things with pointers. E.g.

struct blah
{
    size_t size;
    char *file;
    int line;
};


struct blah *p = malloc(sizeof(struct blah) + size); 
... more lines of code goes here to fll in size, file and line in blah. 
void *np = (p+1); 

This means that your stored pointer doesn't point to the start of the block at all.

Answer 2

The big problem with #1 is programs that do things that are undefined (or questionable), but that generally work. Things like using pointers as magic constants that get stored in files or sent across network connections and generally forgotten about locally, but then they might come back (from the file or network) and be dereferenced later. Or the old "xor prev and next pointers" trick for a linked list with only one pointer of overhead per node.