What does clang++'s option -fno-strict-enums do?

Question

Two months ago, I reported, as a clang++ bug, that the C++ program below sets z to 4294967295 when compiled with clang++ -O2 -fno-strict-enums.

enum e { e1, e2 } e;

long long x, y, z;

char *p;

void f(void) {
    e = (enum e) 4294967295;
    x = (long long) e;
    y = e > e1;
    z = &p[e] - p;
}

My bug report was closed as invalid because the program is undefined. My feeling was that using the option -fno-strict-enums made it defined.

As far as I know, Clang does not have documentation worthy of the name, because it aims at being compatible with GCC with respect to the options it accepts and their meaning. I read GCC's documentation of the option -fno-strict-enums as saying that the program should set the value of z to -1:

-fstrict-enums

Allow the compiler to optimize using the assumption that a value of enumerated type can only be one of the values of the enumeration (as defined in the C++ standard; basically, a value that can be represented in the minimum number of bits needed to represent all the enumerators). This assumption may not be valid if the program uses a cast to convert an arbitrary integer value to the enumerated type.

Note that only the option -fstrict-enums is documented, but it seems clear enough that -fno-strict-enums disables the compiler behavior that -fstrict-enums enables. I cannot file a bug against GCC's documentation, because generating a binary that sets z to -1, what I understand -fno-strict-enums to mandate, is exactly what g++ -O2 -fno-strict-enums does.

Could anyone tell me what -fno-strict-enums does in Clang (and in GCC if I have misunderstood what it does in GCC), and whether the value of the option has any effect at all anywhere in Clang?

For reference, my bug report is here and the Compiler Explorer link showing what I mean is here. The versions used as reference are Clang 10.0.1 and GCC 10.2 targeting an I32LP64 architecture.

Answer 1

The effect of -fno-strict-enums is to cancel -fstrict-enums. That is, the compiler is not allowed to optimize using the assumption that a value of enumerated type can only be one of the values of the enumeration. I would like to emphasize that the word choice is "allowed", not "required". It can be difficult to see the impact of no longer allowing something that was not done in the first place. Still, I think I've found an example where this can be seen.

First, I would like to clarify "the values of the enumeration" in the context of the question. The enumeration e has two enumerators, with the values 0 and 1. The smallest number of bits required to represent these values is 1. Thus, the values of the enumeration are all values that can be represented by 1 bit. This happens to coincide with the values of the enumerators in this case, but is not guaranteed in other examples.

Next, let's remove one line from the question's code.

enum e { e1, e2 } e;

long long x, y, z;

char *p;

void f(void) {
    //e = (enum e) 4294967295;
    x = (long long) e;
    y = e > e1;
    z = &p[e] - p;
}

The line I removed interferes with the strict-enum flag. That flag allows the compiler to make an assumption that is not necessary when the compiler knows exactly what the value of e is. The compiler can reasonably choose to not assume that e can hold only 0 or 1 when quite clearly it was just given a different value. (This interference is not dependent upon 4294967295 being too large for a 32-bit signed integer, but merely on 4294967295 being a compile-time value. As another example, assigning (enum e) 2 to e would also cause this interference.)

Focus on the assignment y = e > e1. If -fno-strict-enums is in effect, the only optimization available is to replace e1 with 0. However, if we can assume that e can be only 0 or 1 (the values of the enumeration, which happen to also be the values of the enumerators), another optimization becomes available.

If e is 0, the following have the same value:

• (long long) (e > e1)
• (long long) (0 > 0)
• (long long) false
• (long long) e

If e is 1, the following have the same value:

• (long long) (e > e1)
• (long long) (1 > 0)
• (long long) true
• (long long) e

In either case, we can skip the comparison and simply cast e to a long long. This is reflected in the assembly generated by clang 10 for the line y = e > e1.

With -fstrict-enums

movq    %rax, y(%rip)

With -fno-strict-enums

xorl    %ecx, %ecx
testl   %eax, %eax
setg    %cl
movq    %rcx, y(%rip)

An optimization has been made with -fstrict-enums that was not allowed with -fno-strict-enums.