Is Option<T> optimized to a single byte when T allows it?

Question

Suppose we have an enum Foo { A, B, C }.

Is an Option<Foo> optimized to a single byte in this case?

Bonus question: if so, what are the limits of the optimization process? Enums can be nested and contain other types. Is the compiler always capable of calculating the maximum number of combinations and then choosing the smallest representation?

Answer 1

The compiler is not very smart when it comes to optimizing the layout of enums for space. Given:

enum Option<T> { None, Some(T) }
enum Weird<T> { Nil, NotNil { x: int, y: T } }
enum Foo { A, B, C }

There's really only one case the compiler considers:

An Option-like enum: one variant carrying no data ("nullary"), one variant containing exactly one datum. When used with a pointer known to never be null (currently, only references and Box<T>) the representation will be that of a single pointer, null indicating the nullary variant. As a special case, Weird will receive the same treatment, but the value of the y field will be used to determine which variant the value represents.

Beyond this, there are many, many possible optimizations available, but the compiler doesn't do them yet. In particular, your case will not berepresented as a single byte. For a single enum, not considering the nested case, it will be represented as the smallest integer it can.