Introspecting struct definition in C++?

Question

So I'm working on a file format for storing structured data defined over some kind of range. The simplest example would be something like float values over time, but I'm designing it to allow multiple dimensions and arbitrary data at each point. I'd like to store the structure of the data in the file header as well because it will allow for some neat features.

At first I thought I'd have the user build up their data structure using a hierarchy of classes, so you could do something like this:

pf_type data = record(PF_DOUBLE) (
                  record("position")(
                     field("xp") &
                     field("yp") &
                     field("zp")) &
                  record("velocity")(
                     field("xv") &
                     field("yv") &
                     field("zv")) &
                  record("acceleration")(
                     field("xa") &
                     field("ya") &
                     field("za")));

which would be equivalent to something like this in C:

struct {
   struct position {
       double xp,yp,zp;
   }
   struct velocity {
       double xv,yv,zv;
   }
   struct acceleration {
       double xa,ya,za;
   }
}

It's not terrible, but it still requires the user to separately define a c struct to actually use when reading/writing the data.

I thought it'd be neat if they could just define a regular c-struct that they passed to my library and I introspected upon it to get the information to write to the file. Though I have no idea if something like that is remotely possible in C++. My thought is that it might be using some template metaprogramming magic but that it'd be extremely messy. So I thought I'd solicit some ideas from those that know more about C++ than I do.

Answer 1

Not exactly what you're looking for, but probably a very good source for inspiration, would be Google's Protobufs. They take a slightly different approach than the ones that you discussed in your question. With protobuf, first you describe your data structures in .proto files, and then the protobuf compiler (protoc) codegens boilerplate code for one of several different languages, including C++.

The codegened code is fully capable of reflecting over the structures that were defined in the .proto file, and what protobuf is most optimized for is serializing binary data for being sent over the wire. Obviously, you'll pay a little bit of a performance hit, but it'll get you some really high quality reflection, and the perf hit for simple accesses isn't all that high.

Essentially, clients of your library could pass you a filled in protobuf. Either by directly calling you, or via a network call.

Answer 2

Template machinery is too primitive to be able to do this kind of stuff reasonably. A better option in my opinion is to have a separate easy to edit and easy to parse file describing the data structure and then from that definition generate the C++ code you need.

Writing code generators in C++ is however annoying because C++ I/O, parsing, formatting and string manipulation is suboptimal, my suggestion is to use Python or another very high level language for that part.

Everything can be then probably put in a Makefile with proper dependencies (i.e. if the description file changes you regenerate the C++ files) and so the build is still automatic.

Having a "normal" C++ file instead of template trickery is going to make your life a lot easier (compile time, error messages, compilers incompatibilities for hard-core template stuff, support for debugging the generated code).