Storing Enormous Static Variables in C++

Question

I have a string of information that is roughly 17 kb long. My program will not generate this string or read it into a buffer - the data is already initialized, I want it to be compiled as is from within my code, like you would a static variable. Moreover, I'd much prefer it is within my executable, and not stored within a project file. I've never before encountered such an issue, what is the best way to go around this? Should I include as resource, or literally copy and paste the enormous stream of data into a variable? What would you recommend?

Forgot to mention, am using VisualStudio C++ 2015 if that matters

Answer 1

The GNU linker ld has the ability to directly include custom data as the .data section of an object file:

 ld -r -b binary -o example.o example.txt

The resulting example.o file has symbols defined to access start and end of the embedded data (just look at the file with objdump to see how they're named).

Now I don't know whether the linker coming with Visual Studio has a similar ability, but I guess you could use the GNU linker either via mingw or also via cygwin (since the generated object file won't reference the standard lib you won't need the emulation lib that comes with cygwin).The resulting object file apparently can just be added to your sources like a regular source file.

Of course this manual workflow isn't too good if the data changes often...

Alternatively you can write a simple program which puts the contents of the file in a C string, like:

unsigned char const * const data = { 
  0x12, 0x34, 0x56 };

Of course there's already such a program (xdd) but I don't know whether it's available to you. One potential issue is that you could reach the limit for the length of string literals that way. To get around that you could try a (multidimensional) char array.

(When writing this answer I found this blog post very helpful.)