Including huge string in our c++ programs?

Question

I am trying to include huge string in my c++ programs, Its size is 20598617 characters , I am using #define to achieve it. I have a header file which contains this statement

#define "<huge string containing 20598617 characterd>" 

When I try to compile the program I get error as fatal error C1060: compiler is out of heap space

I tried following command line options with no success

/Zm200 /Zm1000 /Zm2000 

How can I make successful compilation of this program?

Platform: Windows 7

Answer 1

You can't, not reliably. Even if it will compile, it's liable to break the runtime library, or the OS assumptions, and so forth.

If you tell us why you're trying to do it, we can offer lots of alternatives. Deciding how to handle arbitrarily large data is a major part of programming.

Edited to add:

Rather than guess, I looked into MSDN:

Prior to adjacent strings being concatenated, a string cannot be longer than 16380 single-byte characters.

A Unicode string of about one half this length would also generate this error.

The page concludes:

You may want to store exceptionally large string literals (32K or more) in a custom resource or an external file.

What do other compilers say?

Further edited to add:

I created a file like this:

char s[] = {'x','x','x','x'}; 

I kept doubling the occurrences of 'x', testing each one as an #include file.

An 8388608 byte string succeeded; 16777216 bytes failed, with the "out of heap space" error.

Answer 2

I suspect you are running into a design limit on the size of a character string. Most people really think that a million characters is long enough :-}

To avoid such design limits, I'd try not to put the whole thing into a single literal string. On the suspicion that #define macro bodies likewise have similar limits, I't try not to put the entire thing in a single #define, either.

Most C compilers will accept pretty big lists of individual characters as initializers. If you write

char c[]={ c1, c2, ...  c20598617 }; 

with the c_i being your individual characters, you may succeed. I've seen GCC2 applications where there were 2 million elements like this (apparantly they were loading some type of ROM image). You might even be able to group the c_i into blocks of K characters for K=100, 1000, 10000 as suits your tastes, and that might actually help the compiler.

You might also consider running your string through a compression algorithm, putting the compressed result into your C++ file by any of the above methods, and decompressing after the program was loaded. I suspect you can get a decompression algorithm into a few thousand bytes.