fwrite() alternative for large files on 32-bit system

Question

I'm trying to generate large files (4-8 GB) with C code. Now I use fopen() with 'wb' parameters to open file binary and fwrite() function in for loop to write bytes to file. I'm writing one byte in every loop iteration. There is no problem until the file is larger or equal to 4294967296 bytes (4096 MB). It looks like some memory limit in 32-bit OS, because when it writes to that opened file, it is still in RAM. Am I right? The symptom is that the created file has smaller size than I want. The difference is 4096 MB, e.g. when I want 6000 MB file, it creates 6000 MB - 4096 MB = 1904 MB file.

Could you suggest other way to do that task?

Regards :)

Part of code:

unsigned long long int number_of_data = (unsigned int)atoi(argv[1])*1024*1024; //MB
char x[1]={atoi(argv[2])};

fp=fopen(strcat(argv[3],".bin"),"wb");

    for(i=0;i<number_of_data;i++) {
        fwrite(x, sizeof(x[0]), sizeof(x[0]), fp);
    }

fclose(fp);

Answer 1

fwrite is not the problem here. The problem is the value you are calculating for number_of_data.

You need to be careful of any unintentional 32-bit casting when dealing with 64-bit integers. When I define them, I normally do it in a number of discrete steps, being careful at each step:

unsigned long long int number_of_data = atoi(argv[1]); // Should be good for up to 2,147,483,647 MB (2TB)
number_of_data *= 1024*1024; // Convert to MB

The assignment operator (*=) will be acting on the l-value (the unsigned long long int), so you can trust it to be acting on a 64-bit value.

This may look unoptimised, but a decent compiler will remove any unnecessary steps.