I need to allocate huge file without zeroing it's content. I'm producing this steps fopen => ftruncate => fclose => mmap => (...work...) => munmap
with huge file sizes (hundreds of gigabytes). App hangs on termination for a few minutes while system is trying zeroing file bytes – IMHO because of ftruncate
usage.
ftruncate(ofd, 0);
#ifdef HAVE_FALLOCATE
int ret = fallocate(ofd, 0, 0, cache_size);
if (ret == -1) {
printf("Failed to expand file to size %llu (errno %d - %s).\n", cache_size, errno, strerror(errno));
exit(-1);
}
#elif defined(HAVE_POSIX_FALLOCATE)
int ret = posix_fallocate(ofd, 0, cache_size);
if (ret == -1) {
printf("Failed to expand file to size %llu (errno %d - %s).\n", cache_size, errno, strerror(errno));
exit(-1);
}
#elif defined(__APPLE__)
fstore_t store = {F_ALLOCATECONTIG, F_PEOFPOSMODE, 0, cache_size, 0};
int ret = fcntl(ofd, F_PREALLOCATE, &store);
if (ret == -1) {
store.fst_flags = F_ALLOCATEALL;
ret = fcntl(ofd, F_PREALLOCATE, &store);
}
if (ret == -1) { // read fcntl docs - must test against -1
printf("Failed to expand file to size %llu (errno %d - %s).\n", cache_size, errno, strerror(errno));
exit(-1);
}
struct stat sb;
ret = fstat(ofd, &sb);
if (ret != 0) {
printf("Failed to write to file to establish the size.\n");
exit(-1);
}
//ftruncate(ofd, cache_size); <-- [1]
#endif
It seems it does not work with commented line [1]
. But uncommenting this line produces file zeroing which I am trying to avoid. I really don't care dirty file content before writing. I just wanna avoid hang on app termination.
SOLUTION:
According to @torfo's answer, replaced all my Apple-related code with this few lines:
unsigned long long result_size = cache_size;
int ret = fcntl(ofd, F_SETSIZE, &result_size);
if(ret == -1) {
printf("Failed set size %llu (errno %d - %s).\n", cache_size, errno, strerror(errno));
exit(-1);
}
But only works for superuser!
This is MacOS X, apparently.
You can try to replace the ftruncate
call with
fcntl(ofd, F_SETSIZE, &size);
(note needs root privilege and might create a security hole because it might provide access to old file contents that was previously there, so must be handled with extreme caution. The "dirty file content" you don't care about might actually be the user's bank account passwords he deleted a week ago...)
MacOS X doesn't really support sparse files - It does create and maintain them, but its file system driver is very eager to fill the holes as soon as possible.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With