Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How avoid using global variable when using nftw

I want to use nftw to traverse a directory structure in C.

However, given what I want to do, I don't see a way around using a global variable.

The textbook examples of using (n)ftw all involve doing something like printing out a filename. I want, instead, to take the pathname and file checksum and place those in a data structure. But I don't see a good way to do that, given the limits on what can be passed to nftw.

The solution I'm using involves a global variable. The function called by nftw can then access that variable and add the required data.

Is there any reasonable way to do this without using a global variable?

Here's the exchange in previous post on stackoverflow in which someone suggested I post this as a follow-up.

like image 979
user1071847 Avatar asked Apr 23 '12 13:04

user1071847


1 Answers

Using ftw can be really, really bad. Internally it will save the the function pointer that you use, if another thread then does something else it will overwrite the function pointer.

Horror scenario:

thread 1:  count billions of files
thread 2:  delete some files
thread 1:  ---oops, it is now deleting billions of 
              files instead of counting them.

In short. You are better off using fts_open.

If you still want to use nftw then my suggestion is to put the "global" type in a namespace and mark it as "thread_local". You should be able to adjust this to your needs.

/* in some cpp file */
namespace {
   thread_local size_t gTotalBytes{0};  // thread local makes this thread safe
int GetSize(const char* path, const struct stat* statPtr, int currentFlag, struct FTW* internalFtwUsage) {
    gTotalBytes+=  statPtr->st_size;
    return 0;  //ntfw continues
 }
} // namespace


size_t RecursiveFolderDiskUsed(const std::string& startPath) {
   const int flags = FTW_DEPTH | FTW_MOUNT | FTW_PHYS;
   const int maxFileDescriptorsToUse = 1024; // or whatever
   const int result = nftw(startPath.c_str(), GetSize, maxFileDescriptorsToUse , flags);

  // log or something if result== -1
  return gTotalBytes;
}
like image 148
Kjell Hedström Avatar answered Oct 24 '22 12:10

Kjell Hedström