fuse: Setting offsets for the filler function in readdir

Question

I am implementing a virtual filesystem using the fuse, and need some understanding regarding the offset parameter in readdir.

Earlier we were ignoring the offset and passing 0 in the filler function, in which case the kernel should take care.

Our filesystem database, is storing: directory name, filelength, inode number and parent inode number.

How do i calculate get the offset?

Then is the offset of each components, equal to their size sorted in incremental form of their inode number? What happens is there is a directory inside a directory, is the offset in that case equal to the sum of the files inside?

Example: in case the dir listing is - a.txt b.txt c.txt
And inode number of a.txt=3, b.txt=5, c.txt=7

Offset of a.txt= directory offset
Offset of b.txt=dir offset + size of a.txt
Offset of c.txt=dir offset + size of b.txt

Is the above assumption correct?

P.S: Here are the callbacks of fuse

Answer 1

The selected answer is not correct

Despite the lack of upvotes on this answer, this is the correct answer. Cracking into the format of the void buffer should be discouraged, and that's the intent behind declaring such things void in C code - you shouldn't write code that assumes knowledge of the format of the data behind void pointers, use whatever API is provided properly instead.

The code below is very simple and straightforward, as it should be. No knowledge of the format of the Fuse buffer is required.

Fictitious API

This is a contrived example of what some device's API could look like. This is not part of Fuse.

// get_some_file_names() - 
// returns a struct with buffers holding the names of files.
// PARAMETERS
// * path   - A path of some sort that the fictitious device groks.
// * offset - Where in the list of file names to start.
// RETURNS
// * A name_list, it has some char buffers holding the file names
//   and a couple other auxiliary vars.
//   
name_list *get_some_file_names(char *path, size_t offset);

Listing the files in parts

Here's a Fuse callback that can be registered with the Fuse system to list the filenames provided by get_some_file_names(). It's arbitrarily named readdir_callback() so its purpose is obvious.

int readdir_callback(      char  *path, 
                           void  *buf,      // This is meant to be "opaque".
                fuse_fill_dir_t  *filler,   // filler takes care of buf.
                          off_t  off,       // Last value given to filler.
          struct fuse_file_info  *fi        )
{
    // Call the fictitious API to get a list of file names.
    name_list *list = get_some_file_names(path, off);

    for (int i = 0; i < list->length; i++)
    {
        // Feed the file names to filler() one at a time.
        if (filler(buf, list->names[i], NULL, off + i + 1)) 
        { 
            break;   // filler() returned 1, requesting a break.
        }
        incr_num_files_listed(list);
    }

    if (all_files_listed(list))
    {
        return 1;    // Tell Fuse we're done.
    }

    return 0;
}

The off (offset) value is not used by the filler function to fill its opaque buffer, buf. The off value is, however, meaningful to the callback as an offset base as it provides file names to filler(). Whatever value was last passed to filler() is what gets passed back to readdir_callback() on its next invocation. filler() itself only cares whether the off value is 0 or not-0.

Indicating "I'm done listing!" to Fuse

To signal to the Fuse system that your readdir_callback() is done listing file names in parts (when the last of the list of names has been given to filler()), simply return 1 from it.

How off Is Used

The off, offset, parameter should be non-0 to perform the partial listings. That's its only requirement as far as filler() is concerned. If off is 0, that indicates to Fuse that you're going to do a full listing in one shot (see below).

Although filler() doesn't care what the off value is beyond it being non-0, the value can still be meaningfully used. The code above is using the index of the next item in its own file list as its value. Fuse will keep passing the last off value it received back to the read dir callback on each invocation until the listing is complete (when readdir_callback() returns 1).

Listing the files all at once

int readdir_callback(      char  *path, 
                           void  *buf, 
                fuse_fill_dir_t  *filler, 
                          off_t  off,
          struct fuse_file_info  *fi        )
{
    name_list *list = get_all_file_names(path);

    for (int i = 0; i < list->length; i++)
    {
        filler(buf, list->names[i], NULL, 0);
    }
    return 0;
}

Listing all the files in one shot, as above, is simpler - but not by much. Note that off is 0 for the full listing. One may wonder, 'why even bother with the first approach of reading the folder contents in parts?'

The in-parts strategy is useful where a set number of buffers for file names is allocated, and the number of files within folders may exceed this number. For instance, the implementation of name_list above may only have 8 allocated buffers (char names[8][256]). Also, buf may fill up and filler() start returning 1 if too many names are given at once. The first approach avoids this.

Answer 2

The offset passed to the filler function is the offset of the next item in the directory. You can have the entries in the directory in any order you want. If you don't want to return an entire directory at once, you need to use the offset to determine what gets asked for and stored. The order of items in the directory is up to you, and doesn't matter what order the names or inodes or anything else is.

Specifically, in the readdir call, you are passed an offset. You want to start calling the filler function with entries that will be at this callback or later. In the simplest case, the length of each entry is 24 bytes + strlen(name of entry), rounded up to the nearest multiple of 8 bytes. However, see the fuse source code at http://sourceforge.net/projects/fuse/ for when this might not be the case.

I have a simple example, where I have a loop (pseudo c-code) in my readdir function:

int my_readdir(const char *path, void *buf, fuse_fill_dir_t filler, off_t offset, struct fuse_file_info *fi)
{
   (a bunch of prep work has been omitted)
   struct stat st;
   int off, nextoff=0, lenentry, i;
   char namebuf[(long enough for any one name)];

   for (i=0; i<NumDirectoryEntries; i++)
   {
      (fill st with the stat information, including inode, etc.)
      (fill namebuf with the name of the directory entry)
      lenentry = ((24+strlen(namebuf)+7)&~7);
      off = nextoff; /* offset of this entry */
      nextoff += lenentry;
      /* Skip this entry if we weren't asked for it */
      if (off<offset)
         continue;
      /* Add this to our response until we are asked to stop */
      if (filler(buf, namebuf, &st, nextoff))
         break;
   }
   /* All done because we were asked to stop or because we finished */
   return 0;
}

I tested this within my own code (I had never used the offset before), and it works fine.