Node.js and the Filesystem: is this a race condition?

Question

I have the following code inside a class. (It's coffeescript-- and it's for a couchdb utility!-- but this is really a node.js question). I'm attempting to do things The Node Way, using Node 0.49, and that means using asynchronous calls for filesystem operations. At first, I was pulling my hair out because this.sentinel went to zero several times during the course of processing, so I know I'm doing something wrong there. But then I hit an even weirder issue: down in load_directory, see those console.log() calls? Watch when happens when I run this.

check_sentinel: ->
    @sentinel--
    if @sentinel == 0
        @emit('designDirLoaded', @object)

load_file: (rootdir, filename, object) ->
    @sentinel++
    fname = path.join(rootdir, filename)
    @manifest.push(fname)
    fs.readFile fname, (err, data) =>
        object[filename] = data
        @check_sentinel()

load_directory: (dirpath, object) ->
    @sentinel++
    fs.readdir dirpath, (err, files) =>
        for fname in files
            console.log("X1: ", fname)
            fs.stat path.join(dirpath, fname), (err, stats) =>
                console.log("X2: ", fname)
                if stats.isFile()
                    @load_file(dirpath, fname, object)
                if stats.isDirectory()
                    object[fname] = {}
                    @load_directory(path.join(dirpath, fname), object[fname])
        @check_sentinel()

Here's what I get:

X1:  memberByName.js
X1:  memberByClub.js
X2:  memberByClub.js
X2:  memberByClub.js

This is surreal, and it looks a lot like a race condition. "memberByName" gets passed to fs.stat(), which in turn passes "memberByClub" to load_file(), implying... what? That because load_file() returned immediately, it raced around and presented the next file name in the array to the function call? Or do I have some misunderstanding about the persistence of values in a given scope?

Answer 1

No, what you see is expected. One thing you have to remember is that fs.stat is asynchronous. So, the outer loop (for fname in files) will finish looping before any of the callbacks to fs.stat is called.

The reason why you see memberByClub.js twice is that you are using fname in the logging statement, but that variable is from the closure, which has changed by the time your callback to fs.stat is called.

You can wrap the inner loop with do (fname) => to get the correct logging statements, but I think you need to restructure your code to achieve what you are trying to do with the whole class.

load_directory: (dirpath, object) ->
    @sentinel++
    fs.readdir dirpath, (err, files) =>
        for fname in files
            do (fname) =>
                console.log("X1: ", fname)
                fs.stat path.join(dirpath, fname), (err, stats) =>
                    console.log("X2: ", fname)
                    if stats.isFile()
                         @load_file(dirpath, fname, object)
                    if stats.isDirectory()
                        object[fname] = {}
                        @load_directory(path.join(dirpath, fname), object[fname])
        @check_sentinel()