Asynchronous code in Node.js Express

Question

I'm trying to create a MVC pattern with Node.js Express, but this seems to be a quite impossible task when doing asynchronous code.

Case in point: I'm fetching results from a NeDB database in the model, like so:

controllers/database.js

// Database management stuff with NeDB
var data = {};
db.findOne({ _id: 1 }, (error, doc) => {
    if (error) console.error(error);
    data = doc;
}

And now, I'm going to use it in a controller called dbcontroller.js:

var nedb = require('../models/nedb.js');

module.exports.list = function(req, res, next) {
    res.render('listpage', {
        data: nedb.data,
    });

And in the routes/routes.js file:

var express = require('express');
var router = express.Router();
var dbcontroller = require('../controllers/dbcontroller.js');
// Other controllers...

router.get('/list', dbcontroller.list);
// Other route definitions...

module.exports = router;

Cute arrangement, isn't it? Now, sure enough, the NeDB findOne() function is asynchronous, so it's going to execute after module.exports, and that won't do. The solution that came to mind was, of course, putting the module.exports definition inside the findOne() callback!

db.findOne({ _id: 1 }, (error, doc) => {
    if (error) console.error(error);
    module.exports.data = data;
}

I'm sure this is an anti-pattern somewhere, but it works. Or does it? The real drama starts now.

When it's called by the controller, the findOne() function also finishes after all of the controller logic, meaning nedb.data will be undefined. I'll have to do some crazy stunt now...

database.js

module.exports.execute = function (callback) {
    db.findOne({ _id: 1 }, (error, doc) =>
    {
        if (error) console.error(error);
        module.exports.data = doc;
        callback();
    });
}

dbcontroller.js:

nedb.execute(export);

function export() {
    module.exports.list = function(req, res, next) {
        res.render('listpage', {
            data: nedb.data,
        });
    };
};

And it gets worse. The routes.js file now can't find dbcontroller.list, presumably because it's still defined after the route asks for it. Now, will I have to start putting route definitions in callbacks too?

My point is, this whole asynchronicity thing seems to be completely taking over my code structure. What was previously tidy code now is becoming an ugly mess because I can't put code in the place it belongs. And that's just one async function, NeDB has lots of asynchronous functions, and while that's great, I don't know if I can handle it if just findOne() is already giving me such a headache.

Maybe I don't really have a choice and I must mess up my code in order for it to work? Or maybe I'm missing something really basic? Or maybe libraries like async or promises will do what I'm looking for? What do you think?

Answer 1

In order for async code to work, you need to be notified of when it's done by either a callback, or promises, or something else. To make your examples above work, all you need to do is call res.render after you're sure that data is defined. To do this, you can either return a promise from your database fetches, or pass it a callback. I would suggest promises, since they're much easier to use because they help avoid "callback hell", and Node has had native support for them for a long time now.

controllers/database.js

module.exports.fetch = function() {
  return new Promise(function(resolve, reject) {
    db.findOne({ _id: 1 }, (error, doc) => {
      if (error) reject(error);
      resolve(doc);
   });
}

Then, in your controller, you just need to call fetch (or however you end up defining your database fetching code) before you render your response.

dbcontroller.js

var nedb = require('../models/nedb.js');

module.exports.list = function(req, res, next) {
  nedb.fetch().then((data) => {
    res.render('listpage', {
        data: nedb.data,
    });
  }).catch((err) => { /* handle db fetch error */ });
}

The thing to keep in mind is that express doesn't expect you to call res.render right away. You can execute a lot of async code, and then render the response when it's done.

Answer 2

It is usually bad practice to set module data inside of callback functions since require loads modules synchronously. Passing ES6 Promises between modules is a possible option to eliminate the need for excessive callbacks.

database.js

module.exports = new Promise(function (resolve, reject) {
    db.findOne({ _id: 1 }, (error, doc) => {
        if (error) reject(error);
        resolve(doc);
    });
});

dbcontroller.js

var nedb = require('../models/nedb.js');

module.exports.list = nedb.then(data => {
    return (req, res, next) => { //return router callback function
        res.render('listpage', { data }) //ES6 shorthand object notation
    }
}).catch(/* error handler */);

Further guidance can be found in this answer: Asynchronous initialization of Node.js module