Structure of big nodejs project - How to handle interdependencies

Question

I have a nodejs project that is getting rather large rather fast. Due to my limited understanding of modules and require I am sure that I can do some things to clean up the code and structure.

<app dir>
  -app.js
  - <modules>
     -module1.js
     -module2.js
  - <routes>
     -route1.js
     -route2.js
  - <utilties>
     -utility1.js
     -utility2.js
  - <authentication>
     -local.js
     -basic.js
     -index.js

My app.js is pretty messy as I have dependencies between my files. IE I want to pass my models and authentication to my routes, utilities can be used by a bunch of things.

var app = express();
... // do express setup

var authenticationStrategy = // get auth strategy from config
var auth = require('./auth)(authenticationStrategy);

var utility1 = require('./utilities/utility1.js');
var utility2 = require('./utilities/utility2.js');
var utilities = {
  utility1: utility1,
  utility2: utility2
}

var Model1 = require('./models/model1')(utilities);
var Model2 = require('./models/model2')(utility1);
var models = {
  Model1: Model1,
  Model2: Model2
}

// Dynamically import all routes
fs.readdirSync('routes').forEach(function(file) {
  if (file[0] == '.') return;
  var route = file.substr(0, file.indexOf('.'));
  require('./routes/' + route)(app, models, utilities);
});

...

etc

I now know that I can put an index.js in each folder to clean things up but that still leaves me with having to save off things like utilities and pass that into other require calls.

I can put each require in lower modules only where its needed but then I end up climbing the directory structure with my require which also seems messy, i.e.:

model1.js: var utility1 = require('../utilities/utility1.js');

Basically I think my problem is that modules in lower level folders depend on other modules in other folders. With I feel like I should pull in all dependencies in app.js and pass them to the require modules that need them.

Any advice. I have been trying to restructure this for a couple days and I just keep banging my head against the wall as nothing I do is really making it any better.

Best would be a good node project layout that uses things like mongoose, express, w/ custom modules. and show a good way to handle interdependencies between modules.

Answer 1

Edit: https://github.com/gagle/node-getmod

I also had headaches to properly structure my first real and BIG node.js project. The biggest problem you have is the relative path used in the require() calls. It's a pain because these paths are relative to the current file, so you tipically have paths like these:

../../../a.js
./a/b/c/a.js
../../a/b/c/a.js

All of them pointing to the same file. This is inhuman. Hard to read, hard to maintain.

You shouldn't put your modules inside the node_modules directory and simply require the module by its name (require("a")) because this directory is used by third party modules. If you need to do maintenance tasks, like perform a full upgrade of all modules, deleting them and installing them again, etc. they are extremly easy and safe to do. If you put your there your modules you'll have a big problem if you don't have a good documentation and you work in a medium-big team.

After a few hours I ended with this technique:

NodeJS local modules for complex application structures

With the mod() function you have a good control over your modules. In my real and BIG project my mod function is the following:

"use strict";

var serverType = require ("./server-type");

global.mod = function (file){
    var dir = file.substring (0, file.search (/\/|\\/));
    if (!dir) dir = file;
    if (dir === "browser" || dir === "common" || dir === "mobile" ||
            dir === "stream"){
        //Relative from the web directory
        return require ("../" + file);
    }
    if (dir === "api"){
        //Relative from the api directory
        return require ("../../" + (file === "api" || file === "api/latest"
                ? file
                : "api/" + APP.versions.api + "/" + file.substring (4)));
    }
    if (dir === "build"){
        //Relative from the build directory
        return require ("../../" + file);
    }
    //Tries to load the module inside browser or mobile directories depending on
    //the cwd
    return require ("../" + serverType () + "/" + file);
};

If I call to mod("api") it returns my api module, the core of my website.

If I call to mod("common/db") it returns the db wrapper.

If I call to mod("build/...") it returns the module needed to build the static assets (compiles css and js)

etc

---

Now, you misunderstand how the modules must be used. Example:

var utility1 = require('./utilities/utility1.js');
var utility2 = require('./utilities/utility2.js');
var utilities = {
  utility1: utility1,
  utility2: utility2
}

var Model1 = require('./models/model1')(utilities);
var Model2 = require('./models/model2')(utility1);
var models = {
  Model1: Model1,
  Model2: Model2
}

Here you are passing a the utilities module to another module and this is not needed. You can get the utilities module from inside the model1 and model2 modules.

Very basic example:

A depends on B

//app.js
require ("./a").a ();

//a.js
var b = require ("./b");
module.exports = {
  a: function (){
    b.b ("I'm A");
  }
};

//b.js
module.exports = {
  b: function (msg){
    console.log ("B says: " + msg);
  }
};

You don't need to do this:

//app.js
var b = require ("./b");
require ("./a").a (b);

//a.js
module.exports = {
  a: function (b){
    b.b ("I'm A");
  }
};

//b.js
module.exports = {
  b: function (msg){
    console.log ("B says: " + msg);
  }
};

Is this what you are looking for?

Answer 2

1- create a file in the project root call it settings.js

2- inside this file add this code

module.exports = {
    POST_MAX_SIZE : 40 , //MB
    UPLOAD_MAX_FILE_SIZE: 40, //MB
    PROJECT_DIR : __dirname
};

3- inside node_modules create a new module name it "settings" and inside the module index.js write this code:

module.exports = require("../../settings");

4- and any time you want your project directory just use

var settings = require("settings");
settings.PROJECT_DIR; 

in this way you will have all project directories relative to this file ;)