How to debug slow Ember CLI/Broccoli builds

Question

My Ember CLI project is currently taking 8-9 seconds to build, and I'd like to understand why. The project is not that large (~180 files under app/ including hbs and scss).

Here's my brocfile: https://gist.github.com/samselikoff/874c90758bb2ce0bb210

However, even if I comment my entire Brocfile out and export just the app variable, the build still takes 5-6 seconds.

I'm not quite sure how debug. Here's my slowest tree logs:

Build successful - 8874ms.

Slowest Trees                  | Total          
-------------------------------+----------------
TreeMerger (appAndDependencies)| 1286ms         
TreeMerger (vendor)            | 1275ms         
CompassCompiler                | 1204ms         
StaticCompiler                 | 1185ms         
TreeMerger (stylesAndVendor)   | 1151ms         
TreeMerger (allTrees)          | 706ms          
StaticCompiler                 | 625ms    

Answer 1

UPDATE: If you are using ember-cli version 0.1.0 or newer, this hack is probably not necessary. ember-cli now symlinks files instead of copying. You may still get a performance improvement on windows or slow disks.

---

Broccoli (used by ember-cli) stores its temporary state in the file system, thus it's very file I/O dependent. Try to reduce the number of files in your public/, vendor/ and bower_components/ directories. All files inside these folders will be copied at least once per rebuild cycle. The size and number of files in the folders affects performance greatly.

Essentially, every time you change a file, broccoli is copying files between the many directories inside <ember app>/tmp/. In the case of your bower_components/ dir, it appears to be copying every single file more than once. It needs to do this because you might use app.import('some.js') in your Brocfile.js, you might also @import "some.scss" in SASS/LESS files. There is no way to know which files you actually need, so it copies all of them.

If you remove the files that you do not need from bower_components/ and vendor/, you will notice better build times.

A real world example

If you install the highcharts.com#3.0.5 bower dependency, you also get a special gift of 2829 files (198MB) in your bower_components/ dir. Imagine the unnecessary file system reads and copies that are happening there.

Here is a snippet of my cleaned dir structure:

$ find bower_components -type f | grep highcharts
bower_components/highcharts.com/js/highcharts-more.src.js
bower_components/highcharts.com/js/highcharts.src.js

Notice that only the .js files remain, I removed everything else. That's 2827 removed files. Highcharts is an extreme example, but most of your dependencies have 5 times as many files than you actually need.

Positive future ahead

The ember-cli team are hard at work improving performance of the underlying broccoli ecosystem. Work has already begun and some real world apps (with large trees) are seeing performance improvements reducing rebuild time from 4 seconds to 600ms. Using symlinks instead of copying is showing drastic improvements.

For those of us who have large scale apps, lots of bower deps and many crying team members, who need a solution now:

A temporary solution

One way of keeping your bower_components/ clean, is to check the dependencies into version control. This allows you to use git clean to prune your directory with ease:

bower install —-save d3
git add -—force bower_components/d3/d3.js # force, because bower_components/ is gitignored
git commit -m "Added d3.js"

// Brocfile.js
app.import('bower_components/d3/d3.js');

Every time you do a bower install you will likely get all the extra cruft back in your dir. git clean easily removes non version controlled files:

git clean -f -d -x bower_components/
ember serve

After doing this, it took a single rebuild (time to build after changing a file) from 20 seconds down to 3.5 seconds (we have a pretty large app).

If you do go this path, dont forget the bower deps needed by Ember:

bower_components/ember/ember.js
bower_components/ember/ember.prod.js
bower_components/ember-cli-shims/app-shims.js
bower_components/ember-cli-test-loader/test-loader.js
bower_components/ember-data/ember-data.js
bower_components/ember-data/ember-data.prod.js
bower_components/ember-load-initializers/ember-load-initializers.js
bower_components/ember-resolver/dist/modules/ember-resolver.js
bower_components/jquery/dist/jquery.js
bower_components/loader/loader.js
bower_components/handlebars/handlebars.js
bower_components/handlebars/handlebars.runtime.js

Here's the git command for you:

bower install
git add -f bower_components/ember/ember.js bower_components/ember/ember.prod.js  bower_components/ember-cli-shims/app-shims.js  bower_components/ember-cli-test-loader/test-loader.js  bower_components/ember-data/ember-data.js  bower_components/ember-data/ember-data.prod.js  bower_components/ember-load-initializers/ember-load-initializers.js  bower_components/ember-resolver/dist/modules/ember-resolver.js  bower_components/jquery/dist/jquery.js  bower_components/loader/loader.js  bower_components/handlebars/handlebars.js bower_components/handlebars/handlebars.runtime.js
git commit -m "Added ember-cli dependencies"
git clean -f -d -x bower_components/