Import an Existing JavaScript Project into a Grunt/Brunch Project

Question

I watched Paul Irish's talk announcing Yeoman (www.yeoman.io), and I'm hooked on the concept of running a continuous build environment. Not content to wait for a Yeoman invite, I tried Grunt and Brunch. Both install easily, and I can get new projects up and running with minimal effort.

I don't understand how one would migrate an existing project into either platform. My project uses a single namespace and uses two conventions for modules (one for instancing another for utility), each of which are wrapped in self-executing anonymous functions which export to the instance or the namespace.

I have at least 200 modules and many more simple, helper function exports to the namespace; so it is not at all efficient to use the console to create these in a grunt/brunch project and then manually import each module individually. Further, I'm using at least 15 different 3rd-party JavaScript tools. It is not clear to me how to bring these in.

What is the most efficient way to take a large, existing project and migrate it into Grunt/Brunch with the least amount of refactoring and support for arbitrary 3rd party tools?

Update: of the two, I've found Brunch a bit easier to cope with. If you use the stock "skeleton" (that is "template"--from the command line {in the folder you want the change to occur} execute "brunch new [project_name] --skeleton git://github.com/brunch/simple-js-skeleton.git") for pure JS, you get a new folder structure which is actually quite responsive. Anything you drop into to 'app' (your own code) or 'vendor' (3rd-party) folders will get automatically recompiled for you on file edit (when you run "brunch watch").

This is great, except. According the documentation, you control the order vendor scripts are compiled and concatenated together from the Brunch config.coffee file (JSON text file). Changes to this file seem to have no effect, thus you end up with 3rd party race conditions from plugins expecting other plugins.

Further, when you drop your own code into the auto-created 'app' folder you do get an auto-compiled, real-time, as-you-edit version of your code; but it's not accessible. Brunch obfuscates the window object, so my initial namespace declaration to window.myNameSpace fails and all subsequent library calls to the namespace fail as well. This has something to do with Brunch's module system, for which I can find no documentation.

I solved this by placing my namespace class in the 'vendor' folder, which ensured that it attached to the window object; however, now there is a race condition: my namespace isn't always available for all of my modules.

The problem is now this:

Once you have copied all of your internal and external libraries into a Brunch project, how do you configure the app to load them in a sane order?

Answer 1

This is a bit of an opus, but I finally figured it out. When I started with Brunch, it was not obvious how to make the first step: import my directory structure. It took me a few passes over the documentation, before it became obvious:

1. Execute brunch new MyAppName -s https://github.com/damassi/Javascript-App-Skeleton, which will generate a skeleton folder structure and config.coffee file
2. For me, the only relevant folders in this structure were 'app' (the raw src content for CSS, JS and HTML), 'public' (the destination for compiled content and the location servicing the NodeJS server) and 'vendor' (the place for 3rd party files).
3. Brunch creates a config.coffee file at the root of the directory structure with this content: files: javascripts: defaultExtension: 'js' joinTo: 'javascripts/app.js': /^app/ 'javascripts/vendor.js': /^vendor/ order: before: [ 'vendor/scripts/console-helper.js', 'vendor/scripts/jquery-1.7.1.min.js' ]
4. The 'joinTo' property of this object confused me, until I realized that 'javascripts' is really just a mask for 'client-side code' and that 'apps.js' is effectively a call to 'get all *.js files in the folder "app", recursively'.
5. Once this is clear, all you need to do is drop your content into 'app'. I placed my *.html and images files into the 'assets' subfolder and put all of my JavaScript content into lib.
6. At this point, you can run brunch build and brunch watch, and your project is up and running, compiling in real-time as you make changes, live reloading in the browser.

While Brunch is better out of the box than Grunt at the ease of using step 6, where it failed for me is the nature of compilation in Brunch. Every JavaScript file gets wrapped in a CommonJS module and the module name is based on the relative path and file name ('lib/core/ajax', etc.). The CommonJS philosophy is not for me, and the work involved in refactoring my library to use CommonJS is huge.

So, back to Grunt. Once I understood how to import a project into Brunch, importing into Grunt was a snap. I'm on windows, so all grunt calls use grunt.cmd.

1. Call grunt init:jquery (this can be anywhere, I moved the created directory structure into my existing project folder)
2. Like Brunch, you get an auto generated directory structure and config file (grunt.js), but it's much, much thinner. Grunt's config looks like this: concat: { dist: { src: ['<config:lint.files>'], dest: 'dist/<%= pkg.name %>.js' } }, min: { dist: { src: ['<banner:meta.banner>', '<config:concat.dist.dest>'], dest: 'dist/<%= pkg.name %>.min.js' } }, qunit: { files: ['test/**/*.html'] }, lint: { files: ['grunt.js', 'src/**/*.js', 'test/**/*.js'] }, watch: { files: '<config:lint.files>', tasks: 'lint qunit' }
3. This looked a bit alien to my mind at first, but it's actually quite elegant. The 'min' property defines the final, concatenated, linted and minified file that my web app will be serving. Its source value is '', which is Grunt magic to look at the value of concat object's dist dest property value, which is then derived from lint's file's property value. So, you define the resources you want to be linted, concatenated, minified and output to a destination at the lint level.
4. Once this piece is in place, you have to do a bit of extra work to get the build, watch and server pieces in place. In grunt, when the server is finished executing, it quits. That means that if you execute the grunt server task, it will start the server and with no other tasks to do, quit.
5. My first mistake was to bundle the server task with the watch's task, by setting watch.task = 'server lint qunit'. This works for the first change you make to the source, but the second change will attempt to start a second instance of the server on the same port and fail. Instead, you can register a task grunt.registerTask('dev', 'server watch qunit'); and call grunt dev to get a server running with real-time, continuous build.
6. Next, my HTML content depended on server-side includes to assemble the page. I couldn't figure out how to get this working in Node, and client-side includes using <object/> don't work, as they insert the content (in my case various <script/> and <link/> elements) into an Iframe, which of course breaks my module pattern (My namespace is in a different window object than the window object of the iframes). Fortunately, grunt's concat object is a multitask and it can concatenate anything. So I added my HTML files to concat, and my single-page app was ready to go.
7. Next, because the Node server is running on a different port than my IIS instance, you have the cross domain ajax issue. This SO article started me on the right path, but I ended up needing the following changes to IIS default content headers: Access-Control-Allow-Credentials: true Access-Control-Allow-Headers: X-Requested-With, X-Prototype-Version, Content-Type, Origin, Allow Access-Control-Allow-Methods: PUT, GET, POST, DELETE, OPTIONS Access-Control-Allow-Origin: http://localhost:88
8. Finally, since I'm using jQuery for my default AJAX handler, I needed to add this to my ajax options: xhrFields: { withCredentials: true }
9. Obviously, there are security implications here; but since this will only affect my dev environment and will not be pushed to Production, I think it's OK.
10. Last but not least, I spent an hour trying to debug an error on minification through Uglify, which was conveniently answered by this SO post. Since Visual Studio insists on inserting BOM all over the pace ("UTF-8 With Signature" is the euphemism), but UTF-8 Cast fixes this in quick order.

Once I figured all of this out, Grunt seems to be working pretty well. I have not yet had a chance to begin testing the actual process of development in this new continuous build environment; but this is what it took to get to being able to start.

Answer 2

config.coffee isn't really json rather than real js / coffeescript, but the order editing should work. Can you open an issue in brunch bugtracker with exact config order?

I don't think there's a fast way of rewriting your app to use modules rather than global window variables. Globals are considered as a bad taste, by the way. But your solution could work, yep.