Gulp - Handling multiple themes and folders

Question

I am trying to create an ultimate gulpfile that we can use on one of our big sites (one with multiple themes depending on the section of the site you are in). I'm trying to get it to only run the process it needs to run and not recompile everything.

Let me layout exactly what i'm trying to achieve:

Folder Structure

src/
    master-theme/
        css/
            style.scss
            partials/
                _a.scss
                _b.scss
        img/
            a.jpg
            b.jpg

    sub-theme/
        css/
            style.scss
            partials/
                _c.scss
                _d.scss
        img/
            c.png
            d.jpg

I want these files to be compressed/compiled and to end up in the destination folder with the same folder structure (just replace src with dest in your mind)

The Problem

At the moment i can get it to do what I want - but the gulpfile compiles and compresses everything. E.g. if I add an image tosub-theme/img it will run the image compression for all the "themes". I am using gulp-changed but it still means that it is looking at all the images accross the site.

The same is also for the sass - if I update _c.scss, but the master css and the sub-theme css get compiled which is obviously not desired.

Current Solution

I don't really have one at the moment. Right now I am using gulp-file-tree to generate a json file of the folder structure, then whenever a file is changed, looping through that with a function (which I know is horrible - but a solution which currently works)

var tree = require('./build/tree.json');
var children = tree.children;

for (var i = children.length - 1; i >= 0; i--) {

    var child = children[i];

    if(child.isDirectory)
        task(child)
}

There task() is a gulp tasks passed in (e.g. Sass compilation)

The folder structure is not up for discussion - I don't want this to turn into a 'structure your files differently' kind of thing. There are several other factors involved which are not related to this issue as to why we are this way (Sorry I had to say that...)

I'm open to trying anything as i've stared at this file for days now.The tasks I am trying to run are:

• Sass compilation
• Sprite generation
• SVG sprite to PNG sprite
• Image compression
• Javascript compression

Thanks in advance for your help. If a solution is found, I'll write a proper post about it so that others will hopefully not feel my pain...

Answer 1

I'm doing pretty much the same thing, and I think I've nailed it.

gulpfile.js:

var gulp = require('gulp'),
    debug = require('gulp-debug'),
    merge = require('merge-stream'),
    sass = require('gulp-sass'),
    less = require('gulp-less'),
    changed = require('gulp-changed'),
    imagemin = require('gulp-imagemin'),
    prefix = require('gulp-autoprefixer'),
    minifyCSS = require('gulp-minify-css'),
    browserSync = require('browser-sync'),
    reload = browserSync.reload,
    path = require('path'),
    glob = require('glob');

// Log errors to the console
function errorHandler(error) {
    console.log(error.toString());
    this.emit('end');
}

function processThemeFolder(src) {
    function debugTheme(type) {
        return debug({ title: 'theme ' + theme + ' ' + type});
    }

    var theme = path.basename(src);
    var dest = 'public/themes/' + theme;

    return merge(
        gulp
            .src([src + '/sass/**/*.scss'])
            .pipe(changed(dest + '/css', { extension: '.css' }))
            .pipe(debugTheme('sass'))
            .pipe(sass())
            .pipe(minifyCSS())
            .pipe(gulp.dest(dest + '/css')),
        gulp
            .src([src + '/less/**/*.less'])
            .pipe(changed(dest + '/css', { extension: '.css' }))
            .pipe(debugTheme('less'))
            .pipe(less())
            .pipe(minifyCSS())
            .pipe(gulp.dest(dest + '/css')),
        gulp
            .src([src + '/js/**/*.js'])
            .pipe(changed(dest + '/js'))
            .pipe(debugTheme('js'))
            .pipe(uglify())
            .pipe(gulp.dest(dest + '/js')),
        gulp
            .src([src + '/img/**/*.{png,jpg,gif}'])
            .pipe(changed(dest + '/img'))
            .pipe(debugTheme('img'))
            .pipe(imagemin())
            .pipe(gulp.dest(dest + '/img'))
    ).on('change', reload);
}

gulp.task('themes', function() {
    var srcThemes = glob.sync('resources/themes/*');
    return merge(srcThemes.map(processThemeFolder));
});

// ...

The key here is to use gulp-changed to only pass through the changed files. The rest is cream on top.

The compilation streams all show a debug line detailing what files are going into the stream. On a change in the stream, the browserSync is notified to reload the browsers, using streaming (if possible). The theme task is only completed once all its compilation streams are done, and the over-all themes task will only be marked as done when all the themes are done.

The theme's source files are stored in resources/themes/themename, and writes its output to public/themes/themename.

This is working very well for me, YMMV. :-)