Including Grails plugin migrations in the changelog of the application

Question

I am developing a modular application with Grails 2.x and I would like to organize our database migrations in a way that every plugin/module hosts their own migrations in their grails-app/migrations folder. (as opposed to putting all migrations into the migrations folder of the composing application project)

The composing application project itself normally doesn't have any migrations since these are stored in the plugins.

Is it possible to build a "master" changelog in the application project and only reference the plugin migration files in the proper order? This way I could have a very clean and migration system which handles plugin dependency problems as well (I place the plugin migration files references in the order which respects the plugin dependencies, which may change over time, so ordering of the migrations can be fairly complex).

By default the Database Migrations plugin doesn't seem to check/execute plugin migrations at all (inline plugins in my case). The docs don't say anything about this scenario, they concentrate on the simple, non-modular development workflow.

I have seen a solved issue with the DB Migrations plugin that somehow allows running migration files outside of the migrations directory of the app but I assume that would use some kind of hardcoded relative/absolute paths which would not be very elegant.

The most elegant solution would be if I could also specify the plugin for each migration "include" statement so my master changelog would look like this.

databaseChangeLog = {
  include plugin:'core'       ,file:'000-initial.groovy'
  include plugin:'accounting' ,file:'000-initial.groovy'
  include plugin:'core'       ,file:'001-drop-constr-XXX.groovy'
  include plugin:'accounting' ,file:'001-add-col-yyy-to-posting-table.groovy'
}

Is it possible to implement something similar with the current Database Migrations Plugin?

Any help/idea is appreciated.

Answer 1

I have worked out the following solution until the Grails Migrations plugin supports modular projects better:

During development, the plugin-related migrations are kept/developed in the plugin's "migration" folder and named as in the question (e.g.: 000-initial.groovy).

Each migration contains a package definition which relates it to the plugin:

package core

The migrations files should be placed into subfolders within the "migrations" folder according to the package definition.

A plugin-specific changelog can be created for testing the latest migrations within the plugin. In this case the plugin should have a database connection defined properly and ideally, this should point to a db schema reserved for plugin-testing. Naturally, the plugin's DB settings can point to the dev schema of the application itself

We will have the following files within the core plugin's "migrations" folder:

changelog.groovy
core/000-initial.groovy
core/001-drop-constr-XXX.groovy

When we need to test the migrations in the application environment (but latest in the release phase), we copy all of the tested migrations to the application project's "migrations" folder so we will get something like this:

core/000-initial.groovy
core/001-drop-constr-XXX.groovy
accounting/000-initial.groovy
accounting/001-add-col-yyy-to-posting-table.groovy

We include the migrations in the application's master changelog which will look like this (satisfying plugin dependencies):

databaseChangeLog = {    
  include file: 'core/000-initial.groovy'
  include file: 'accounting/000-initial.groovy'
  include file: 'core/001-drop-constr-XXX.groovy'
  include file: 'accounting/001-add-col-yyy-to-posting-table.groovy'
}

After copying the files we can test the migrations on the development database schema of the application.

If changes need to be made to the copied migrations we need to make sure that the changes are synchronized back to the original migration files in the plugin's migrations folder (since other applications may use those plugin migrations).

Answer 2

My solution is to define different folders like dev, test, live and common for common scripts between different environments.

The Directory Structure looks like:

P.S. changelog_ENVIRONMENT are under migrations

My Config.groovy looks like:

/**
* Database Migration Config
*/
//Enable database migration to run automatically
grails.plugin.databasemigration.updateOnStart = true

environments {
    development {
            grails.plugin.databasemigration.updateOnStartFileNames = ['changelog_dev.groovy']
    }
    test {
            grails.plugin.databasemigration.updateOnStartFileNames = ['changelog_test.groovy']
    }
    production {
        grails.plugin.databasemigration.updateOnStartFileNames = ['changelog_live.groovy']
    }
}

Then, the changelog_environment can refer to any folder. Note: Package name with just one word is used because java style package names (com.example.class) was not working for me.

Samples:
migrations/common/001-common-script.groovy

package common

databaseChangeLog = {

    changeSet(author: "author", id: "some-id") {

    }
}

migrations/dev/001-dev-script.groovy

package dev

databaseChangeLog = {

    changeSet(author: "author", id: "some-other-id") {

    }
}

migrations/changelog_dev.groovy

import dev.*
import common.*

databaseChangeLog = {

    include file: 'common/001-common-script.groovy'
    include file: 'dev/001-dev-script.groovy'
}