XML Schema Migration

Question

I am working on a project where we need to save data in an XML format. The problem is, over time we expect the format / schema for our data to change. What we want to be able to do is to produce scripts to migrate our data across different schema versions. We distribute our product to thousands of customers so we need to be able to run / apply these scripts at customer sites (so we can't just do the conversions by hand). I think that what we are looking for is some kind of XML data migration tool. In my mind the ideal tool could:

1. Do an "XML diff" of two schema to identify added/deleted/changed nodes.

2. Allow us to specify transformation functions. So, for example, we might add a new element to our schema that is a function of the old elements. (E.g. a new element C where C = A+B, A + B are old elements).

So I think I am looking for a kind of XML diff and patch tool which can also apply transformation functions. One tool I am looking at for this is Altova's MapForce . I'm sure others here have had to deal with XML data format migration. How did you handle it?

Edit: One point of clarification. The "diff" I plan to do is on the schema or .xsd files. The actual changes will be made to particular data sets that follow a given schema. These data sets will be .xml files. So its a "diff" of the schema to help figure out what changes need to be made to data sets to migrate them from one scheme to another.

Answer 1

"Do an "XML diff" of two schema to identify added/deleted/changed nodes."

XSD's are text, so this is trivial.

However, if you make dramatic structural changes to the XSD's, the automated diff will be largely useless.

If you make small, cosmetic changes to the XSD's, this might be helpful.

"Allow us to specify transformation functions..."

Wouldn't that be nice. Sadly, the odds of there being some trivial change ("new element C where C = A+B, A + B are old elements") are almost nil. Why make that kind of trivial change?

No, when you "... distribute our product to thousands of customers", you don't make trivial, cosmetic changes. You save up the changes so that they are truly epic, and "create significant value."

No, the odds of there being an automated schema migration is nearly zero.

Instead, design for migratability.

1. Be sure that the version number is prominent in your XSD paths. Ideally, in the XSD name itself.

2. Each XSD change is a Serious Governance Issue (SGI™). Everyone participates. And you write the migration scripts right then and there. Not afterwards. Not with tools. But as part of an XSD change.

   Schema don't spontaneously change. Someone changes them for a reason. That someone can specify the changes so someone else can write (or update) the migration script.

This is far, far too serious to trust to "automated" tools. This requires real brains of real people really focusing on this as if their jobs depended on it.