Java JAXB - Writing XML files with restart logic

Question

I'm creating a very large XML file (700mb +) that process large amounts of data via batch. The program serves as an interface between a extremely large sybase database and an application. I currently have the xsd schema bound to classes. I need a way of being able to write the XML with restart logic in mind.

I.E. being able to know where I left off. Or in other words, if the program fails, I need to be able to see what the was last wrote to the XML file so it can pick up where I left off. Here's an exmaple.

<root>
  <WorkSet>
    <Work>
      <Customer>
    <Work>
      <Customer>
  <WorkSet>
    <Work>
      .....
<root>

Say the program fails after writing a write 'work' or 'workset' node. Is there a way to pick up where I left off processing? I'm trying to avoid reading the XML file back into memory due to the shear size of the XML file (Say it finishes 500mb of XML and fails).

Thanks for the help.

Answer 1

If you could split your data to independent WorkSet elements you can write them out one at a time with JAXB's fragment mode (when JAXB does not write the headers). Later simply concatenate the files and add the missing XML declaration, opening end closing tags.

It's is possible that you have to modify your generated classes for this. I mean adding @XmlRootElement to the WorkSet java class. If one WorkSet is still big for one step you can do this with Work too, but you have to generate somehow the missing tags.