How to Improve XML read write performance

Question

I have two .NET applications running independently(can start in any order, or may be only one running) which uses XML as data store. So both applications can read and write to XML file. To keep the data updated , i'm loading the XML file every time from the disk before read and write operations. And i'm using XPath query for querying the particular node. Now there is performance issue observed in this method as there are read and write requests on XML every second from one application(uses polling, and cannot be changed) I'm not sure what exactly is causing the performance hit, but i believe its continuous read write.

I tried using memory mapped files from .NET 4.0 but i'm restricted to use .NET 3.5 and not any higher versions.

Can anyone help me out on this?

Note : XML nodes have some common attributes , different number of attributes and one ID which i use for XPath querying.

Answer 1

IF you're sure that performance hit comes from I/O and you can't change both application there is really a little you can do.

First solution with zero changes to existing application code: use a RAM disk. If they're using that file as shared memory you can do it without any other change. If data is persistent you may need to perform a background copy to another media after each writing. Performance won't be as good as a true shared memory but at least you won't have to wait for slow I/O operations.

Second solution with changes only in the application that must read data: often the parsing of a XML file is pretty slow (specially if you're using XmlDocument and the file isn't very little). In this case, using XmlReader, you have to make your read code more complicated and to forget about XPath queries but its performance will be many times better than XmlDocument and it won't slow down increasing the file size.

Small (or not so small) updates: if code of the second application (I guess the one that will read the file) can be changed you can do a little to improve its performance. First of all do not read the file each time. Check its timestamp, register a FileSystemWatcher for that file or whatever else but do not read/parse file each time. When you did this you can go one step forward: read/parse the file only when it changes, prepare your XmlDocument on background (another thread) and make it available for polling requests. If requests are spaced they may even see a very quick response time (but profile performance of XmlDocument XPath query for your typical file).

EDIT: here you can find a RAM disk provided by Microsoft. It's pretty simple and naive but usually you/we don't need much more than that. Moreover it's an example on the DDK so you'll get source code too (in this case...just for fun).