Do I want an ORM?

Question

We have an object model being used across three applications. Two programs collect data, another reads it and generates reports. The system is very disconnected, so we cannot have a single database all the programs talk to.

Right now, the programs just use a common library to populate an object model and serialize/deserialize to the disk. Specifically, we're using XML serialization.

There are a couple problems with this model. 1) XML could be considered wasteful. The files could get large and unwieldy. Honestly, file size isn't a huge concern right now. 2) My biggest concern is memory foot print. The entire file is loaded into an object model, operated on, then saved.

Hopefully I've conveyed my worry, at some point we will run into memory issues with this application during runtime. Enough data will get collected into a single "database" (xml file) that it cannot be loaded into memory all at once.

What I would like to have, is access to my object model backed by file storage instead of memory. I want the changes to the object model to be minimal. When an object is accessed, it comes from the disk and when it is set, it is saved (automatically, if possible).

We have looked into NHibernate with SQLite, SQL Compact 4.0 and EF 4, and LINQ to XML (briefly). I've also used db4o in the past for caching objects to disk, but that was an unrelated project.

Before I dive in and commit time to learning one of these, I'd like to know if my idea makes sense. Can I have an object model that'll "magically" cache to a storage medium instead of just bloating my memory footprint infinitely? What's the shortest path to get this done, even if it isn't the most elegant?

Are there other technologies that could help me? Memory mapped files, linq-to-sql, Lazy(T) (for only fetching objects from files when needed possibly).

I realize this is an open ended question. I'm looking for a big picture response and details if someone out there has real world experience doing this. Links would be helpful...

Thanks.

Answer 1

Yes, ORM's map an object model to a relational model. They do a pretty good job of hiding all the plumbing work that goes into reading and writing data to the database, caching data, managing memory, etc. They are also very capable of caching parts of an object graph and can do things to improve performance such as lazy loading data.

Answer 2

I've just finished migrating a (mostly "inherited") web application backed by XML files to NHibernate exactly out of the same concerns. I also was in the same situation of having never used NHibernate before and wanting to learn in the process. The idea does make sense. You will indeed be able to load in memory just the parts you actually need (as opposed to the whole DB) and many more benefits than that.

Based on the other things you ask for (minimal changes to your object model, easy to migrate from actual to ORM-based applications) I'm not sure you'll get them so easily. With ORMs like NHibernate and EF4, model classes are very lightweight: they're basically little more than property containers. Applications based on XML files tend to have more logic directly in the model: that logic you will likely have to move to the data access layer. Redesigning the model and data access layer will likely be the most time consuming task you will face. I know it was for me.

Another thing I infer from your question (you say you can't have all three programs talking to the same DB and you mention SQLite and SQL Compact) is that you're replicating your data among your three applications by copying the file physically. How do you detect changes and how aligned do you need the 3 DBs to be? How do you currently merge changes (in case 2 of the 3 apps you have can write data)? Depending on how you replicate data, this is something an ORM may or may not help you with.

Edit some more points based on your comments

• If you want at one point to merge the files into one DB and you're not already sure it will be a Microsoft product, then NHibernate is the best choice. But if you are now using LINQ extensively (as per another of your comments) and want a more seamless transition I would go for EF4: Nhibernate.Linq is not really complete and some constructs don't work.
• Both NHibernate and EF4 are very documented and come with very good tutorials on how to build a complete application from scratch, so the learning part is really easy.
• Both have a "model first" approach where you get a tool that creates your DB for you based on your model classes, so if your model classes are very lightweight you should be able to use them with little modification.
• the thing that took me some time (and still is sometimes hard for me to get working) in NHibernate is configuring cascades to behave the way I expected: e.g. what happens to "related" objects when you delete an object?