Is it bad to not use a DB but use in memory objects?

Question

I've been tasked to write a small app to be used by a single user. This app will pull in ~500 employee names/departments from our master employee DB. Then the user will enter like 5 fields for each employee. Those 5 fields will typically only change once a year, but could be once a month worst case. I only am supposed to keep track of 2 years worth at any given time.

I've looked at SQLite and SQL CE and I'm just not thrilled by either of them. SQL CE doesn't want to allow the data file to reside on a network share. (Only a single user but they store all their documents on their private share that is backed up daily).

SQLite seems like it would fit the bill better but it doesn't integrate as well into Visual Studio without wrappers or anything.

The other thing to consider is that our people are versed in MS' SQL Server and little else so having something that they understand vs SQLlite will be an important thing to my boss.

So my question is: What if I store the data in Objects in memory and serialize them to disk when saving. I've done a quick test and with 10k people (our use will only be 500-1000 max) and 10 years each (or 10 months if they update their data every month, highly unlikely) only caused my demo app to use 30MB of memory. Also populating that data was instantanous even with using GUID's to randomly fill all the strings. Is this a bad idea? Its a fairly simple app and in this case it seems ok to me.

Answer 1

I see a few issues with the idea of persisting business data using object serialization:

These aren't necessarily show-stoppers for the idea, but rather something to think about...

1. The data can't be queried, reported or inspected. It's entirely opaquely captured by the application.
2. Debugging serialized data is harder than being able to view the corresponding data in a database, or even a format like CSV.
3. There's no atomicity - it possible to corrupt your entire "database" with one power failure or application crash.
4. If the data model changes, updating the existing persisted entities requires a version of the app that can read both the old and new format. With a database, you can just add a column (or sub table).
5. There's no clean way to implement concurrent access. What happens if more than one user want to view or edit the data?

One thing I've learned, is that small apps tend to grow and become "large apps". When organizations guess incorrectly about the potential value of an application, they tend to incur the costs of this kind of unexpected, organic growth later.

You also mention that you liked at SQLLite and didn't like it. What is it that you didn't like? What kinds of problems did you anticipate?

If you're just looking for a way to "cut corners" to get this done quicker - that may be ok in the short term - but be careful - these kinds of decisions have way of coming back to bite you.

Answer 2

By serializing the data you lose:

• SQL-style search
• ability to insert/update/delete individual records
• well known and understood
• not language specific
• accessibility for other and non-local applications

You gain

• easy to code
• simple backup (make sure you've thought about backup!)
• fewer dependencies

From your description of your goals and constraints I can't see any specific issues with your approach.

Another thought. It sounds like you're saving a simple table-like data structure, so you might want to thing about saving it in a human-readable form like a comma-separated-values file or even XML. That way you're not dependent on the language you're currently using.