Do Websites need Local Databases Anymore?

Question

If there's a better place to ask this, please let me know.

Every time I build a new website/blog/shopping-cart/etc., I keep trying to do the following:

• Extract out common functionality into reusable code (Rubygems and jQuery plugins mostly)
• If possible, convert that gem into a small service so I never have to deal with a database for the objects involved (by service, I mean something lean and mean, usually built with the Sinatra Web Framework with a few core models).

My assumption is, if I can remove dependencies on local databases, that will make it easier and more scalable in the long run (scalable in terms of reusability and manageability, not necessarily database/performance). I'm not sure if that's a good or bad assumption yet. What do you think?

I've made this assumption because of the following reason:

Most serious database/model functionality has been built on the internet somewhere.

Just to name a few:

• Social Network API: Facebook
• Messaging API: Twitter
• Mailing API: Google
• Event API: Eventbrite
• Shopping API: Shopify
• Comment API: Disqus
• Form API: Wufoo
• Image API: Picasa
• Video API: Youtube ...

Each of those things are fairly complicated to build from scratch and to make as optimized, simple, and easy to use as those companies have made them.

So if I build an app that shows pictures (picasa) on an Event page (eventbrite), and you can see who joined the event (facebook events), and send them emails (google apps api), and have them fill out monthly surveys (wufoo), and watch a video when they're done (youtube), all integrated into a custom, easy to use website, and I can do that without ever creating a local database, is that a good thing?

I ask because there's two things missing from the puzzle that keep forcing me to create that local database:

• Post API
• RESTful/Pretty Url API

While there's plenty of Blogging systems and APIs for them, there is no one place where you can just write content and have it part of some massive thing. For every app, I have to use code for creating pretty/restful urls, and that saves posts. But it seems like that should be a service!

Question is, is that what the website is? ...That place to integrate the worlds services for my specific cause... and, sigh, to store posts that only my site has access to. Will everyone always need "their own blog"? Why not just have a profile and write lots of content on an established platform like StackOverflow or Facebook?

... That way I can write apps entirely without a database and know that I'm doing it right.

Note: Of course at some point you'd need a database, if you were doing something unique or new. But for the case where you're just rewiring information or creating things like videos, events, and products, is it really necessary anymore??

Answer 1

I think you have pretty much answered your own question in the question:

• If you can find an third-party web-based service that meets your website's requirements for persistence, then your website does not need a local database.

• If you cannot, then it does.

But the "requirements" issues are more than just technical ones. Assuming that you identify a remote database / persistence service, there are many reasons why it might not be suitable:

• It might not provide the detailed functionality your customer site requires; e.g. ability to do certain types of queries, scalability, etc.
• It might be too expensive to use given your customer's site projected hit rate.
• It might be too risky in terms of:
  • end-user privacy,
  • service availability / reliability,
  • long-term viability of the service and
  • long term stability of the service's APIs.

This is not to say that these concerns cannot be addressed. But at least the "risk" issues need to be discussed with your customers, since they will ultimately need to deal with them if things go bad.

However, when all is said and done, the one big advantage of a local database over a remote one is that you and your customer have complete control over a local database.