MongoDB as static files provider?

Question

It's MongoDB a good candidate to serve static files (files, video) as cdn. I searching a solid way to store big amounts of data (>+To) remplacing S3 and managing cache features. Experiences.

Thanks.

Answer 1

I suppose that the answer to this question is no, because when you are using a CDN to store static files you make your server free from static data requests. But when you are choosing between a file system and mongodb Ii suppose that mongodb will be faster, beacause if there is enough ram on the server, mongo will load all data to the memory.

Also I've found some links that can help you make your choice: www.markus-gattol.name, groups.google.com

From the documentation:

When to use GridFS

Lots of files. GridFS tends to handle large numbers (many thousands) of files better than many file systems.

User uploaded files. When users upload files you tend to have a lot of files, and want them replicated and backed up. GridFS is a perfect place to store these as then you can manage them the same way you manage your data. You can also query by user, upload date, etc... directly in the file store, without a layer of indirection

Files that often change. If you have certain files that change a lot - it makes sense to store them in GridFS so you can modify them in one place and all clients will get the updates. Also can be better than storing in source tree so you don't have to deploy app to update files.

When not to use GridFS

Few small static files. If you just have a few small files for a website (js,css,images) its probably easier just to use the file system.

Note that if you need to update a binary object atomically, and the object is under the document size limit for your version of MongoDB (16MB for 1.8), then you might consider storing the object manually within a single document. This can be accomplished using the BSON bindata type. Check your driver's docs for details on using this type.

Answer 2

Generally the answer would be yes, you can use mongodb to do this but mongodb does not have an http interface to serve files like apache/random-webserver does. If you need to add additional feature, like application authentication for those files then it might make more sense, as an example.

You can create an infrastructure around mongodb where you replicate changes and write a http front-end to serve files (like nginx-gridfs or any of the other gridfs http components). But you would need to build/integrate these, test and deploy it all. Using a file-system with a standard web-server to provide the files is very well tested and documented; many systems use rsync to efficiently replicate files to many nodes.

There are so many things that a cdn does that you would have to re-implement it might not make sense to tackle all of that yourself, but that is really another question.