Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Storing image files in Mongo database, is it a good idea?

Tags:

When working with mysql, it is a bad idea to store images as BLOB in the database, as it makes the database quite large which is harmful for normal usage of the database. Then, it is better to save image files on disk and save link to them within the database.

However, I think this is different for MongoDB, as increasing the database file size has a negligible influence on performance (this is the reason that MongoDB can successfully handle billions of records).

Do you think it is better to save image files on MongoDB (as GridFS) to reduce number of files stored on the server; or still it is better to keep the database as small as possible?

like image 243
Googlebot Avatar asked Oct 09 '11 13:10

Googlebot


People also ask

Is storing images in a database a good idea?

Storing images in a database table is not recommended. There are too many disadvantages to this approach. Storing the image data in the table requires the database server to process and traffic huge amounts of data that could be better spent on processing it is best suited to.

Can we store images in MongoDB database?

So for storing an image in MongoDB, we need to create a schema with mongoose. For that create the file `model. js` file and define the schema. The important point here is that our data type for the image is a Buffer which allows us to store our image as data in the form of arrays.

Is it better to store images in database or filesystem?

Generally databases are best for data and the file system is best for files. It depends what you're planning to do with the image though. If you're storing images for a web page then it's best to store them as a file on the server. The web server will very quickly find an image file and send it to a visitor.

Should I store files in Mongo?

Storing the image as document in mongodb would be a bad idea, as the resources which could have been used to send a large amount of informational data would be used for sending files.


2 Answers

The problem isn't so much that the database gets big, databases can handle that (although MongoDB isn't as good as many other in that respect). The problem is that to send the data to the client it first has to be moved into RAM by the database, then copied over to the application's memory, then handed off to the kernel to be sent through the socket. It's wasting lots of RAM and CPU cycles. The reason it's better to have large files in the filesystem is that it's easier to get around copying it, you can ask the kernel to stream the file from disk to the socket directly.

The downside of storing large files in the filesystem is that it's much harder to distribute. Using a database, and something like Mongo's GridFS makes it possible to scale out. You just have to make sure you don't copy the whole file into the application's memory at once, but a chunk at a time. Most web app frameworks have some support for sending chunked HTTP responses nowadays.

like image 160
Theo Avatar answered Oct 18 '22 23:10

Theo


The answer is yes. Back in the old cave-man days, servers had mutable file systems you could change. This was great till we tried to scale things.

Cave-people nowadays build apps with immutable deployments. Heroku and Dokku are examples of this. Because the web app server has no state, they can be created, upgraded, scaled, and destroyed easily.

Since we still have files, we need to put them somewhere. There are several solutions: nfs, our database, someone elses database.

  • nfs is a 'network file system' which let's you do file i/o on network resources. If you're dealing with the network anyways, IMHO it doesn't add much value unless it's what you know already.

  • Our database - For MongoDB there are two options: (file > 16mb) ? GridFS : BinData

  • Someone elses database - Some are basic like Amazon S3 and some offer extra services like Cloudinary or Dropbox.

If you're on an big-budget enterprise team and someone spends 40 hrs a week taking care of servers then sure - use the file system. If you're building web apps that scale, putting files in the DB makes sense.

If you're concerned about performance:

1) Using a proxy (e.g. nginx) or a CDN to host your content for clients. Your server should just be serving cache misses.

2) Use streaming IO Nodeschool has a cool tutorial for Node.js.

like image 40
Michael Cole Avatar answered Oct 18 '22 22:10

Michael Cole