Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What data type should I use to store an image with MongoDB?

I have an image with base64, e.g.

data:image/jpeg;base64,/9j/4AAQSkZJRgABAgAAAQABAAD/7QCcUGhvdG9zaG9w....

How to save in the database? What should be the type of the field in schema? Buffer?

like image 274
Ani Alaverdyan Avatar asked Jul 02 '17 09:07

Ani Alaverdyan


People also ask

What is the best way to store images in MongoDB?

You can store the images in the MongoDB database by creating a schema with Mongoose. The schema is defined by creating a file, model. js . The data type Buffer is used for storing the images in the database form of arrays.

Can I use MongoDB to store images?

So for storing an image in MongoDB, we need to create a schema with mongoose. For that create the file `model. js` file and define the schema. The important point here is that our data type for the image is a Buffer which allows us to store our image as data in the form of arrays.

How does MongoDB store images as binary data?

There are three ways to store the images in MongoDB: GridFS, Within Document, and Referencing an external URL. If an image or any binary file which is of size less than 16MB can be stored in MongoDB using Binarydata ( bindata ) type.

What kind of data is good for MongoDB?

SQL. NoSQL databases like MongoDB are a good choice when your data is document-centric and doesn't fit well into the schema of a relational database, when you need to accommodate massive scale, when you are rapidly prototyping, and a few other use cases.


1 Answers

The short answer is store as "Binary", for which in a mongoose schema you can use Buffer to do this.

The longer form is to demonstrate a round trip of conversion starting with the original binary and back again. Base64 encoding/decoding is not a necessary step in most real world cases, and is just there for demonstration:

  • Read an image (or any binary data) from file
  • Base64 encode that data (just to show it can be done) - optional
  • Turn back into Binary data from Base64 ( just to show it can be done ) - optional
  • Store Binary data in the database
  • Read Binary data from the database
  • Output binary data to a new file

So the Schema Part is simple, just use Buffer:

var albumSchema = new Schema({
  name: String,
  image: Buffer
})

Then all we are going to do is follow the process and put the binary data into the property and read it back out again.

Note though that if you are coming directly from a string with a MIME type on it like :

  data:image/png;base64,long-String

Just use a JavaScript .split() and take the second array index for the base64 string itself:

var string = "data:image/png;base64,long-String"
var bindata = new Buffer(string.split(",")[1],"base64");

Here's a listing with a complete demo in and out:

const async = require('async'),
      mongoose = require('mongoose'),
      Schema = mongoose.Schema,
      fs = require('fs');

mongoose.Promise = global.Promise;
mongoose.set('debug',true);
mongoose.connect('mongodb://localhost/test');

var albumSchema = new Schema({
  name: String,
  image: Buffer
})

const Album = mongoose.model('Albumn', albumSchema);


async.series(
  [
    (callback) =>
      async.each(mongoose.models,(model,callback) =>
        model.remove({},callback),callback),

    (callback) =>
      async.waterfall(
        [
          (callback) => fs.readFile('./burger.png', callback),

          (data,callback) => {
            // Convert to Base64 and print out a bit to show it's a string
            let base64 = data.toString('base64');
            console.log(base64.substr(0,200));

            // Feed out string to a buffer and then put it in the database
            let burger = new Buffer(base64, 'base64');
            Album.create({
              "title": "burger",
              "image": burger
            },callback)
          },

          // Get from the database
          (album,callback) => Album.findOne().exec(callback),

          // Show the data record and write out to a new file.
          (album, callback) => {
            console.log(album);
            fs.writeFile('./output.png', album.image, callback)
          }

        ],
        callback
      )


  ],
  (err) => {
    if (err) throw err;
    mongoose.disconnect();
  }
)

NOTE The example was originally given with asyncJS and older mongoose API, which notably has different connection options as shown in more modern and current API examples. Refer to these instead for testing on current NodeJS LTS releases:

Or with a a bit more modern syntax and API usage for comparison:

const fs = require('mz/fs');
const { Schema } = mongoose = require('mongoose');

const uri = 'mongodb://localhost:27017/test';
const opts = { useNewUrlParser: true };

mongoose.Promise = global.Promise;
mongoose.set('debug', true);
mongoose.set('useFindAndModify', false);
mongoose.set('useCreateIndex', true);

const albumSchema = new Schema({
  name: String,
  image: Buffer
});

const Album = mongoose.model('Album', albumSchema);

(async function() {

  try {

    const conn = await mongoose.connect(uri, opts);

    await Promise.all(
      Object.entries(conn.models).map(([k, m]) => m.deleteMany())
    )

    let data = await fs.readFile('./burger.png');

    // Convert to Base64 and print out a bit to show it's a string
    let base64 = data.toString('base64');
    console.log(base64.substr(0,200));

    // Feed out string to a buffer and then put it in the database
    let burger = new Buffer(base64, 'base64');
    await Album.create({ "title": "burger", "image": burger });

    // Get from the database
    // - for demo, we could have just used the return from the create() instead
    let album =  Album.findOne();

    // Show the data record and write out to a new file.
    console.log(album);
    await fs.writeFile('./output.png', album.image)


  } catch(e) {
    console.error(e);
  } finally {
    mongoose.disconnect()
  }

})()

And even with "plain promises" where that is either preferred or you are still using a NodeJS without async/await support. But you really should not be, considering v6.x reaches end of life in April 2019:

// comments stripped - refer above
const fs = require('mz/fs');
const { Schema } = mongoose = require('mongoose');

const uri = 'mongodb://localhost:27017/test';
const opts = { useNewUrlParser: true };

mongoose.Promise = global.Promise;
mongoose.set('debug', true);
mongoose.set('useFindAndModify', false);
mongoose.set('useCreateIndex', true);

const albumSchema = new Schema({
  name: String,
  image: Buffer
});

mongoose.connect(uri, opts)
  .then(conn =>
    Promise.all(
      Object.entries(conn.models).map(([k, m]) => m.deleteMany())
    )
  )
  .then(() => fs.readFile('./burger.png'))
  .then(data => {
    let base64 = data.toString('base64');
    console.log(base64.substr(0,200));
    let burger = new Buffer(base64, 'base64');
    return Album.create({ "title": "burger", "image": burger });
  })
  .then(() => Album.findOne() )
  .then(album => {
    console.log(album);
    return fs.writeFile('./output.png', album.image)
  })
  .catch(console.error)
  .then(() => mongoose.disconnect());

And here's a burger.png to play with:

enter image description here

Also kudos to How to reduce image size on Stack Overflow which allows the sample image here to not appear as "huge" as it originally was, and yet still download at full size.

like image 160
Neil Lunn Avatar answered Oct 02 '22 17:10

Neil Lunn