FindAndUpdate How to check if document was really updated

Question

Imagine the following model:

var Office = 
{
    id: 1,
    name: "My Office",
    branches: 
    [
      {
        adddress: "Some street, that avenue",
        isPrincipal: true,
      },
      {
        adddress: "Another address",
        isPrincipal: false,
      },      
    ]
}

I'd like to remove a branch, but we can't let the user remove the principal branch from an office. So here's my function:

remove: function(body)
{
  return new Promise(function(resolve, reject)
  {
    return Office.findByIdAndUpdate(1, { $pull: {'branches': {_id: body.branch.id}}}, { new: true })
    .then(function(updatedOffice){
      resolve(updatedOffice)
    })
    .catch(function(error){
      reject(error);
    });

  })
}    

I have some doubts here:

1. As you can see I haven't included the another WHERE on the isPrincipal property, this is because I don't know how can I determine whether the office object got actually changed. Because the object will alway be retrieved but... How can I know this for sure?
2. Is FindByIdAndUpdate the best approach considering we can't let a user delete the principal branch, and if he's trying to do so, we have to show a warning.

Answer 1

The only real reliable way to see if an update was applied for something like a $pull is to basically check the returned document and see if the data you intended to $pull is still in there or not.

That's for any of the "findAndUpdate" variety of actions, and there is a valid reason for that as well as it also being the case that a plain .update() will actually "reliably" tell you if the modification was in fact made.

To walk through the cases:

Check the Returned Content

This basically involves looking at the array in the returned document in order to see if what we asked to remove is actually there:

var pullId = "5961de06ea264532c684611a";

Office.findByIdAndUpdate(1,
  { "$pull": { "branches": { "_id": pullId } } },
  { "new": true }
).then(office => {
  // Check if the supplied value is still in the array
  console.log(
    "Still there?: %s",
    (office.branches.find( b => b._id.toHexString() === pullId))
      ? true : false
  );
}).catch(err => console.error(err))

We use .toHexString() in order to compare the actual value from an ObjectId since JavaScript just does not do "equality" with "Objects". You would check on both "left" and "right" if supplying something that was already "cast" to an ObjectId value, but in this case we know the other input is a "string".

Just use .update(), "It's reliable"

The other case here to consider brings into question if you "really need" the returned modified data anyway. Because the .update() method, will reliably return a result telling you if anything was actually modified:

Office.update(
  { "_id": 1 },
  { "$pull": { "branches": { "_id": pullId } } },
).then(result => {
  log(result);
}).catch(err => console.error(err))

Where result here will look like:

{
  "n": 1,
  "nModified": 1,        // <--- This always tells the truth, and cannot lie!
  "opTime": {
    "ts": "6440673063762657282",
    "t": 4
  },
  "electionId": "7fffffff0000000000000004",
  "ok": 1
}

And in which the nModified is a "true" indicator of whether something "actually updated". Therefore if it's 1 then the $pull actually had an effect, but when 0 nothing was actually removed from the array and nothing was modified.

This is because the method actually uses the updated API, which does have reliable results indicating actual modifications. The same would apply to something like a $set which did not actually change the value because the the value supplied was equal to what already existed in the document.

findAndModify Lies!

The other case here you might think of when looking closely at the documentation is to actually inspect the "raw result" and see if the document was modified or not. There is actually an indicator in the specification for this.

The problem is ( as well as requiring more work with Promises ) that the result is not actually truthful:

var bogusId = "5961de06ea264532c684611a"; // We know this is not there!

Promise((resolve,reject) => {
  Office.findByIdAndUpdate(1,
    { "$pull": { "branches": { "_id": bogusId } } },
    { "new": true, "passRawResult" },
    (err,result,raw) => {        // We cannot pass multiple results to a Promise
      if (err) reject(err);
      resolve({ result, raw });   // So we wrap it!
    }
  )
})
.then(response => log(response.raw))
.catch(err => console.error(err));

The problem here is that even when we "know" this should not modify, the response says otherwise:

{
  "lastErrorObject": {
    "updatedExisting": true,
    "n": 1                     // <--- LIES! IT'S ALL LIES!!!
  },
  "value": {
    "_id": 1,
    "name": "My Office",
    "branches": [
      {
        "address": "Third address",
        "isPrincipal": false,
        "_id": "5961de06ea264532c6846118"
      }
    ],
    "__v": 0
  },
  "ok": 1,
  "_kareemIgnore": true
}

So even after all that work to get the "third" argument out of the callback response, we still did not get told the correct information about the update.

---

Concluding

So if you want to "reliably" do this with a single request ( and you cannot reliably do that with multiple requests since the document could change in between! ) then your two options are:

1. Check the returned document to see if the data you wanted to remove is still there.

2. Forget returning a document and trust that .update() always tells you the "truth" ;)

Which one of these you use depends on the application usage pattern, but those are the two different ways of returning an "reliable" result.

---

Bit of a Listing

So just to be sure, here's a listing that goes through all the examples and demonstrates what they actually return:

const async = require('async'),
      mongoose = require('mongoose'),
      Schema = mongoose.Schema;

mongoose.Promise = global.Promise;
mongoose.set('debug',true);

mongoose.connect('mongodb://localhost/test');

const branchesSchema = new Schema({
  address: String,
  isPrincipal: Boolean
});

const officeSchema = new Schema({
  _id: Number,
  name: String,
  branches: [branchesSchema]
},{ _id: false });

const Office = mongoose.model('Office', officeSchema);

function log(data) {
  console.log(JSON.stringify(data,undefined,2))
}

const testId = "5961a56d3ffd3d5e19c61610";

async.series(
  [
    // Clean data
    (callback) =>
      async.each(mongoose.models,(model,callback) =>
        model.remove({},callback),callback),

    // Insert some data and pull
    (callback) =>
      async.waterfall(
        [
          // Create and demonstrate
          (callback) =>
            Office.create({
              _id: 1,
              name: "My Office",
              branches: [
                {
                  address: "Some street, that avenue",
                  isPrincipal: true
                },
                {
                  address: "Another address",
                  isPrincipal: false
                },
                {
                  address: "Third address",
                  isPrincipal: false
                }
              ]
            },callback),

          // Demo Alternates
          (office,callback) =>
            async.mapSeries(
              [true,false].map((t,i) => ({ t, branch: office.branches[i] })),
              (test,callback) =>
                (test.t)
                  ? Office.findByIdAndUpdate(office._id,
                      { "$pull": { "branches": { "_id": test.branch._id } } },
                      { "new": true , "passRawResult": true },
                      (err,result,raw) => {
                        if (err) callback(err);
                        log(result);
                        log(raw);
                        callback();
                      })
                  : Office.findByIdAndUpdate(office._id,
                      { "$pull": { "branches": { "_id": test.branch._id } } },
                      { "new": true } // false here
                    ).then(result => {
                      log(result);
                      console.log(
                        "Present %s",
                        (result.branches.find( b =>
                          b._id.toHexString() === test.branch._id.toHexString() ))
                          ? true : false
                      );
                      callback();
                    }).catch(err => callback(err)),
              callback
            )
        ],
        callback
      ),

    // Find and demonstate fails
    (callback) =>
      async.waterfall(
        [
          (callback) => Office.findOne({},callback),

          (office,callback) =>
            async.eachSeries([true,false],(item,callback) =>
              (item)
                ? Office.findByIdAndUpdate(office._id,
                    { "$pull": { "branches": { "_id": testId } } },
                    { "new": true, "passRawResult": true },
                    (err,result,raw) => {
                      if (err) callback(err);
                      log(result);
                      log(raw);
                      callback();
                    }
                  )
                : Office.findByIdAndUpdate(office._id,
                    { "$pull": { "branches": { "_id": testId } } },
                    { "new": true }
                  ).then(result => {
                    console.log(result);
                    console.log(
                      "Present %s",
                      (result.branches.find( b =>
                        b._id.toHexString() === office.branches[0]._id.toHexString()))
                        ? true : false
                    );
                    callback();
                  })
                  .catch(err => callback(err)),
              callback)

        ],
        callback
      ),

    // Demonstrate update() modified shows 0
    (callback) =>
      Office.update(
        {},
        { "$pull": { "branches": { "_id": testId } } }
      ).then(result => {
        log(result);
        callback();
      })
      .catch(err => callback(err)),

    // Demonstrate wrapped promise
    (callback) =>
      Office.findOne()
        .then(office => {
          return new Promise((resolve,reject) => {
            Office.findByIdAndUpdate(office._id,
              { "$pull": { "branches": { "_id": testId } } },
              { "new": true, "passRawResult": true },
              (err,result,raw) => {
                if (err) reject(err);
                resolve(raw)
              }
            );
          })
        })
        .then(office => {
          log(office);
          callback();
        })
        .catch(err => callback(err))

  ],
  (err) => {
    if (err) throw err;
    mongoose.disconnect();
  }
);

And the output it produces:

Mongoose: offices.remove({}, {})
Mongoose: offices.insert({ _id: 1, name: 'My Office', branches: [ { address: 'Some street, that avenue', isPrincipal: true, _id: ObjectId("5961e5211a73e8331b44d74b") }, { address: 'Another address', isPrincipal: false, _id: ObjectId("5961e5211a73e8331b44d74a") }, { address: 'Third address', isPrincipal: false, _id: ObjectId("5961e5211a73e8331b44d749") } ], __v: 0 })
Mongoose: offices.findAndModify({ _id: 1 }, [], { '$pull': { branches: { _id: ObjectId("5961e5211a73e8331b44d74b") } } }, { new: true, passRawResult: true, upsert: false, remove: false, fields: {} })
{
  "_id": 1,
  "name": "My Office",
  "__v": 0,
  "branches": [
    {
      "address": "Another address",
      "isPrincipal": false,
      "_id": "5961e5211a73e8331b44d74a"
    },
    {
      "address": "Third address",
      "isPrincipal": false,
      "_id": "5961e5211a73e8331b44d749"
    }
  ]
}
{
  "lastErrorObject": {
    "updatedExisting": true,
    "n": 1
  },
  "value": {
    "_id": 1,
    "name": "My Office",
    "branches": [
      {
        "address": "Another address",
        "isPrincipal": false,
        "_id": "5961e5211a73e8331b44d74a"
      },
      {
        "address": "Third address",
        "isPrincipal": false,
        "_id": "5961e5211a73e8331b44d749"
      }
    ],
    "__v": 0
  },
  "ok": 1,
  "_kareemIgnore": true
}
Mongoose: offices.findAndModify({ _id: 1 }, [], { '$pull': { branches: { _id: ObjectId("5961e5211a73e8331b44d74a") } } }, { new: true, upsert: false, remove: false, fields: {} })
{
  "_id": 1,
  "name": "My Office",
  "__v": 0,
  "branches": [
    {
      "address": "Third address",
      "isPrincipal": false,
      "_id": "5961e5211a73e8331b44d749"
    }
  ]
}
Present false
Mongoose: offices.findOne({}, { fields: {} })
Mongoose: offices.findAndModify({ _id: 1 }, [], { '$pull': { branches: { _id: ObjectId("5961a56d3ffd3d5e19c61610") } } }, { new: true, passRawResult: true, upsert: false, remove: false, fields: {} })
{
  "_id": 1,
  "name": "My Office",
  "__v": 0,
  "branches": [
    {
      "address": "Third address",
      "isPrincipal": false,
      "_id": "5961e5211a73e8331b44d749"
    }
  ]
}
{
  "lastErrorObject": {
    "updatedExisting": true,
    "n": 1
  },
  "value": {
    "_id": 1,
    "name": "My Office",
    "branches": [
      {
        "address": "Third address",
        "isPrincipal": false,
        "_id": "5961e5211a73e8331b44d749"
      }
    ],
    "__v": 0
  },
  "ok": 1,
  "_kareemIgnore": true
}
Mongoose: offices.findAndModify({ _id: 1 }, [], { '$pull': { branches: { _id: ObjectId("5961a56d3ffd3d5e19c61610") } } }, { new: true, upsert: false, remove: false, fields: {} })
{ _id: 1,
  name: 'My Office',
  __v: 0,
  branches:
   [ { address: 'Third address',
       isPrincipal: false,
       _id: 5961e5211a73e8331b44d749 } ] }
Present true
Mongoose: offices.update({}, { '$pull': { branches: { _id: ObjectId("5961a56d3ffd3d5e19c61610") } } }, {})
{
  "n": 1,
  "nModified": 0,
  "opTime": {
    "ts": "6440680872013201413",
    "t": 4
  },
  "electionId": "7fffffff0000000000000004",
  "ok": 1
}
Mongoose: offices.findOne({}, { fields: {} })
Mongoose: offices.findAndModify({ _id: 1 }, [], { '$pull': { branches: { _id: ObjectId("5961a56d3ffd3d5e19c61610") } } }, { new: true, passRawResult: true, upsert: false, remove: false, fields: {} })
{
  "lastErrorObject": {
    "updatedExisting": true,
    "n": 1
  },
  "value": {
    "_id": 1,
    "name": "My Office",
    "branches": [
      {
        "address": "Third address",
        "isPrincipal": false,
        "_id": "5961e5211a73e8331b44d749"
      }
    ],
    "__v": 0
  },
  "ok": 1,
  "_kareemIgnore": true
}