The value of "offset" is out of range. It must be >= 0 && <= 17825792

Question

    You have triggered an unhandledRejection, you may have forgotten to catch a Promise rejection:
 RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 && <= 17825792. Received 17825796
     at Buffer.write (buffer.js:1019:5)
     at serializeObjectId (/home/ubuntu/workspace/assurance-api/node_modules/bson/lib/bson/parser/serializer.js:274:14)
     at serializeInto (/home/ubuntu/workspace/assurance-api/node_modules/bson/lib/bson/parser/serializer.js:935:17)
     at serializeObject (/home/ubuntu/workspace/assurance-api/node_modules/bson/lib/bson/parser/serializer.js:347:18)
     at serializeInto (/home/ubuntu/workspace/assurance-api/node_modules/bson/lib/bson/parser/serializer.js:727:17)
     at serializeObject (/home/ubuntu/workspace/assurance-api/node_modules/bson/lib/bson/parser/serializer.js:347:18)
     at serializeInto (/home/ubuntu/workspace/assurance-api/node_modules/bson/lib/bson/parser/serializer.js:941:17)
     at BSON.serialize (/home/ubuntu/workspace/assurance-api/node_modules/bson/lib/bson/bson.js:64:28)
     at Msg.serializeBson (/home/ubuntu/workspace/assurance-api/node_modules/mongodb/lib/core/connection/msg.js:126:22)
     at Msg.makeDocumentSegment (/home/ubuntu/workspace/assurance-api/node_modules/mongodb/lib/core/connection/msg.js:118:33)
     at Msg.toBin (/home/ubuntu/workspace/assurance-api/node_modules/mongodb/lib/core/connection/msg.js:104:25)
     at MessageStream.writeCommand (/home/ubuntu/workspace/assurance-api/node_modules/mongodb/lib/cmap/message_stream.js:55:28)
     at Connection.write (/home/ubuntu/workspace/assurance-api/node_modules/mongodb/lib/cmap/connection.js:361:26)
     at _command (/home/ubuntu/workspace/assurance-api/node_modules/mongodb/lib/core/wireprotocol/command.js:128:10)
     at command (/home/ubuntu/workspace/assurance-api/node_modules/mongodb/lib/core/wireprotocol/command.js:28:5)
     at writeCommand (/home/ubuntu/workspace/assurance-api/node_modules/mongodb/lib/core/wireprotocol/write_command.js:47:3)

i've been sending an axios.post with a payload of data and files to my express back-end , i catch the files in form of base64 and pop the front letters which contain "data/png base64;" when i try to create a new instance with the base64 data i get the following error when trying to save the document to mongodb, using mongoose, i looked this error up in vain, is the files too large?

Answer 1

For those having the same issue. I have some extra information, after some testing I concluded the following things:

When you make a MongoDB insert request that changes an object so it becomes larger than the maximum allowed size (16mb) you will get the following error:

MongoError: BSONObj size: 20971588 (0x1400044) is invalid. Size must be between 0 and 16793600(16MB) First element: _id: ObjectId('60a277f50a5f8b012a761672')
    at Function.create ...
    {
  driver: true,
  index: 0,
  code: 10334
}    

When you do a MongoDB query that is too large (like in the question). You will be presented with the following error:

RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 && <= 17825792. Received 17825794
    at validateOffset (buffer.js:104:3)
    at Buffer.write (buffer.js:1055:5)
    ... {
  code: 'ERR_OUT_OF_RANGE'
}

This is important because if you do for example an upsert that inserts or updates 10mb of data your query surpasses the max ~17mb of a query. (not entirely sure this was intended by the developers)

Lastly when you do an insert that does not exceed the maximum query size, but does exceed the maximum BSON document size you will be presented with this error:

MongoError: object to insert too large. size in bytes: 16777272, max size: 16777216
    at Function.create ...
     {
  driver: true,
  index: 0,
  code: 2
}

This was tested with:

• Node v14.16.0
• MongoDB node module v3.6.4

I hope this helps anyone who landed on this question.