Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to limit maximum reference of the parental node in mongodb

I have to create a tree structure with mongodb and of the four methods suggested by the official doc (parental reference, child reference, array of ancestors, materialized paths, nested sets) I have decided to use array of ancestors because the tier of the tree will be quite deep potentially thousands of levels or more. So I thought the penalty in storage could be compensated by faster speed due to less query being required when looking for all the parents of a node.

The problem is that the tree has to be strictly binary. So for example node 1 will only have node 2 and node 3 as its children and no more. Unfortunately only way I can think of doing this right now is to find all the documents that reference the specific parent that I would like to reference in the next document and if the count does not exceed 2, I insert the document referencing the target parent.

Obviously the problem is the multithreaded nature of mongodb and even if it wasn't, the request to look up the count that references the target parent could come in between the scanning and insertion of the previous document which would cause a third insertion if there were one document already referencing the target parent.

Is there a way I can make sure that only two nodes can reference a parent node in multithreaded environment of both mongodb and the server?

like image 874
forJ Avatar asked Feb 20 '18 01:02

forJ


People also ask

How do I limit the number of records in MongoDB?

MongoDB – limit() Method In MongoDB, the limit() method limits the number of records or documents that you want. It basically defines the max limit of records/documents that you want. Or in other words, this method uses on cursor to specify the maximum number of documents/ records the cursor will return.

What is the limitation of a document in MongoDB?

The maximum BSON document size is 16 megabytes. The maximum document size helps ensure that a single document cannot use excessive amount of RAM or, during transmission, excessive amount of bandwidth. To store documents larger than the maximum size, MongoDB provides the GridFS API.

What is MongoDB Index Key limit?

A collection cannot have more than 64 indexes. The length of the index name cannot be longer than 125 characters. A compound index can have maximum 31 fields indexed.

How do you define maximum number of documents in collection?

Unless you create a capped collection, there is no limit to the number of documents in a collection. There is a 16MB limit to the size of a document (use gridfs in this situation) and limits in the storage engine for the size of the database and data.


1 Answers

If you are using mongo version 3.6, you can enable schema validation using $jsonSchema at the server side

each document we update/insert will be validated against the validation schema, it the validation fails an error will be thrown and no modification will be made on the document

sample node collection schema

{
   _id : string,
   parent : [string, null],
   children : string[2]
}

validation schema

db.createCollection("node", {
   validator: {
      $jsonSchema: {
         bsonType: "object",
         required: [ "_id" ],
         properties: {
            parent: {
               bsonType: ["string", "null"],
               description: "must be a string"
            },
            children: {
               bsonType: ["array"],
               items : { bsonType: ["string"] },
               minItems: 0,
               maxItems: 2,
               description: "must be a array of string and max is 2"
            }
         }
      }
   }
});

inserts [with valid documents]

> db.node.insert( { _id: "Books", children: [ "Programming" ], parent: null } )
WriteResult({ "nInserted" : 1 })
> db.node.insert( { _id: "Programming", children: [ "Databases", "Languages" ], parent: "Books" } )
WriteResult({ "nInserted" : 1 })
> db.node.insert( { _id: "Languages", children: [ ], parent: "Programming" } )
WriteResult({ "nInserted" : 1 })
> db.node.insert( { _id: "Databases", children: [ "MongoDB", "dbm" ], parent: "Programming" } )
WriteResult({ "nInserted" : 1 })
> db.node.insert( { _id: "MongoDB", children: [ ], parent: "Databases" } )
WriteResult({ "nInserted" : 1 })
> db.node.insert( { _id: "dbm", children: [ ], parent: "Databases" } )
WriteResult({ "nInserted" : 1 })
> 

find

> db.node.find()
{ "_id" : "Books", "children" : [ "Programming" ], "parent" : null }
{ "_id" : "Programming", "children" : [ "Databases", "Languages" ], "parent" : "Books" }
{ "_id" : "Languages", "children" : [ ], "parent" : "Programming" }
{ "_id" : "Databases", "children" : [ "MongoDB", "dbm" ], "parent" : "Programming" }
{ "_id" : "MongoDB", "children" : [ ], "parent" : "Databases" }
{ "_id" : "dbm", "children" : [ ], "parent" : "Databases" }

insert with invalid document [children size > 2]

> db.node.insert({_id : "1", children : ["c1", "c2", "c3"], parent : "p1"})
WriteResult({
    "nInserted" : 0,
    "writeError" : {
        "code" : 121,
        "errmsg" : "Document failed validation"
    }
})
> 

insert failed with a validation error

update - trying to add 3rd child for _id Databases, failed with validation error

> db.node.updateOne( { _id: "Databases"}, {$push : {children: [ "Oracle" ]}} )
2018-02-25T21:00:08.087+0530 E QUERY    [thread1] WriteError: Document failed validation :
WriteError({
    "index" : 0,
    "code" : 121,
    "errmsg" : "Document failed validation",
    "op" : {
        "q" : {
            "_id" : "Databases"
        },
        "u" : {
            "$push" : {
                "children" : [
                    "Oracle"
                ]
            }
        },
        "multi" : false,
        "upsert" : false
    }
})
WriteError@src/mongo/shell/bulk_api.js:466:48
Bulk/mergeBatchResults@src/mongo/shell/bulk_api.js:846:49
Bulk/executeBatch@src/mongo/shell/bulk_api.js:910:13
Bulk/this.execute@src/mongo/shell/bulk_api.js:1154:21
DBCollection.prototype.updateOne@src/mongo/shell/crud_api.js:572:17
@(shell):1:1
> 

please refer schema-validation & jsonSchema for more options, adding validation to existing collection and handling validation failures

like image 123
Saravana Avatar answered Oct 23 '22 04:10

Saravana