Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

mongodb - capped collections with nodejs

I'm trying to set up and update some capped collections in MongoDB using Node.js (using the native MongoDB driver). My goal is to, upon running app.js, insert documents into a capped collection, and also to update existing documents in a capped collection. Both of these are running on setInterval(), so every few seconds.

My questions:

  1. I want to create a collection if the collection does not already exist, but if it does I want to insert a document into it instead. What is the correct way to check this?
  2. With capped collections, should I even be explicitly creating them first before inserting anything into them? Normally I believe you can just insert things into a collection without explicitly creating them first, but in this case I need to ensure they are capped. Once the capped collection exists I know how to insert new documents into it, the problem is I need some way to handle the app being used for the first time (on a new server) where the collection doesn't already exist, and I want to do this creation using node and not having to jump into the mongo cli.
  3. The trick here is that the collection needs to be capped, so I can do something like: db.createCollection("collectionName", { capped : true, size : 100000, max : 5000 } ). That will create the capped collection for me, but every time I call it it will call createCollection() instead of updating or inserting - if I call createCollection(), once the collection already exists, will it completely overwrite the existing collection?
  4. An alternative is to turn a collection into a capped one with: db.runCommand({"convertToCapped": "collectionName", size: 100000, max : 5000 });. The problem with this is that node doesn't see runCommand() as a valid function and it errors. Is there something else that I'm meant to be calling to get this to work? It works in the mongo cli but not within node
  5. What type of query do you use to find the first document in a collection? Again, within the mongo cli I can use db.collections.find() with some query, but within node it states that find() is not a valid function
  6. How would I use collection.update() to add some new fields to an existing document? Lets say the document is some simple object like {key1: "value", key2: "value"} but I have an object that contains {key3: "value"}. Key 3 does not exist in the current document, how would I add that to what currently exists? This is somewhat related to #4 above in that I'm not sure what to pass in as the query parameter given that find() doesn't seem to play well with node.
like image 731
Simon Avatar asked Oct 19 '22 16:10

Simon


1 Answers

Regarding your questions 1 - 4 about capped collections and creating them automatically, there are several ways to do this. On the one hand, you could run a script to initialise your database so that it has the capped collections available to your client when you run it for the first time. On the other hand, you could have a check to see if there are any documents in the given collection before inserting a document. If there are, you just insert your document and if there aren't, you create the capped collection and then insert the document as a callback to that function. It would work something like this:

var host = "localhost",
port = 27017,
dbName = "so";

var MongoClient = require('mongodb').MongoClient, Server = require('mongodb').Server;
var mongoclient = new MongoClient(new Server(host, port));
var db = mongoclient.db(dbName);

db.open(function(err, db) {

    if(err) throw err;

    // Capped collection.
    var capped = db.collection('capped');

    // Document to be inserted.
    var document =  { "foo": 1, "bar": 1 }

    capped.find().count(function(err, count) {

        if(err) throw err;

        if (count === 0) {          
            console.log("Creating collection...");
            db.createCollection("capped",
                                { "capped": true,
                                  "size": 100000,
                                  "max": 5000 },
                                function(err, collection) {
                if(err) throw err;              

                // Insert a document here.
                console.log("Inserting document...");
                collection.insert(document, function(err, result) {
                    if (err) throw err;                 
                });
            });

        } else {

            // Insert your document here without creating collection.
            console.log("Inserting document without creating collection...");
            capped.insert(document, function(err, result) {
                if (err) throw err;             
            });
        }

    });
});

Regarding question 5, you can use findOne() to find a document in the collection, though this is not necessarily the first or last. If you want to guarantee the first or last, you can run a find() with a sort() and limit() of 1. Sorting by _id ascending should give you the first document. More information here.

// Sort 1 for ascending, -1 for descending.
capped.find().sort([["_id", 1]]).limit(1).nextObject(function(err, item) {
    console.log(item);
});

Finally for question 6, you just use the $set operator with the update() method. More information here.

capped.update({ "foo": 1 }, { "$set": { "bar": 2 } }, {}, function(err, result) {
    console.log(result);
});

Note that you can only update documents in place for capped collections, so you cannot do the insert of the extra field you mention. There are other restrictions enumerated here that you might want to be aware of.

[EDIT: Add updating nested fields in last document.]

If you want to update a nested field in the first or last document (use 1 or -1 in the sort, respectively), you can fetch the document, extract the _id, then perform an atomic update on that document. Something like this:

capped.find().sort([["_id", -1]]).limit(1).nextObject(function(err, item) {       
    if(err) throw err;
    capped.update({ "_id": item._id },
              { "$set": { "timeCollected": 15, "publicIP.ip" : "127.0.0.1" }},
              function(err, result) {

        if(err) throw err;
        console.log(result);    
    });
});  

Note that even when updating a field that exists in a document in a capped collection, you need to ensure that the new value fits in the space allocated for the document. So, for example, updating a string value from "1" to "127.0.0.1" will not necessarily work.

like image 179
Juan Carlos Farah Avatar answered Oct 22 '22 08:10

Juan Carlos Farah