Writing insert statements with knex.js on Node API confusion

Question

I've a problem I can't really seem to wrap my head around. It's very specific to the Knex.JS implementation and I'm sure has nothing to do with PostgreSQL.

The following implementation works. When inserting a moderate amount (~500 statements). On larger amounts this fails due to other reasons. Regardless, the following will not work for my use case, I need something like the next section.

import knex = require("knex");
(function (items) {
  let db = knex.table("items");

  db.truncate();

  let foo = [];
  items.forEach(function(item) {
    foo.push({
       id : item.id,
       item_data : JSON.stringify(item)
    });
  });

  db.insert(foo).then(function () { /*..*/ });

}(items))

But the following doesn't:

import knex = require("knex");
(function (items) {
  let db = knex.table("items");

  db.truncate();

  let foo = [];
  items.forEach(function(item) {
    db.then(function() {
        return db.insert(foo).into("items");
    });
  });

  db.then(function () { console.log("Done"); });

}(items))

What doesn't work is this:

• An inconsistent amount of rows are inserted. In some implementations it's a lot MORE than I have items (?!)
• I get a lot of duplicate key errors in this implementation, since I have a unique constraint

Additional information:

• The set contains no duplicate keys
• I'm using PostgreSQL as backend

The question is mostly how to implement the desired behaviour. The ideal situation deals in chunks of say 500 "items". I've already posted a question with the project (https://github.com/tgriesser/knex/issues/826) but I'm hoping some people of the Knex.JS community are more active here on SO.

Answer 1

Your solution is correct (promise chaining), however since you're using Knex it ships with Bluebird which already provides a utility method for this:

var Promise = require("bluebird"); // also used internally by Knex so free to require

Promise.each(items, db.insert.bind(db)); 

Would do the same thing as:

items.forEach(function(item) {
  chain = chain.then(function () {
     return db.insert(item);
  });
});