Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Maximum number of live replications in PouchDB

I am working on an Ionic app with PouchDB that needs to sync tables with remote CouchDB server. In the constructor of my database.ts provider I have 6 methods:

this.initialiseDB_Ord();
this.initialiseDB_IngProd();
this.initialiseDB_Ing();
this.initialiseDB_Prod();
this.initialiseDB_OrdProd();
this.initialiseDB_Ut();

Each of these methods does the following (I pick the first one as example):

this._DB_Ord = new PouchDB('orders');
this._remoteDB_Ord = this.addressIP + '/orders';
this._syncOpts_Ord = {  live : true,
                        retry : true,
                        continuous : true};
this._DB_Ord.sync(this._remoteDB_Ord, this._syncOpts_Ord)
.on('change',(info) => {
     console.log('Handling syncing change');
     console.dir(info);
}).on('paused',(info)=> {
     console.log('Handling syncing pause');
     console.dir(info);
}).on('active', (info) => {
     console.log('Handling syncing resumption');
     console.dir(info);
}).on('denied', (err) =>{
     console.log('Handling syncing denied');
     console.dir(err);
}).on('complete', (info) =>{
     console.log('Handling syncing complete');
     console.dir(info);
}).on('error', (err)=>{
     console.log('Handling syncing error');
     console.dir(err);
});

then I have a handleSyncing method as follows:

handleSyncingUt() {
  this._DB_Ut.changes({
       since             : 'now',
       live              : true,
       include_docs      : true,
       attachments   : true
  })
  .on('change', (change) =>
  {
     console.log('Handling change');
     console.dir(change);
  })
  .on('complete', (info) =>
  {
     console.log('Changes complete');
     console.dir(info);
  })
  .on('error',  (err) =>
  {
     console.log('Changes error');
     console.log(err);
  });
}

If I have at maximum 5 Databases it works fine. When the sixth DB is added it doesn't synchronize local pouchDB and remote couchDB in real time but only when the app is first opened.

Can someone help me?

like image 465
Giuseppe Faraci Avatar asked Jan 04 '18 16:01

Giuseppe Faraci


People also ask

How fast is CouchDB replication?

The documents range in size from a few bytes to many KiloBytes (150KB or so). When the server is unloaded, the following replication filter function takes about 2.5 minutes to complete. When the server is loaded, it takes >10 minutes.

How does CouchDB replication work?

Replication Procedure. During replication, CouchDB will compare the source and the destination database to determine which documents differ between the source and the destination database. It does so by following the Changes Feeds on the source and comparing the documents to the destination.

Why would you use PouchDB along with CouchDB?

CouchDB and PouchDB are build around the idea of syncing your data. But not only live sync, but losing the connection, and continuing accessing and changing your data. And once your back online, sync it. With PouchDB on the clients browser and CouchDB on the backend your web app can become offline first capable.


2 Answers

@lossleader is right on the money about the max number of connections in a browser/WebView. At Quizster, we have this same problem as Quizster has to sync as many as 10 PouchDB instances simultaneously. This is done using Howler to subscribe to changes for a set of databases. Upon a change to one of these databases, Quizster issues a one-time (not live) sync between the PouchDB instance and the CouchDB cluster.

For a little more background: Quizster has to sync this many PouchDB instances as:

  1. Doing this avoids having to replicate even more data between databases, which leads to lower latency with shared data
  2. Quizster uses a lot of database views to speed up replication and keep the size of the data sets small. And, each view effectively leads to its own PouchDB instance.

I'm planning on open sourcing more of the Quizster stack soon and hope to release some tutorials as well.

like image 164
redgeoff Avatar answered Oct 12 '22 01:10

redgeoff


Browsers have a maximum number of sockets per domain and per page, and you are exceeding it with live:true connections. Chrome shares the first limit across tabs so you can verify multiple tabs of fewer connections cause the problem in chrome, or by raising the limit in firefox's about:config.

You can't really fix this limit for normal users, and need to adjust how you manage connections (for example a worker can manage one db's sync for many tabs in chrome) and a data design might be changed to observing only one "changes" db constantly to know when to run live:false passes on other dbs.

like image 37
lossleader Avatar answered Oct 11 '22 23:10

lossleader