Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pouchdb replicate from couchdb : very slow

I got a ~10k entries(~30Mo, no attachment) database in my couchDB.

Using Pouchdb browser-side, when replicating from couch, it takes really a while to complete...

What surprise me is the amount of requests my couch receives during it (thousands!, I guess as many as documents) — is that normal ?

Is there a way to "bulk" those requests and generally accelerate the replicating process ?

Thank you.

like image 400
abernier Avatar asked Oct 14 '14 20:10

abernier


People also ask

Why would you use PouchDB along with CouchDB?

CouchDB and PouchDB are build around the idea of syncing your data. But not only live sync, but losing the connection, and continuing accessing and changing your data. And once your back online, sync it. With PouchDB on the clients browser and CouchDB on the backend your web app can become offline first capable.

How does CouchDB replication work?

Replication Procedure. During replication, CouchDB will compare the source and the destination database to determine which documents differ between the source and the destination database. It does so by following the Changes Feeds on the source and comparing the documents to the destination.

Is PouchDB a NoSQL?

PouchDB is an open-source, NoSQL, in-line database. It is designed after CouchDB, which is a NoSQL database that powers npm.

Where is PouchDB data stored?

PouchDB works offline as well as online with the same efficiency. It works offline by storing the data locally and synchronizing it to the servers and CouchDB when online. It stores data locally using IndexedDB and WebSQL in the browser.


1 Answers

I assume you're using the PouchDB.replicate function

In that case, try modifying the batch_size option:

PouchDB.replicate('mydb', 'http://localhost:5984/mydb', {batch_size: large_val})

where large_val is higher than the default of 100. The higher the value, the faster the replication should go, but the more memory it will use, so be careful.

See the API reference

Edit: Also note the option batches_limit which defaults to 10. This is how many requests may run in parallel at any time, so the number of documents in memory equals batch_size * batches_limit.

like image 79
Tom Grant Avatar answered Sep 22 '22 15:09

Tom Grant