In need of an embeddable NoSQL database that handles ~1Gb datasets, persisted on disk

Question

I am building an Electron app, for which I need to select an embeddable NoSQL database. In fact, this database is supposed to hold a local subset of data stored on an ArangoDB remote backend. I have been searching the Internet a lot, but fail so far to converge to an ultimate candidate. I hope that somebody could advise me from experience.

Typical datasets amount to possibly ~tens of thousands of documents, and I can imagine cases where the set would amount to ~1Gb over time. Furthermore, I have the need for secondary indexes.

I have looked at PouchDB, UnQlite, LokiJS, LevelDB, NeDB, LinvoDB...

In the end, NeDB and LinvoDB seem like reasonable candidates with persistence to disk (SQlite-like), where NeDB cannot handle large datasets; something which LinvoDB, a fork of NeDB, seems to be able to handle. LinvoDB does not load the whole database in memory, but appears to index "everything" by default and keep that in memory.

On the other hand, I have tried to follow several conversations regarding their indexes, where NeDB appears to suggest in their documentation that they are persisted to disk (https://github.com/louischatriot/nedb#indexing), once built, which appears then again to be negated by LinvoDB (sorry, I lost many of the quotes/sources in the vast amount of tabs open...), suggesting indexes are to be build from scratch on launch. (And it may also be I misunderstand NeDB's documentation althogether.)

Basically, what I need, is a JS database solution for an Electron app, which may hold "considerable" but not "huge" amounts of data. The app's loading times should be reasonable (i.e., not discourage usage), while being responsive (i.e., database should contain secondary indexes) and respecting the user's resources as much as possible.

Questions:

1. Has anybody any experience with above or other embedded NoSQL databases, by which any of these or others could be recommended for my use case?
2. If indeed LinvoDB's indexes need to be rebuilt from scratch every time I launch the app, could that be a significant performance hit (loading time of the order of seconds)? (Surely I'd have to benchmark this...)
3. ArangoDB is not embeddable, but perhaps I should consider to just deploy it as a service alongside my native app? This link NoSQL database: ArangoDB appears to suggest that the developers themselves do not discourage this. Would this be overkill and/or not user friendly? A performance hit?

Any advise would really be appreciated.

Answer 1

Have the same need, seems linvodb3 is the best choice currently. It's under positive developing and the target is dedicated to Electron desktop environment.