Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to access Google Chrome's IndexedDB/LevelDB files?

I want to use Google Chrome's IndexedDB to persist data on the client-side.

Idea is to access the IndexedDB outside of chrome, via Node.JS, later on. The background is the idea to track usage behaviour locally and store the collected data on the client for later analysis without a server backend.

From my understanding, the indexedDB is implemented as a LevelDB. However, I cannot open the levelDB with any of the tools/libs like LevelUp/LevelDown or leveldb-json.

I'm always getting this error message:

leveldb-dump-to-json --file test.json --db https_www.reddit.com_0.indexeddb.leveldb  events.js:141     throw er; // Unhandled 'error' event         ^   OpenError: Invalid argument: idb_cmp1 does not match existing   comparator : leveldb.BytewiseComparator       at /usr/local/lib/node_modules/leveldb-  json/node_modules/levelup/lib/levelup.js:114:34 Christians-Air:IndexedDB  

Can anybody please help? It seems as if the Chrome implementation is somehow special/different.

like image 652
Chris Avatar asked Jan 29 '16 00:01

Chris


People also ask

Where is IndexedDB stored Chrome?

More specifically, IndexedDB data is stored in the browser profile folder.

What is IndexedDB LevelDB?

IndexedDB – a quick re-primer LevelDB is a pure key-value store with raw binary keys and values. In order to understand how IndexedDB is being implemented, we need to know how the structure, metadata, and records of the IndexedDB store are translated into the raw keys and values in LevelDB.

What is LevelDB in Chrome?

LevelDB is used as the backend database for Google Chrome's IndexedDB and is one of the supported backends for Riak. Additionally, Bitcoin Core and go-ethereum store the blockchain metadata using a LevelDB database. Minecraft Bedrock Edition uses a modified version for chunk and entity data storage.


2 Answers

Keys in leveldb are arbitrary binary sequences. Clients implement comparators to define ordering between keys. The default comparator for leveldb is something equivalent to strncmp. Chrome's comparator for Indexed DB's store is more complicated. If you try and use a leveldb instance with a different comparator than it was created with you'll observe keys in seemingly random order, insertion would be unpredictable or cause corruption - dogs and cats living together, mass hysteria. So leveldb lets you name the comparator (persisted to the database) to help detect and avoid this mistake, which is what you're seeing. Chrome's code names its comparator for Indexed DB "idb_cmp1".

To inspect one of Chrome's Indexed DB leveldb instances outside of chrome you'd need to implement a compatible comparator. The code lives in Chrome's implementation at content/browser/indexed_db/indexed_db_backing_store.cc - and note that there's no guarantee that this is fixed across versions. (Apart from backwards compatibility, of course)

like image 184
Joshua Bell Avatar answered Sep 22 '22 17:09

Joshua Bell


It's implemented and public available on github now

  • C# example
  • Python example
like image 25
daisy Avatar answered Sep 22 '22 17:09

daisy