How can I enable users to efficiently save the contents of an indexedDB object store to a file?

Question

I am storing a large amount of small objects in IndexedDB. I would like to give the user the ability to export one of the object stores to a file that they can "download".

I have read this blog article. Which describes reading the data, JSON.stringifying the data, encoding it with encodeURIComponent, and placing it as the href for a link they can use to download the data. Something like this:

var transaction = db.transaction([objectstore], "readonly");
var content = [];
var objectStore = transaction.objectStore(objectstore);

objectStore.openCursor().onsuccess = function(event) {
    var cursor = event.target.result;
    if (cursor) {
        content.push({key:cursor.key,value:cursor.value});
        cursor.continue();
    }
}; 

transaction.oncomplete = function(event) {
    var serializedData = JSON.stringify(dataToStore);
    link.attr("href",'data:Application/octet-stream,'+encodeURIComponent(serializedData));
    link.trigger("click");
};

That is fine, except the object store will have millions of records and I don't feel that this will be performant enough. Is there a way to more directly allow the user to save an object store off as a file (in a way I can import again via the webpage).

---

Edit From some notes in the comments I rewrote a little of how this works, in order to get a little more juice out of it. The new code is similar to:

var transaction = db.transaction([objectstore], "readonly");
var objectStore = transaction.objectStore(objectstore);

objectStore.getAll().onsuccess = function(evt) {
    var url = window.URL.createObjectURL(new Blob(evt.target.results, {'type': 'application/octet-stream'}));
    link.attr('href', url);
    link.trigger('click');
};

Which will give me results like:

• 10k records, 975.87ms average export time
• 100k records, 5,850.10ms average export time
• 1mil records, 56,681.00ms average export time

As you can see 1 million records takes about a minute to export. Is there any better way to be doing this? (I also tried using a cursor instead of .getAll(), but cursors are slower)

Answer 1

IDBObjectStore.getAll is not part of the IndexedDB standard and it's using a cursor under the covers.

Note: Mozilla has also implemented getAll() to handle this case (and getAllKeys(), which is currently hidden behind the dom.indexedDB.experimental preference in about:config). these aren't part of the IndexedDB standard, so may disappear in the future. We've included them because we think they're useful. The following code does precisely the same thing as above:

objectStore.getAll().onsuccess = function(event) {
  alert("Got all customers: " + event.target.result);
};

There is a performance cost associated with looking at the value property of a cursor, because the object is created lazily. When you use getAll() for example, Gecko must create all the objects at once. If you're just interested in looking at each of the keys, for instance, it is much more efficient to use a cursor than to use getAll(). If you're trying to get an array of all the objects in an object store, though, use getAll().

The only way to fetch records where you don't know the key is to use a cursor. So no I don't think there's a better way. But you need to ask yourself if this is faster than fetching the records from a server.