Emscripten: is IDBFS more RAM efficient than MEMFS?

Question

I am packaging the data for my WebAssembly game using emcc --preload-file command, which puts it into MEMFS and keeps it in RAM until the webpage is closed.

The game typically opens a file, reads it's contents, closes it, and never touches the file again, it is not using mmap.

Would it be more memory efficient to download all game data into IndexedDB / Emscripten IDBFS on first launch? Does the web browser load the whole IndexedDB into RAM, does it free up RAM after the file in IDBFS was closed?

My target hardware is Safari on iPhone XS with 2GB RAM, and the game data is around 60 MB.

Answer 1

I believe the accepted answer doesn't emphasize an important technical detail about Emscripten's IDBFS.

IDBFS is an additional layer on top of MEMFS with persistence features added through syncfs calls.

All filesystem contents are present in memory and may be optionally either loaded from IndexedDB before use (by calling syncfs(true, ...)) and/or saved into IndexedDB after use (syncfs(false, ...)).

In the current implementation, you cannot prevent additional memory use by using IDBFS. IDBFS is like a ramdisk that can be synchronized from/into persistent storage. It is not like directly mounting a persistent storage.

Answer 2

As you mentioned, MEMFS is an in-memory "filesystem" and it works in any browser. IndexedDB is a proper database and it's optimal use-case is for storing large amounts of data (more than what can fit in RAM).

Often times when an app or website is loaded for the first time there's a lot of API calls, authentication, and other work that needs to be done. Doing this every time the application opens up is not optimal so IndexedDB can be used as a way to speed up repeated visits. This way, the latest app state could be stored in IndexedDB and the app can then sync in the background (see stale-while-revalidate).

If you think your game could benefit from this, then you could consider using it. If you think you could also do this would MEMFS, however, I would recommend that since IndexedDB has some nuances.

In the specific case of safari on IOS, you can't store blobs in IndexedDB. You can, however, convert the blob into an ArrayBuffer but you'll also have to store the MIME type alongside the buffer in order to do the conversion correctly (since ArrayBuffer doesn't have a MIME type but blob does).

Writing to storage in IndexedDB may also fail. This could happen for a variety of reasons. Maybe the user is using private mode, or maybe they don't have enough disk space.

If the user clears their cache on their phone, the IndexedDB data may be cleared as well. You may need to handle this.

I've run into some of these errors in the past using IndexedDB and it can be a headache. I would recommend Memfs.

As for the implementation details of IDBFS, it may or may not remain constant across browsers and platforms. What I can say for sure, however, is that unless you are saving a lot of data that needs to be computed in your first application, try profiling memfs and if it works well enough, then you can stick with it.