How to avoid local web app data being erased?

Question

We'd like to have our web app available offline, mainly on mobile devices. We've written code for that, using a service worker. Application data is stored in an IndexedDb and the application code (html, js, css, etc) is stored in the SW cache. So far so good. We are aware that the user can delete the browser cache and our data, that's not a problem. But what about the browser itself wiping the app data ? We haven't found a comprehensive specification for that, the main info we found are:

1) the StorageManager feature that is currently marked as "experimental" (since 2016);

2) a short article from Google here about it (also from 2016).

The code sample is the following:

if (navigator.storage && navigator.storage.persist)
  navigator.storage.persist().then(granted => {
    if (granted)
      alert("Storage will not be cleared except by explicit user action");
    else
      alert("Storage may be cleared by the UA under storage pressure.");
  });

The Google article says:

When storage on the local machine is running tight (“under storage pressure”), user agents automatically clear storage to make more available space. Of course, for offline apps, this may be unfortunate, as they may not have synced their data to the server yet, or they may be apps that the user expects to just work offline (like a music player); so the Storage spec defines two different modes for storage for a given domain - “best effort” and “persistent”. The default mode, of course, is “best effort”. Storage for a domain that is “best effort” (aka “not persistent”) can be cleared automatically, without interrupting or asking the user. However, “persistent” data will not be automatically cleared. (If the system is still under storage pressure after clearing all non-persistent data, the user will need to manually clear any remaining persistent storage.)

...

Beginning with Chrome 55, Chrome will automatically grant the persistence permission if any of the following are true:

• The site is bookmarked (and the user has 5 or less bookmarks)
• The site has high site engagement
• The site has been added to home screen
• The site has push notifications enabled

The permission is automatically denied in all other cases.

The goal is to ensure that users can rely on their favorite web apps and not find they have suddenly been cleared.

That's for Chrome 55, let's suppose the information is up to date. A first glance, their goal sounds reasonable, but if you take a closer look the implementation is geared for "big" sites (à la Google) and not for niche applications that are more task-oriented.

Indeed, when testing on various Android phones on Chrome 80+, the persistence is always refused, with no user interaction. So, "best effort" it is.

We could have stopped the investigation here and called it a day. After all, current phones and PC are sporting ungodly amount of storage, and we only use a few hundred of KB, so we should be fine. Problem is, we're not: testing on a brand new flagship Android phone with Chrome, our code is erased only with a few seconds of fiddling (closing and opening the page a few times is enough). On other platforms it's different, but Android+Chrome will get the most use.

Oddly, only the code in the SW cache (<100KB) is erased, and the bigger IndexedDb is not. So we tried to also put the code in the IndexedDb, and it seems more "persistent" that way, but the code to manage that is also more involved, so it feels somewhat hackish.

Are we alone with that problem ? If not, how are you people dealing with it ?

Bonus question: is there more up to date documentation on the subject somewhere ?

Answer 1

If I understand you correctly, the main issue is that a Chrome browser on Android keeps emptying the browser cache for a website which does not fulfill the Chrome conditions for automatically granted persistence permission.

Up to now, I have not been in this situation yet, but I observed the behavior, you are quoting from https://developers.google.com/web/updates/2016/06/persistent-storage - I just did not know up to now that it explicitly documented.

I see three ways how some websites enforce persistent user data storage:

1. Repeatedly ask the user to add the website to the home-screen and / or to enable push-notifications. I observe that this request comes often somehow under false-flag, i.e. something as "Subscribe to notifications to show your appreciation" rather than "Are we allowed to store data persistently". It may even go so far that a "Install our app" essentially means that a full-screen Chrome instance is added to the home screen of a mobile device, rather than a real app from an app store.

2. Some sites offer a Chrome extension which allows them to store stuff there and even gain more access rights. I personally do not suggest that approach, it somehow creates a strange feeling for security-aware users.

3. Yet another alternative would be an hybrid approach that you offer a native app which in fact is just a customized browser. Bear me with me a second, if this sounds strange at first glance. This option is in fact quite readily available e.g. in React Native as react-native-webbrowser component. It obviously requires programming effort, but quite a few news sites seem to use as approach.

Option 1 and 2 both stick with Chrome, but clearly do not apply for you, since you just want to avoid bothering the user.

Option 3 is unconventional, but may be an option worth considering. Only users who are programmers might realize that they are being somewhat fooled to install something which is essentially just a browser. Nevertheless it is indeed a clean solution: The app store takes care of access right control and you give the user the full choice what happens with the data.