Firebase Android offline performance

Question

When storing approximately 5000 sub nodes under a single node, initialising firebase becomes very slow when making use of the offline capabilities. It takes ~30 seconds before the first query is executed. Once initialised, executing subsequent queries (e.g. listing the first 25 sub nodes) takes less than a second.

I'm making use of the following properties to enable the offline capabilities: Firebase.getDefaultConfig().setPersistenceEnabled(true); firebase.keepSynced(true);

My structure looks like this:

<root>
 |-my-app-name
   |-<uid>
     |-node
       |-sub node 1
       |-...
       |-sub node 5000

Keep synced is set on the <uid> node. The sub nodes are presented in a Recycler View. Preferably, I would like to list all (instead of 25 per page) but I understand that this is not possible since there is no Cursor like mechanism (as Android provides for SQLite) available for working with Firebase.

Is this by design and do I revise my data structure? Or can I reduce initialisation time in another way?

I provided some logging below. As you can see, a lot of garbage collection is going on. Does Firebase evaluate the whole database when initializing?

Thanks! Niels

04-01 15:59:12.029 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 43005(1717KB) AllocSpace objects, 0(0B) LOS objects, 4% free, 31MB/32MB, paused 5.674ms total 57.402ms
04-01 15:59:13.415 2222-2240/abcdef W/art: Suspending all threads took: 6.600ms
04-01 15:59:13.424 2222-2245/abcdef W/art: Suspending all threads took: 9.339ms
04-01 15:59:13.433 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 7097(281KB) AllocSpace objects, 0(0B) LOS objects, 0% free, 32MB/32MB, paused 11.175ms total 27.105ms
04-01 15:59:13.821 2222-2245/abcdef I/art: Background partial concurrent mark sweep GC freed 101674(5MB) AllocSpace objects, 18(530KB) LOS objects, 35% free, 28MB/44MB, paused 3.400ms total 152.664ms
04-01 15:59:15.107 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 394024(15MB) AllocSpace objects, 0(0B) LOS objects, 20% free, 30MB/38MB, paused 1.865ms total 152.182ms
04-01 15:59:15.817 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 218328(8MB) AllocSpace objects, 0(0B) LOS objects, 19% free, 31MB/38MB, paused 1.711ms total 112.325ms
04-01 15:59:16.451 2222-2240/abcdef W/art: Suspending all threads took: 27.786ms
04-01 15:59:16.465 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 190591(7MB) AllocSpace objects, 0(0B) LOS objects, 18% free, 31MB/38MB, paused 1.832ms total 107.416ms
04-01 15:59:16.472 2222-2245/abcdef W/art: Suspending all threads took: 6.823ms
04-01 15:59:17.084 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 178714(6MB) AllocSpace objects, 0(0B) LOS objects, 15% free, 32MB/38MB, paused 1.717ms total 105.529ms
04-01 15:59:17.629 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 163584(6MB) AllocSpace objects, 0(0B) LOS objects, 14% free, 33MB/38MB, paused 1.743ms total 110.764ms
04-01 15:59:18.941 2222-2240/abcdef W/art: Suspending all threads took: 5.078ms
04-01 15:59:19.691 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 95627(3MB) AllocSpace objects, 0(0B) LOS objects, 8% free, 35MB/38MB, paused 7.190ms total 86.171ms
04-01 15:59:19.961 2222-2240/abcdef W/art: Suspending all threads took: 18.208ms
04-01 15:59:20.965 2222-2245/abcdef W/art: Suspending all threads took: 5.254ms
04-01 15:59:20.990 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 55899(2MB) AllocSpace objects, 0(0B) LOS objects, 5% free, 36MB/38MB, paused 6.799ms total 66.923ms
04-01 15:59:22.495 2222-2240/abcdef W/art: Suspending all threads took: 45.180ms
04-01 15:59:22.509 2222-2245/abcdef W/art: Suspending all threads took: 14.254ms
04-01 15:59:22.562 2222-2245/abcdef I/art: Background partial concurrent mark sweep GC freed 198174(6MB) AllocSpace objects, 3(487KB) LOS objects, 32% free, 33MB/49MB, paused 16.949ms total 215.369ms
04-01 15:59:23.811 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 392437(15MB) AllocSpace objects, 0(0B) LOS objects, 18% free, 35MB/43MB, paused 1.936ms total 168.222ms
04-01 15:59:24.480 2222-2240/abcdef W/art: Suspending all threads took: 22.464ms
04-01 15:59:24.497 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 227043(8MB) AllocSpace objects, 0(0B) LOS objects, 18% free, 35MB/43MB, paused 1.723ms total 117.855ms
04-01 15:59:25.173 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 203910(7MB) AllocSpace objects, 0(0B) LOS objects, 16% free, 36MB/43MB, paused 1.694ms total 112.618ms
04-01 15:59:25.181 2222-2245/abcdef W/art: Suspending all threads took: 7.301ms
04-01 15:59:25.784 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 185627(7MB) AllocSpace objects, 0(0B) LOS objects, 14% free, 37MB/43MB, paused 1.719ms total 115.362ms
04-01 15:59:26.345 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 167066(6MB) AllocSpace objects, 0(0B) LOS objects, 13% free, 37MB/43MB, paused 1.651ms total 106.055ms
04-01 15:59:26.865 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 154535(6MB) AllocSpace objects, 0(0B) LOS objects, 11% free, 38MB/43MB, paused 1.644ms total 104.888ms
04-01 15:59:28.357 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 151375(5MB) AllocSpace objects, 33(671KB) LOS objects, 9% free, 39MB/43MB, paused 2.740ms total 104.176ms
04-01 15:59:29.006 2222-2240/abcdef W/art: Suspending all threads took: 19.232ms
04-01 15:59:29.060 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 133554(5MB) AllocSpace objects, 29(580KB) LOS objects, 10% free, 39MB/43MB, paused 1.563ms total 100.220ms
04-01 15:59:30.173 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 131062(4MB) AllocSpace objects, 31(637KB) LOS objects, 9% free, 39MB/43MB, paused 1.653ms total 102.705ms
04-01 15:59:31.245 2222-2245/abcdef I/art: Background sticky concurrent mark sweep GC freed 122085(4MB) AllocSpace objects, 26(522KB) LOS objects, 8% free, 39MB/43MB, paused 2.380ms total 100.776ms
04-01 15:59:32.024 2222-2240/abcdef W/art: Suspending all threads took: 20.662ms

PS: This is a cross post: https://groups.google.com/forum/#!topic/firebase-talk/migEAwv26ns

Answer 1

Initializing means setValue to that node right? So, initializing 5000 sub nodes under a single node taking ~30s seems very unusual to me. I've worked with almost same size of data under a single node with a far better performance. So I'm not sure about how much attributes you are putting under a single sub-node, but anyway, I guess you need to check the performance again. I think you're using onCompleteListener on setValue to calculate the time spent to initialize the data, as the UI view doesn't provide you the exact time and often slower than the actual operation time.

Preferably, I would like to list all (instead of 25 per page) but I understand that this is not possible since there is no Cursor like mechanism (as Android provides for SQLite) available for working with Firebase.

I'm not quite sure about your purpose though, but what I could suggest for these type of cases is to maintain both Sqlite and Firebase database. Let me clarify.

The idea is to maintain the same copy of the Firebase database for a specific user in the user's phone. So that the local database can serve your purpose fully whenever needed. You can query the database and can use CursorLoader with which you've handful experience.

It has some other advantages too. You can handle the offline sync with your own mechanism. When the internet is down, store the data you want to sync afterwards in your local Sqlite database and then when the connection is up, you'll get a callback in your BroadcastReceiver. You can easily setValue the offline data to Firebase then. Firebase makes this simpler of course, but anyway, as you're very much concerned about the performance, you can give this a try.

The behaviour of GC you posted is usual when your application is doing too much work. Firebase basically uses WebSocket to maintain the connection to the remote database. So I think you need to check if you're keeping unnecessary connections to the Firebase database. Try to use removeListener when the listeners are no longer necessary.

Does Firebase evaluate the whole database when initializing?

I'm not sure yet what you've meant by initializing, but yes, if you're taking the same node again for setValue to that node, it replaces the previous data with the new set of data.