Firebase listeners take up huge amount of memory

Question

I have a node worker running along side my app on Heroku, that listens on certain paths on my Firebase database. Issue is that listening to these paths seem to take up a huge amount of memory. If I listen for changes on a path like the one below with 13000 items in it, if takes up 147mb of total memory on my Heroku server:

setInterval =>
  @ref.log_memory('Listener interval')
, 1000

@ref.firebaseClient.child('listings').on 'child_changed', (snap) =>
  @ref.log('child_changed')

Output:

22:39:07 worker.1 | info:    Memory: 35mb total - 66mb rss - 23mb heapUsed
22:39:08 worker.1 | info:    Memory: 36mb total - 67mb rss - 18mb heapUsed
22:39:09 worker.1 | info:    Memory: 37mb total - 69mb rss - 23mb heapUsed
22:39:10 worker.1 | info:    Memory: 54mb total - 72mb rss - 25mb heapUsed
22:39:11 worker.1 | info:    Memory: 54mb total - 82mb rss - 33mb heapUsed
22:39:13 worker.1 | info:    Memory: 147mb total - 186mb rss - 94mb heapUsed
22:39:14 worker.1 | info:    Memory: 147mb total - 186mb rss - 94mb heapUsed
22:39:15 worker.1 | info:    Memory: 147mb total - 186mb rss - 94mb heapUsed
22:39:15 worker.1 | info:    child_changed
22:39:16 worker.1 | info:    Memory: 147mb total - 186mb rss - 95mb heapUsed
22:39:17 worker.1 | info:    Memory: 147mb total - 186mb rss - 95mb heapUsed

If it normal that Firebase would take up so much memory for a path it listens to? Is it because it fetches all the items in the path and listens to each item? Is there any way to get around this?

Answer 1

Firebase synchronizes the data at the location (or in the query) that you listen on. It keeps a copy of all the active data at that location in memory.

To reduce the memory usage, listen to a location with less data. Or use a query (e.g. with limitToLast()) to reduce the amount of data that is active.

It is typically a good idea to separate your active data from your historical data when using a NoSQL database. By keeping the active data set small, you can reduce the resource usage for many operation. In your case not only would it reduce memory usage in your Heroku server, but it would also reduce the memory/CPU required on the Firebase servers, which will speed up operation as well.