How does Ember Data manage large amount of records?

Question

I have been working with Ember Data and i'm trying to understand some concepts. I have a quite heavy data intensive app, my back-end has endpoints that return a lot of records.

So, basically i have Route's that have something like this.store.findAll('places') which can return thousands of places having each one several text intensive fields like services or description. This is only one of the resources, there are a few more that handle that amount of data as well.

My main concern is that the app hits some kind of limit or becomes unresponsive. So my question is that: How does Ember Data manage large amount of records ? Is there any best practice to handle those kind of scenarios ?

Answer 1

How does Ember Data manage large amount of records?

The same way as it handles a small amount of records. It's not going to do anything special for performance if you try to load/fetch a large number of records. You need to handle that yourself.

Is there any best practice to handle those kind of scenarios?

Unfortunately, no. Pagination of some sort is really the only way to accomplish this. But as you can see in this thread, there's quite a bit of discussion about the "best" way to do it. There are adapters and plugins made to handle this scenario, as well as server-side boilerplate designed to make it easy. But there really is no canonical way of doing pagination with Ember Data.

In my opinion, the best way to handle large amounts of data is to design a query endpoint and implement it on your server, handling everything yourself. This will be the most tailored to your application and the easiest to understand. If it sounds complicated, that's because it is. Data set segmentation/pagination is not a simple problem to solve, you will definitely run into issues along the way. That's why there's no agreed-upon best practice yet.

Update: Javier Cadiz mentioned the JSON API in the comments so I thought I would mention it. The JSON API does seem to be the new defacto standard for Ember Data, and it does specifiy a pagination method. However, the JSON API is fairly new and isn't widely adopted yet. I believe it wasn't until very recently that Ember Data switched to the JSON API adapter as its default. Using this pagination would mostly likely require you to conform to the entire API, not just the pagination aspect. (Although you can always steal certain ideas from it.) Because of that, I'm not sure if I'd call it a best practice just yet.

Bottom line: the JSON API way of pagination may be the way of the future, but it's not currently very popular. (Although that's just my opinion based on what I see/read. There's no saying how many people are using it privately.)