Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

DDD: How to handle large collections

I'm currently designing a backend for a social networking-related application in REST. I'm very intrigued by the DDD principle. Now let's assume I have a User object who has a Collection of Friends. These can be thousands if the app and the user would become very successful. Every Friend would have some properties as well, it is basically a User. Looking at the DDD Cargo application example, the fully expanded Cargo-object is stored and retrieved from the CargoRepository from time to time. WOW, if there is a list in the aggregate-root, over time this would trigger a OOM eventually. This is why there is pagination, and lazy-loading if you approach the problem from a data-centric point of view. But how could you cope with these large collections in a persistence-unaware DDD?

like image 678
Pepster Avatar asked Sep 18 '14 14:09

Pepster


People also ask

What is DDDDD and how is it used?

DDD is good for transactional processing of small amounts of data (OLTP) and doesn’t work well with bulk operations. A bulk operation (or a bulk update) is updating large amounts of data in one database roundtrip. Using a combination of the Specification and Command patterns (adheres to DRY and good for performance).

Can I use DDD to perform bulk updates?

This is also true for this use case of bulk updating a large number of tasks — it falls outside DDD’s "area of comfort". A bulk operation (or a bulk update) is updating large amounts of data in one database roundtrip. If DDD doesn’t work well with bulk updates, what does? That’s where raw SQL shines.

What are the disadvantages of DDD?

The drawback here is the lack of performance: processing and updating tasks separately requires a lot of database roundtrips — one per each such update. DDD in general doesn’t work particularly well outside of OLTP -type operations (transactional processing of smalls amounts of data).

Are there any recommendations from DDD for server side storage?

I don't think there are any recommendations from DDD in particular. But in general, it depends on the specifics of the client and the server. Large amounts of (important) data should be transferred asynchronously to ensure consistency. An intermediate storage might be required here, yes.


1 Answers

As @JefClaes mentioned in the comments: You need to determine whether your User AR indeed requires a collection of Friends.

Ownership does not necessarily imply that a collection is necessary.

Take an Order / OrderLine example. An OrderLine has no meaning without being part of an Order. However, the Customer that an Order belongs to does not have a collection of Orders. It may, possibly, have a collection of ActiveOrders if a customer is limited to a maximum number (or amount) iro active orders. Keeping a collection of historical orders would be unnecessary.

I suspect the large collection problem is not limited to DDD. If one were to receive an Order with many thousands of lines there may be design trade-offs but the order may much more likely be simply split into smaller orders.

In your case I would assert that the inclusion / exclusion of a Friend has very little to do with the consistency of the User AR.

Something to keep in mind is that as soon as you start using you domain model for querying your start running into weird sorts of problems. So always try to think in terms of some read/query model with a simple query interface that can access your data directly without using your domain model. This may simplify things.

So perhaps a Relationship AR may assist in this regard.

like image 72
Eben Roux Avatar answered Sep 19 '22 23:09

Eben Roux