Using ES6 Proxy to lazily load resources

Question

I am building something like an ActiveRecord class for documents stored in MongoDB (akin to Mongoose). I have two goals:

1. Intercept all property setters on a document using a Proxy, and automatically create an update query to be sent to Mongo. I've already found a solution for this problem on SO.

2. Prevent unnecessary reads from the database. I.e. if a function is performed on a document, and that function only ever sets properties, and doesn't ever use an existing property of the document, then I don't need to read the document from the database, I can directly update it. However, if the function uses any of the document's properties, I'd have to read it from the database first, and only then continue on with the code. Example:

   // Don't load the document yet, wait for a property 'read'.
   const order = new Order({ id: '123abc' });
   // Set property.
   order.destination = 'USA'; 
   // No property 'read', Order class can just directly send a update query to Mongo ({ $set: { destination: 'USA' } }).      
   await order.save();                            
   

   // Don't load the document yet, wait for a property 'read'.
   const order = new Order({ id: '123abc' });
   // Read 'weight' from the order object/document and then set 'shipmentCost'.
   // Now that a 'get' operation is performed, Proxy needs to step in and load the document '123abc' from Mongo.
   // 'weight' will be read from the newly-loaded document.
   order.shipmentCost = order.weight * 4.5;     
   await order.save();     
   

How would I go about this? It seems pretty trivial: set a 'get' trap on the document object. If it's the first-ever property 'get', load the document from Mongo and cache it. But how do I fit an async operation into a getter?

Answer 1

arithmetic cannot be async

You can probably initiate an async read from within a getter (I haven't tried it, but it seems legit), but the getter can't wait for the result. So, unless your DB library provides some blocking access calls, this line, where order.weight is fetched just in time and the value used in multiplication, will always be pure fantasy in any lazy-read regime:

order.shipmentCost = order.weight * 4.5

(If your DB library does have blocking reads, I think it will be straightforward to build what you want by using only blocking reads. Try it. I think this is part of what Sequelize's dataLoader does.)

There's no way for multiplication to operate on Promises. There's no way to await an async value that isn't itself async. Even Events, which are not strictly async/await, would require some async facade or a callback pattern, neither of which are blocking, and so neither of which could make that statement work.

This could work, but it forces every caller to manage lazy-loading:

order.shipmentCost = (await order.weight) * 4.5

That approach will deform your whole ecosystem. It would be much better for callers to simply invoke read & save when needed.

Or you might be able to create a generator that works inside getters, but you'd still need to explicitly "prime the pump" for every property's first access, which would make the "fantasy" statement work at the cost of spawning a horrific pre-statement that awaits instead. Again, better to just use read and save.

---

I think what you're hoping for is impossible within javascript, because blocking and non-blocking behavior is not transparent and cannot be made to be. Any async mechanism will ultimately manifest as not-a-scalar.

You would need to create your own precompiler, like JSX, that could transform fantasy code into async/aware muck.

---

Serious advice: use an off-the-shelf persistence library instead of growing your own.

1. The problem space of data persistence is filled with many very hard problems and edge-cases. You'll have to solve more of them than you think.
2. Unless your entire project is "build better persistence tech," you're not going to build something better than what is out there, which means building your own is just the slowest way to get an inferior solution.
3. More code you write is more bugs to fix. (And you're writing tests for this magic persistence library, right?)

If you're trying to build a real app and you just need to interface with Mongo, spend 15 minutes shopping on npm and move on. Life is too short. Nobody will care how "cool" is your hand-rolled database layer that's almost like ActiveRecord (except for some opinionated customizations and bugs and missing features -- all of which will act as a barrier to others and even yourself).