Transforming Results

Question

My colleagues have asked me if given the orderline example, it'd be possible to initialize a viewmodel looking like this:

OrderViewModel
   string OrderId
   string CustomerName
   List<OrderLineViewModel> OrderLines

OrderLineViewModel
   string ProductName
   string ROI
   int Quantity

From an index?

I've tried doing a transform that successfully loads the customer name but could never manage to get the associated product information from the orderline. Can this be done with a transform or would I need to project from index fields?

Cheers,

James

EDIT:

We're trying to populate view models directly from a query. We tried the following index:

public class OrdersViewIndex : AbstractIndexCreationTask<Order>
{
   Map = orders => from order in orders
                   select new {
                                OrderId = order.id
                              };

   Transform = (database, orders) => from order in orders
                                     let customer = database.Load<Customer>(order.customerId)
                                     select new {
                                                  OrderId = order.id,
                                                  CustomerName = customer.Name,
                                                  OrderLines = // This is where I struggled to answer my colleagues questions as i'd need to load product name.
                                                }
}

Answer 1

First, realize that all indexes automatically map the Id into an index entry called __document_id. So there is not much value in mapping it again. All you are doing in this index map is copying it again to another index entry called OrderId.

Second, understand that transforms are not really part of the index, but are just attached to the index definition and executed at runtime. All they really provide is a way to morph the query results on the server. In most cases these are things you can do client-side.

Third, indexes are for querying against non-id fields, and provide possibly stale but eventually consistent results. When you are retrieving documents by their Id (also called the document key), then there is no point in using an index at all. You want to use the .Load() method instead, which provides ACID guarantees, and just retrieves the document from the database.

Now - you had the question of how to get the customer name when your document has just the customer id, and how to get the product name instead of just the product id. Let's assume your documents look like this:

public class Order
{
    public string Id { get; set; }
    public string CustomerId { get; set; }
    public List<OrderLine> OrderLines { get; set; }
}

public class OrderLine
{
    public string ProductId { get; set; }
    public int Quantity { get; set; }
}

public class Customer
{
    public string Id { get; set; }
    public string Name { get; set; }
}

public class Product
{
    public string Id { get; set; }
    public string Name { get; set; }
}

If you are retrieving a single order using its id, you would do the following:

var order = session.Load<Order>(theOrderId);

But now you want to populate some view models like these:

public class OrderVM
{
    public string OrderId { get; set; }
    public string CustomerId { get; set; }
    public string CustomerName { get; set; }
    public List<OrderLineVM> OrderLines { get; set; }
}

public class OrderLineVM
{
    public string ProductId { get; set; }
    public string ProductName { get; set; }
    public int Quantity { get; set; }
}

You would do this using Includes.

var order = session.Include<Order>(x => x.CustomerId)
                   .Include<Order>(x => x.OrderLines.Select(y => y.ProductId))
                   .Load<Order>(theOrderId);

var orderViewModel = new OrderVM
{
    OrderId = order.Id,
    CustomerId = order.CustomerId,
    CustomerName = session.Load<Customer>(order.CustomerId).Name,
    OrderLines = order.OrderLines.Select(x => new OrderLineVM
                 {
                     ProductId = x.ProductId,
                     ProductName = session.Load<Product>(x.ProductId).Name,
                     Quantity = x.Quantity
                 })
};

Despite seeing multiple calls to session.Load(), there is really only one call to the database. The .Include statements made sure all of the related documents were loaded into session with the first call. The subsequent calls just pull it out of the local session.

All of the above is for retrieving a single order by its id. If instead you want to get all orders, or get all orders for a particular customer - then you need to query.

A dynamic query for orders of a particular customer would look like this:

var results = session.Query<Order>().Where(x => x.CustomerId == theCustomerId);

If you wanted to project these to your view models, like before you can use includes:

var results = session.Query<Order>()
    .Customize(x => x.Include<Order>(y => y.CustomerId)
                     .Include<Order>(y => y.OrderLines.Select(z => z.ProductId)))
    .Where(x => x.CustomerId == theCustomerId)
    .Select(x => new OrderVM
    {
        OrderId = x.Id,
        CustomerId = x.CustomerId,
        CustomerName = session.Load<Customer>(x.CustomerId).Name,
        OrderLines = order.OrderLines.Select(y => new OrderLineVM
        {
            ProductId = y.ProductId,
            ProductName = session.Load<Product>(y.ProductId).Name,
            Quantity = y.Quantity
        })
    });

This does work, but you might not want to write this every time. Also, the entire product and customer records have to be loaded in session, when you just wanted a single field from each. This is where transforms can be useful. You can define a static index as follows:

public class Orders_Transformed : AbstractIndexCreationTask<Order>
{
    public Orders_Transformed()
    {
        Map = orders => from order in orders select new { };

        TransformResults = (database, orders) =>
            from order in orders
            select new
            {
                OrderID = order.Id,
                CustomerID = order.CustomerId,
                CustomerName = database.Load<Customer>(order.CustomerId).Name,
                OrderLines = order.OrderLines.Select(y => new
                    {
                        ProductId = y.ProductId,
                        ProductName = database.Load<Product>(y.ProductId).Name,
                        Quantity = y.Quantity
                    })
            };
    }
}

Now when you query, the transform has already set up the data for you. You just have to specify the resulting VM to project into.

var results = session.Query<Order, Orders_Transformed>().As<OrderVM>();

You might have noticed that I didn't include any fields at all in the index map. That is because I wasn't trying to query over any particular field. All of the data came from the document itself - the only entries in the index are the automatically added __document_id entries, and Raven uses those to present data from the document store - for return or for transformation.

Let's say now that I want to query by one of those related fields. For example, I want to get all orders for customers named Joe. To accomplish this, I need to include the customer name in my index. RavenDB 2.0 added a feature that makes this very easy - Indexing Related Documents.

You will need to modify the index map to use the LoadDocument method, as follows:

Map = orders => from order in orders
                select new
                {
                    CustomerName = LoadDocument<Customer>(order.CustomerId)
                };

If you like, you can combine this with either the Includes, or the Transform techniques to get back the full view model.

Another technique would be to store these fields and project from the index. This works very well for single fields like CustomerName, but is probably overkill for complex values like OrderLines.

And finally, another technique to consider is denormalization. Consider for a moment that a Product might have its name changed, or be deleted. You probably don't want to invalidate previous orders. It would be a good idea to copy any product data relevant to the order into the OrderLine object.

public class OrderLine
{
    public string ProductId { get; set; }
    public string ProductName { get; set; }
    public decimal Price { get; set; }
    public int Quantity { get; set; }
}

Once you do that - you no longer have the need to load in product data when retrieving orders. The Transform section becomes unnecessary, and you are left with a simple index projection, as follows:

public class Orders_ByCustomerName : AbstractIndexCreationTask<Order>
{
    public Orders_ByCustomerName()
    {
        Map = orders => from order in orders
                        select new
                        {
                          CustomerName = LoadDocument<Customer>(order.CustomerId).Name
                        };

        Store("CustomerName", FieldStorage.Yes);
    }
}

Which you can query easily with:

var results = session.Query<OrderVM, Orders_ByCustomerName>()
                     .Where(x => x.CustomerName == "Joe")
                     .As<OrderVM>();

Note in the query, the first time I specify OrderVM, I am defining the shape of the index entries. It just sets up lambdas so I can specify x.CustomerName == "Joe". Often, you will see a special "Results" class used for this purpose. It really doesn't matter - I could use any class that had a CustomerName string field.

When I specify .As<OrderVM>() - that is where I actually move from an Order type to an OrderVM type - and the CustomerName field comes along for the ride since we turned on field storage for it.

TL;DR

RavenDB has lots of options. Experiment to find what works for your needs. Proper document design, and careful use of Indexing Related Documents with LoadDocument() will most always remove the need for an index transform.