RavenDb : Update a Denormalized Reference property value

Question

I have implemented the RavenDB Denormalized Reference pattern. I am struggling to wire together the static index and the patch update request required to ensure that my denormalized reference property values are updated when a referenced instance value is changed.

Here is my Domain:

public class User
{
    public string UserName { get; set; }
    public string Id { get; set; }
    public string Email { get; set; }
    public string Password { get; set; }
}

public class UserReference
{
    public string Id { get; set; }
    public string UserName { get; set; }

    public static implicit operator UserReference(User user)
    {
        return new UserReference
                {
                        Id = user.Id,
                        UserName = user.UserName
                };
    }
}

public class Relationship
{ 
    public string Id { get; set; }
    public UserReference Mentor { get; set; }
    public UserReference Mentee { get; set; }
}

You can see that the UserReference contains the Id and the UserName of the referenced User. So now, if I update the UserName for a given User instance then I want the referenced Username value in all the UserReferences to update also. To achieve this I have written a static Index and a Patch Request as follows:

public class Relationships_ByMentorId : AbstractIndexCreationTask<Relationship>
{
    public Relationships_ByMentorId()
    {
        Map = relationships => from relationship in relationships
                                select new {MentorId = relationship.Mentor.Id};
    }
}

public static void SetUserName(IDocumentSession db, User mentor, string userName)
{
    mentor.UserName = userName;
    db.Store(mentor);
    db.SaveChanges();

    const string indexName = "Relationships/ByMentorId";
    RavenSessionProvider.UpdateByIndex(indexName,
        new IndexQuery
        {
                Query = string.Format("MentorId:{0}", mentor.Id)
        },
        new[]
        {
                new PatchRequest
                {
                        Type = PatchCommandType.Modify,
                        Name = "Mentor",
                        Nested = new[]
                                {
                                        new PatchRequest
                                        {
                                                Type = PatchCommandType.Set,
                                                Name = "UserName",
                                                Value = userName
                                        },
                                }
                }
        },
        allowStale: false);
}

And finally a UnitTest that fails because the update is not working as expected.

[Fact]
public void Should_update_denormalized_reference_when_mentor_username_is_changed()
{
    using (var db = Fake.Db())
    {
        const string userName = "updated-mentor-username";
        var mentor = Fake.Mentor(db);
        var mentee = Fake.Mentee(db);
        var relationship = Fake.Relationship(mentor, mentee, db);
        db.Store(mentor);
        db.Store(mentee);
        db.Store(relationship);
        db.SaveChanges();

        MentorService.SetUserName(db, mentor, userName);

        relationship = db
            .Include("Mentor.Id")
            .Load<Relationship>(relationship.Id);

        relationship.ShouldNotBe(null);
        relationship.Mentor.ShouldNotBe(null);
        relationship.Mentor.Id.ShouldBe(mentor.Id);
        relationship.Mentor.UserName.ShouldBe(userName);

        mentor = db.Load<User>(mentor.Id);
        mentor.ShouldNotBe(null);
        mentor.UserName.ShouldBe(userName);
    }
}

Everything runs fine, the index is there but I suspect this is not returning the relationships required by the patch request, but honestly, I have run out of talent. Can you help please?

Edit 1

@MattWarren allowStale=true did not help. However I have noticed a potential clue.

Because this is a unit test I am using an InMemory, embedded IDocumentSession - the Fake.Db() in the code above. Yet when an static index is invoked i.e. when doing the UpdateByIndex(...), it uses the general IDocumentStore, not the specific fake IDocumentSession.

When I change my index definition class and then run my unit-test the index is updated in the 'real' database and the changes can be seen via Raven Studio. However, the fake domain instances (mentor, mentee etc) that are 'saved' to the InMemory db are not stored in the actual database (as expected) and so cannot be seen via Raven Studio.

Could it be that my call to the UpdateByIndex(...) is running against the incorrect IDocumentSession, the 'real' one (with no saved domain instances), instead of the fake one?

Edit 2 - @Simon

I have implemented your fix for the problem outlined in Edit 1 above and I think we are making progress. You were correct, I was using a static reference to the IDocumentStore via the RavenSessionProvder. This is not the case now. The code below has been updated to use the Fake.Db() instead.

public static void SetUserName(IDocumentSession db, User mentor, string userName)
{
    mentor.UserName = userName;
    db.Store(mentor);
    db.SaveChanges();

    const string indexName = "Relationships/ByMentorId";
    db.Advanced.DatabaseCommands.UpdateByIndex(indexName,
                                        new IndexQuery
                                        {
                                                Query = string.Format("MentorId:{0}", mentor.Id)
                                        },
                                        new[]
                                        {
                                                new PatchRequest
                                                {
                                                        Type = PatchCommandType.Modify,
                                                        Name = "Mentor",
                                                        Nested = new[]
                                                                {
                                                                        new PatchRequest
                                                                        {
                                                                                Type = PatchCommandType.Set,
                                                                                Name = "UserName",
                                                                                Value = userName
                                                                        },
                                                                }
                                                }
                                        },
                                        allowStale: false);
}
}

You will note that I also reset the allowStale=false. Now when I run this I get the following error:

Bulk operation cancelled because the index is stale and allowStale is false

I reckon we have solved the first problem, and now I am using the correct Fake.Db, we have encountered the issue first highlighted, that the index is stale because we are running super-fast in a unit-test.

The question now is: How can I make the UpdateByIndex(..) method wait until the command-Q is empty and index is considered 'fresh'?

Edit 3

Taking into account the suggestion for preventing a stale index, I have updated the code as follows:

public static void SetUserName(IDocumentSession db, User mentor, string userName)
{
    mentor.UserName = userName;
    db.Store(mentor);
    db.SaveChanges();

    const string indexName = "Relationships/ByMentorId";

    // 1. This forces the index to be non-stale
    var dummy = db.Query<Relationship>(indexName)
            .Customize(x => x.WaitForNonStaleResultsAsOfLastWrite())
            .ToArray();

    //2. This tests the index to ensure it is returning the correct instance
    var query = new IndexQuery {Query = "MentorId:" + mentor.Id};
    var queryResults = db.Advanced.DatabaseCommands.Query(indexName, query, null).Results.ToArray();

    //3. This appears to do nothing
    db.Advanced.DatabaseCommands.UpdateByIndex(indexName, query,
        new[]
        {
                new PatchRequest
                {
                        Type = PatchCommandType.Modify,
                        Name = "Mentor",
                        Nested = new[]
                                {
                                        new PatchRequest
                                        {
                                                Type = PatchCommandType.Set,
                                                Name = "UserName",
                                                Value = userName
                                        },
                                }
                }
        },
        allowStale: false);
}

From the numbered comments above:

1. Putting in a dummy Query to force the index to wait until it is non-stale works. The error concerning the stale index is eliminated.

2. This is a test line to ensure that my index is working correctly. It appears to be fine. The result returned is the correct Relationship instance for the supplied Mentor.Id ('users-1').

   { "Mentor": { "Id": "users-1", "UserName": "Mr. Mentor" }, "Mentee": { "Id": "users-2", "UserName": "Mr. Mentee" } ... }

3. Despite the index being non-stale and seemingly functioning correctly, the actual Patch Request seemingly does nothing. The UserName in the Denormalized Reference for the Mentor remains unchanged.

So the suspicion now falls on the Patch Request itself. Why is this not working? Could it be the way I am setting the UserName property value to be updated?

...
new PatchRequest
{
        Type = PatchCommandType.Set,
        Name = "UserName",
        Value = userName
}
...

You will note that I am just assigning a string value of the userName param straight to the Value property, which is of type RavenJToken. Could this be an issue?

Edit 4

Fantastic! We have a solution. I have reworked my code to allow for all the new information you guys have supplied (thanks). Just in case anybody has actually read this far, I better put in the working code to give them closure:

The Unit Test

[Fact]
public void Should_update_denormalized_reference_when_mentor_username_is_changed()
{
    const string userName = "updated-mentor-username";
    string mentorId; 
    string menteeId;
    string relationshipId;

    using (var db = Fake.Db())
    {
        mentorId = Fake.Mentor(db).Id;
        menteeId = Fake.Mentee(db).Id;
        relationshipId = Fake.Relationship(db, mentorId, menteeId).Id;
        MentorService.SetUserName(db, mentorId, userName);
    }

    using (var db = Fake.Db(deleteAllDocuments:false))
    {
        var relationship = db
                .Include("Mentor.Id")
                .Load<Relationship>(relationshipId);

        relationship.ShouldNotBe(null);
        relationship.Mentor.ShouldNotBe(null);
        relationship.Mentor.Id.ShouldBe(mentorId);
        relationship.Mentor.UserName.ShouldBe(userName);

        var mentor = db.Load<User>(mentorId);
        mentor.ShouldNotBe(null);
        mentor.UserName.ShouldBe(userName);
    }
}

The Fakes

public static IDocumentSession Db(bool deleteAllDocuments = true)
{
    var db = InMemoryRavenSessionProvider.GetSession();
    if (deleteAllDocuments)
    {
        db.Advanced.DatabaseCommands.DeleteByIndex("AllDocuments", new IndexQuery(), true);
    }
    return db;
}

public static User Mentor(IDocumentSession db = null)
{
    var mentor = MentorService.NewMentor("Mr. Mentor", "[email protected]", "pwd-mentor");
    if (db != null)
    {
        db.Store(mentor);
        db.SaveChanges();
    }
    return mentor;
}

public static User Mentee(IDocumentSession db = null)
{
    var mentee = MenteeService.NewMentee("Mr. Mentee", "[email protected]", "pwd-mentee");
    if (db != null)
    {
        db.Store(mentee);
        db.SaveChanges();
    }
    return mentee;
}


public static Relationship Relationship(IDocumentSession db, string mentorId, string menteeId)
{
    var relationship = RelationshipService.CreateRelationship(db.Load<User>(mentorId), db.Load<User>(menteeId));
    db.Store(relationship);
    db.SaveChanges();
    return relationship;
}

The Raven Session Provider for Unit Tests

public class InMemoryRavenSessionProvider : IRavenSessionProvider
{
    private static IDocumentStore documentStore;

    public static IDocumentStore DocumentStore { get { return (documentStore ?? (documentStore = CreateDocumentStore())); } }

    private static IDocumentStore CreateDocumentStore()
    {
        var store = new EmbeddableDocumentStore
            {
                RunInMemory = true,
                Conventions = new DocumentConvention
                    {
                            DefaultQueryingConsistency = ConsistencyOptions.QueryYourWrites,
                            IdentityPartsSeparator = "-"
                    }
            };
        store.Initialize();
        IndexCreation.CreateIndexes(typeof (RavenIndexes).Assembly, store);
        return store;
    }

    public IDocumentSession GetSession()
    {
        return DocumentStore.OpenSession();
    }
}

The Indexes

public class RavenIndexes
{
    public class Relationships_ByMentorId : AbstractIndexCreationTask<Relationship>
    {
        public Relationships_ByMentorId()
        {
            Map = relationships => from relationship in relationships
                                    select new { Mentor_Id = relationship.Mentor.Id };
        }
    }

    public class AllDocuments : AbstractIndexCreationTask<Relationship>
    {
        public AllDocuments()
        {
            Map = documents => documents.Select(entity => new {});
        }
    }
}

Update Denormalized Reference

public static void SetUserName(IDocumentSession db, string mentorId, string userName)
{
    var mentor = db.Load<User>(mentorId);
    mentor.UserName = userName;
    db.Store(mentor);
    db.SaveChanges();

    //Don't want this is production code
    db.Query<Relationship>(indexGetRelationshipsByMentorId)
            .Customize(x => x.WaitForNonStaleResultsAsOfLastWrite())
            .ToArray();

    db.Advanced.DatabaseCommands.UpdateByIndex(
            indexGetRelationshipsByMentorId,
            GetQuery(mentorId),
            GetPatch(userName),
            allowStale: false
            );
}

private static IndexQuery GetQuery(string mentorId)
{
    return new IndexQuery {Query = "Mentor_Id:" + mentorId};
}

private static PatchRequest[] GetPatch(string userName)
{
    return new[]
            {
                    new PatchRequest
                    {
                            Type = PatchCommandType.Modify,
                            Name = "Mentor",
                            Nested = new[]
                                    {
                                            new PatchRequest
                                            {
                                                    Type = PatchCommandType.Set,
                                                    Name = "UserName",
                                                    Value = userName
                                            },
                                    }
                    }
            };
}

Answer 1

Try changing your line:

RavenSessionProvider.UpdateByIndex(indexName,  //etc

to

db.Advanced.DatabaseCommands.UpdateByIndex(indexName,  //etc

This will ensure that the Update command is issued on the same (Fake) document store that you are using in your unit tests.

Answer to edit 2:

There's no automatic way to wait for non-stale results when using UpdateByIndex. You have a couple of choices in your SetUserName method:

1 - Change your datastore to always update indexes immediately like this (NOTE: this may adversely affect performance):

store.Conventions.DefaultQueryingConsistency = ConsistencyOptions.MonotonicRead;

2 - Run a query against your index, just before the UpdateByIndex call, specifying the WaitForNonStaleResults option:

var dummy = session.Query<Relationship>("Relationships_ByMentorId")
.Customize(x => x.WaitForNonStaleResultsAsOfLastWrite())
.ToArray();

3 - Catch the exception throw when the index is stale, do a Thread.Sleep(100) and retry.

Answer to edit 3:

I've finally figured it out, and have a passing test... can't believe it, but it seems to just have been a caching issue. When you re-load your docs for asserting against, you need to use a different session... e.g.

using (var db = Fake.Db())
{
    const string userName = "updated-mentor-username";
    var mentor = Fake.Mentor(db);
    var mentee = Fake.Mentee(db);
    var relationship = Fake.Relationship(mentor, mentee, db);
    db.Store(mentor);
    db.Store(mentee);
    db.Store(relationship);
    db.SaveChanges();

    MentorService.SetUserName(db, mentor, userName);
}

using (var db = Fake.Db())
{
    relationship = db
        .Include("Mentor.Id")
        .Load<Relationship>(relationship.Id);
    //etc...
}

Can't believe I didn't spot this sooner, sorry.