Are there NOSQL data layer design patterns? [closed]

Question

I've seen a few ideas on this like instead of like Create, Update, Insert, Delete (CRUD) use Get and Put. That's good. However, I haven't seen much yet on how to deal with complexity. I've been told, "just write a method for every query type that you need".

Most of NOSQL seems okay to me until I start thinking about qualifiers (a where clause) -- there could be so many variations. Is there already a good scheme for implementing qualifiers in a sensible way, using just method names & argument conventions? Perhaps there's some kind of verb / noun scheme that works well, but which isn't a language in its own right.

I'm not after the 'right' answer... I'm hoping that there are a few schools of thought that I could learn from.

Found this blog post from the maker of RavenDB: http://ayende.com/blog/4562/ravendb-index-management

Can we implement more than one index on a class?

I even found that it might be possible to serialize anonymous delegates http://blogs.microsoft.co.il/blogs/aviwortzel/archive/2008/06/20/how-to-serialize-anonymous-delegates.aspx I suppose if this is possible, that they might use something like this.

But what if we don't have access to the same assembly (e.g. Silverlight). Found this post here: http://ayende.com/blog/4766/accessing-ravendb-from-silverlight

Is the IEnumerable<T> object searched client-side or server-side? How specific can we get server side in NOSQL in narrowing down the result set before sending it back over the wire, without locking it to one unique id?

UPDATE: I ended up emailing Ayende from RavenDB. He kindly answered the questions that I had (below):

What you can do is write:

  public IEnumerable<T> FindAll(Expression<Func<T,bool>> whereClause)
  {
      return session.Query<T>().Where(whereClause).ToList();
  }

This uses linq to figure out your intent, and then sends the query to the server using RavenDB's syntax. On the server, we analyze your query, and the query optimizer checks to see if there is an existing index that can answer this query, and if there isn't, it will create a temporary index for you.

If you query that temporary index enough, RavenDB will make it permanent. Thus, self optimizing its own operations.

Did you get very far with the "from Silverlight" use case?

We are fully supporting Silverlight.

Can RavenDB handle more than one index server side?

Yes. In fact, we have some customers that have > 500 indexes running with no issues.

END OF INFO FROM Ayende at RavenDB

On designing a query language (i.e. FindAll / where / delegate), mongo seem to achieve a little of this via JSON... http://www.mongodb.org/display/DOCS/Indexes I wish I knew more about it.

This sounds closer: http://www.mongodb.org/display/DOCS/MapReduce

An interesting thread on serializing Serializing anonymous delegates in C#. It's not directly relevant... but I am just trying to look under the hood a little so I know more about potentials.

Answer 1

I am not sure this is applicable to NoSQL, but I implemented a Generic Repository pattern with Raven DB and here is a snippet.

First, I defined a few interfaces

internal interface ISessionProvider : IDisposable
{
    IDocumentSession OpenSession();
    void CloseSession();
}

public interface IDataAccessManager : IDisposable
{
    void Initialize();
    void OpenSession();
    void CloseSession();
}

public interface IRepository<T> where T : Entity
{
    IQueryable<T> Query();
    IEnumerable<T> Find(Func<T, bool> exp);
    T FirstOrDefault(Func<T, bool> exp);

    void Delete(T entity);
    void Add(T entity);
    void Save();

    string PutAttachment(string key, byte[] data);
    Attachment GetAttachment(string key);
    void DeleteAttachment(string key);
}

And this is a shorten implementation

internal class SessionProvider : ISessionProvider
{
    ...

    public IDocumentSession OpenSession()
    {
        session = store.OpenSession();
        return session;
    }

    public void CloseSession()
    {
        if (session != null)
        {
            session.Dispose();
        }
    }
}

public class DataAccessManager : IDataAccessManager
{
    ...

    public void Initialize()
    {       
        store = new DocumentStore
        {
            ConnectionStringName = ConnectionString
        };
        store.Initialize();
        store.DatabaseCommands.EnsureDatabaseExists(dbName);

        provider = new SessionProvider(store);
    }

    public void OpenSession()
    {
        session = provider.OpenSession();
    }

    public void CloseSession()
    {
        provider.CloseSession();
    }
}


public class Repository<T> : IRepository<T> where T : Entity
{
    ...

    public IEnumerable<T> Find(Func<T, bool> exp)
    {
        return AsQuaribale().Where(exp);
    }

    public void Add(T entity)
    {
        session.Store(entity);
    }

    public void Save()
    {
        session.SaveChanges();
    }

    public string PutAttachment(string key, byte[] data)
    {
        Guid? etag = null;
        var metadata = new RavenJObject
        {
            {"owner", Thread.CurrentPrincipal.Identity.Name},
            {"filename", key}
        };
        session.Advanced.DatabaseCommands.PutAttachment(key, etag, data, metadata);

        return key;
    }

    public Attachment GetAttachment(string key)
    {
        return session.Advanced.DatabaseCommands.GetAttachment(key);
    }

    private IQueryable<T> AsQuaribale()
    {
        return session.Query<T>().Customize(x => x.WaitForNonStaleResultsAsOfNow(Timeout));
    }
}

Usage sample

private void SendData()
{
    try
    {
        dataManager.OpenSession();
        repository = new Repository<MyDomainType>();

        ...

        foreach (string path in paths)
        {           
            //read file to memory
            byte[] data = File.ReadAllBytes(path);
            string fName = Path.GetFileName(path);
            myDomainType.Name = fName;

            //save data in memory and metadata to the database
            string key = repository.PutAttachment(
                myDomainType.Id.ToString(), data);

            repository.Add(myDomainType);
        }

        repository.Save();
    }
    catch (Exception ex)
    {
        AppManager.LogException(ex);
    }
    finally
    {
        dataManager.CloseSession();
        dataManager.Dispose();      
    }
}

Sample test for create, which use Find (FirstOrDefault) method for assert

[Test]
public void CreateValueTest()
{
    var repository = ContainerService.Instance.Resolve<IRepository<DummyType>>();
    var expected = new DummyType();
    repository.Add(expected);
    repository.Save();
    DummyType actual = repository.FirstOrDefault(item => item.Id == expected.Id);

    Assert.IsTrue(expected == actual);
}