RavenDb performance issue?

Question

I am running a test to compare using RavenDB for our DB backend versus Entity Framework with MSSQL.

I have my test app setup to check and see if RavenDB has any documents in it at startup, and if it doesn't, it queries the SQL database through Entity Framework to get all the objects (about 31,000 items) and then insert them into the RavenDB. This part is working as expected.

Then I run the test. I query against EF for a set of records and convert them to JSON, and repeat the same with Raven (getting the exact same set of records). I expected that Raven would be faster, since it is optimized for reading, and EF has to join on two tables to get the data back. But that is not the case.

Here is the output from the test:

Entity Framework with MS SQL    RavenDB Percent Difference Raven to EF
796.8954 ms (862 records)   1703.1686 ms (862 records)  213.725490196078
296.8826 ms (862 records)   796.8954 ms (862 records)   268.421052631579
281.2572 ms (862 records)   781.27 ms (862 records) 277.777777777778
281.2572 ms (862 records)   796.8954 ms (862 records)   283.333333333333
296.8826 ms (862 records)   765.6446 ms (862 records)   257.894736842105
312.508 ms (862 records)    765.6446 ms (862 records)   245
296.8826 ms (862 records)   750.0192 ms (862 records)   252.631578947368
296.8826 ms (862 records)   750.0192 ms (862 records)   252.631578947368
359.3842 ms (862 records)   765.6446 ms (862 records)   213.04347826087
281.2572 ms (862 records)   765.6446 ms (862 records)   272.222222222222
281.2572 ms (862 records)   796.8954 ms (862 records)   283.333333333333
281.2572 ms (862 records)   765.6446 ms (862 records)   272.222222222222
281.2572 ms (862 records)   812.5208 ms (862 records)   288.888888888889
265.6318 ms (862 records)   781.27 ms (862 records) 294.117647058824
296.8826 ms (862 records)   796.8954 ms (862 records)   268.421052631579
281.2572 ms (862 records)   765.6446 ms (862 records)   272.222222222222
281.2572 ms (862 records)   828.1462 ms (862 records)   294.444444444444
281.2572 ms (862 records)   781.27 ms (862 records) 277.777777777778
328.1334 ms (862 records)   750.0192 ms (862 records)   228.571428571429
296.8826 ms (862 records)   750.0192 ms (862 records)   252.631578947368
312.508 ms (862 records)    781.27 ms (862 records) 250
296.8826 ms (862 records)   796.8954 ms (862 records)   268.421052631579
281.2572 ms (862 records)   765.6446 ms (862 records)   272.222222222222
312.508 ms (862 records)    781.27 ms (862 records) 250
281.2572 ms (862 records)   734.3938 ms (862 records)   261.111111111111

Here is the code used to run the test:

protected void Page_Load(object sender, EventArgs e)
        {

            int totalTests = 25;


            DataTable dt = new DataTable();
            dt.Columns.Add("Entity Framework with MS SQL");
            dt.Columns.Add("RavenDB");
            dt.Columns.Add("Percent Difference Raven to EF");

            for (int i = 1; i <= totalTests; i++)
            {

                string efMilliseconds = string.Empty;
                string ravenMilliseconds = string.Empty;

                double efMS = 0;
                double ravenMS = 0;

                // EF
                using (tamcEntitiesForRavenTest myObjectContext = new tamcEntitiesForRavenTest())
                {
                    DateTime startTime = DateTime.Now;
                    List<Treatment> efTreatments = myObjectContext.Treatments.Include("Segments").Where(x => x.Jurisdiction == "00145").OrderBy(x => x.Treatment_Date).ToList();
                    string json = JsonConvert.SerializeObject(efTreatments);
                    TimeSpan TotalTime = DateTime.Now - startTime;
                    efMS = TotalTime.TotalMilliseconds;
                    efMilliseconds = string.Format("{0} ms ({1} records)", efMS.ToString(), efTreatments.Count);
                }

                // Raven
                using (var session = DataDocumentStore.Instance.OpenSession())
                {
                    DateTime startTime = DateTime.Now;
                    List<RavenTreatment> ravenTreatments = session.Query<RavenTreatment>().Where(x => x.Jurisdiction == "00145").OrderBy(x => x.Treatment_Date).Take(1000).ToList();
                    string json = JsonConvert.SerializeObject(ravenTreatments);
                    TimeSpan TotalTime = DateTime.Now - startTime;
                    ravenMS = TotalTime.TotalMilliseconds;
                    ravenMilliseconds = string.Format("{0} ms ({1} records)", ravenMS.ToString(), ravenTreatments.Count);
                }

                DataRow dr = dt.NewRow();
                dr[0] = efMilliseconds;
                dr[1] = ravenMilliseconds;
                double percentDifference = (ravenMS * 100) / efMS;
                dr[2] = percentDifference;

                dt.Rows.Add(dr);
            }

            GridView1.DataSource = dt;
            GridView1.DataBind();
        }

I have the RavenDB instance running on the same machine as the SQL server.

Is this the expected performance results? Or is there something I am doing wrong.

Answer 1

Amanda, you need to understand that it's not RavenDB itself that is faster than sql server. MSSQL has been out for years and you can be sure that it is extremely micro-optimized and gets you the best results possible in bulk select scenarios like the one you have shown above. This is not where RavenDB will ever have a chance to beat a product like MSSQL.

However, while it is not the database itself that makes the difference, most of the time, it's the applications and its data access strategies that will bring you the lightning peformance that raven is famous of.

The whole thing is about how a document database allows you to structure your data (aggregate roots, denormalized references, precomputed indexes, etc.). This is not something specific to RavenDB as you can do those things with MongoDB and CouchDB as well, so probably the most intrinsic difference among those databases is, that raven gives you a very nice and easy .NET experience and has out-of-the-box multi document transactions. There are many other sweetspots as well, but those are generally the ones that will make you decide between different databases.