Why is Realm so fast compared to SQLite?

Question

I have seen the benchmarks on the Realm website, how come Realm is so fast compared to SQLite, which has a long development history, being released back in 2000?

I was wondering if someone with knowledge could share some general techniques that Realm uses underneath the hood in the database layer? As of March, 2016, only the bindings are open-sourced and the db layer is still closed off.

Answer 1

The Realm blog has great technical talks on the subject, but there was no single repository that goes into depth about the reasons behind its performance. This post is an effort to condense the videos and articles in an easily accessible manner.

TL;DR

Realm is designed for mobile from the get-go that uses and therefore makes trade-offs that make sense in the mobile platform whereas SQLite is a generic solution that was ported to mobile.

The longer answer

There are of course a lot of small optimizations that Realm does (integer packing, converting common strings to enums), but in this post I'll try to focus on the differences that account for the most performance benefits.

1. The traditional SQLite + ORM abstraction is leaky because ORM simply converts Objects and their methods into SQL statements. As a result, it is a trade-off for developer productivity at the cost of performance. Realm, on the other hand, is an object database, meaning your objects directly reflect your database. This direct access to the database, leads to the next topic: zero-copy.
2. Zero-copy: the traditional way of reading data from a database leads to unnecessary copying (raw data -> deserialized representation -> language-level objects). Realm avoids this by mapping the whole data in-memory, using B+ trees and whenever data is queried, Realm simply calculates the offset, reads from the memory mapped region and returns the raw value.
3. Lazy loading: because the properties are represented in columns instead of rows, it can lazy load the properties as necessary and because of the column structure, reads are much faster, while inserts are slower. But that is a good trade-off to make in the context of a mobile application.
4. MVCC (Multiversion Concurrency Control): Realm also handles concurrency well using the MVCC model, which means that multiple read transactions can be done at the same time and reads can also be done while a write transaction is being committed.