Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Does CQRS With OLTP and OLAP Databases Make Sense?

I have several OLTP databases with API's talking to them. I also have ETL jobs pushing data to an OLAP database every few hours.

I've been tasked with building a custom dashboard showing hight level data from the OLAP database. I want to build several API's pointing to the OLAP database. Should I:

  1. Add to my existing API's and call the OLAP database and use a CQRS type pattern, so reads come from OLAP, while writes come from OLTP. My concern here is that there could be a mismatch in the data between reads and writes. How mismatched the data is depends on how often you run the ETL jobs (Hours in my case).
  2. Add to my existing API's and call the OLAP databases then ask the client to choose whether they want OLAP or OLTP data where API's overlap. My concern here is that the client should not need to know about the implementation detail of where the data is coming from.
  3. Write new API's that only point to the OLAP database. This is a lot of extra work.
like image 410
Muhammad Rehan Saeed Avatar asked Feb 27 '17 12:02

Muhammad Rehan Saeed


People also ask

Can a database be both OLTP and OLAP?

Different OLTP databases can be the source of aggregated data for OLAP, and they may be organized as a data warehouse.

Is OLAP normalized or denormalized?

OLAP uses the data warehouse. Insert, Update, and Delete information from the database. Tables in OLTP database are normalized. Tables in OLAP database are not normalized.


1 Answers

Don't use #1: when management talk of analyzed reports it don't bother data mismatch between ETL process - obviously you will generate a CEO report after finishing ETL for the day

Don't use #2: this way you'll load transnational system with analytic overhead and dissolve isolation between purpose of two systems (not good for operation and maintenance)

Use #3 as its the best way to fetch processed results, Use modern tools like Excel, PowerQuery, PowerBI to allow you to create rich dashboard with speed instead of going into tables and writing APIs.

like image 121
SACn Avatar answered Oct 11 '22 00:10

SACn