Database choice for stock data [closed]

Question

I'm wondering if NoSQL is an option for this scenario:

The input are hourly stock data (sku, amount, price and some more specific) from several sources. Older versions will just get droped. So we won't get over 1 mio. data sets in the near future and there won't be any business intelligence queries like in data warehouses. But there will be aggregations, at least for the minimal price of a group of articles which has to get updated if the article with the minimal price of a group is sold out. In addition to these bulk writes on a frequent base there will be single decrements on the amount of an article which can happen at any time.

The database would be part of a service which needs to give fast responses to requests via REST. So there needs to be some kind of caching. There is no need for stong consistency, but durabiltity.

Further wishlist:

• should scale well for growing request load
• inexpensive technologies in terms of money and complexity (no Oracle cluster)
• no proprietary languages (no PL/SQL)

MongoDB with its aggregation framework seems promising. Can you think of alteratives? (I do not stick to NoSQL!)

Answer 1

I would start with Redis, and here is why:

• "there needs to be some kind of caching" => and that is what Redis is best at. If for any reason you decide that you need "more", you can add "more", but still keep whatever you already developed in Redis as a cache for that "more"

• One Redis is fast. Two Redises are faster. Three Redises are a unit faster than two, etc..

• Learning curve is quite flat, and fun => since set theory really is fun

• Increments / Decrements / Min / Max is a Redis' native talk

• Redis integration with XYZ (you mentioned a need for a REST API) is all over google and github

• Redis is honest <= actually one of my favorite features of Redis

---

MongoDB would work at first, so will ANY other major NoSQL, but why!?

I would go with Redis, and if you decide later you need "more", I would first look at "Redis + SQL db (Postgre/MySQL/etc..)", it will give you both of two worlds => "Caching / Speed" and the "Aggregation Power" in case you aggregations would need to go above and beyond Min/Max/Incr/Decr.

Whoever tells you PostgreSQL "is not fast enough for writing" does not know it.

Whoever tells you that MySQL "is not scalable enough" does not know it (e.g. Facebook runs on MySQL).

As I am already on the roll :) => whoever tells you MongoDB has "replica sets and sharding" does not wish you well, since replica sets and sharding only look sexy from the docs and hype. Once you need to reshard / reorg replica sets, you'll know the price of a wrong shard key selection and magic chunk movements...

Again => Redis FTW!