Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Best data store solution for small mathematical data but fast and with aggregate functions

I'm looking for a data storage solution for a project with these requirements:

  1. The application creates dynamically a containter/table in the store.
  2. For a small period of time (two weeks for example) that table/container gets a huge amount of inserts in parallel. Last read have to be immediately available.
  3. The data inserted is very small, 4 to 6 numeric columns.
  4. Small query/filtering support is required, but no joins or cross queries.
  5. Is needed to execute some aggregate functions like "Count", "Sum", "Max", "Min" and "Avg".

Basically, I need something like Windows Azure Table Storage but with aggregate functions.

What would you recommend?

like image 452
vtortola Avatar asked May 26 '11 14:05

vtortola


People also ask

How do you store a large amount of data in a database?

You can just write it to the database, and when the volume exceeds the capacity of the DB to handle, you can shard the database (= have multiple subsets of the data sit on different database servers). Benefit: you can use a relational DB and don't have to learn anything new.

What type of data you should not store in a database select all that apply?

Finally, you shouldn't store credit card information in your database unless you absolutely need to. This includes credit card owner names, numbers, CVV numbers, and expiration dates.

What are aggregates in SQL?

What Is an Aggregate Function in SQL? An aggregate function in SQL performs a calculation on multiple values and returns a single value. SQL provides many aggregate functions that include avg, count, sum, min, max, etc.


3 Answers

RavenDB supports all you mentioned and more. Its scales very well, and supports aggregate functions via Map/Reduce queries and Linq queries. It also can run in-memory.

like image 87
synhershko Avatar answered Sep 22 '22 13:09

synhershko


MongoDB is also a good choice. It supports group (aggregate) queries on single-node installation. If you need to scale you can create sharded cluster and use map/reduce for aggregation needs - but Mongo map/reduce performance isn't top level when comapring to other solutions.

If you need big performance in map/reduce you can take look at Hadoop

like image 28
Daimon Avatar answered Sep 22 '22 13:09

Daimon


Why don't you use SQL Azure? It support Aggregate function and meet all of your needs, take a look at this table

like image 32
Nasser Hadjloo Avatar answered Sep 22 '22 13:09

Nasser Hadjloo