Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

Best data store solution for small mathematical data but fast and with aggregate functions

Tags:

database

nosql

storage

cloud

distributed-system

I'm looking for a data storage solution for a project with these requirements:

  1. The application creates dynamically a containter/table in the store.
  2. For a small period of time (two weeks for example) that table/container gets a huge amount of inserts in parallel. Last read have to be immediately available.
  3. The data inserted is very small, 4 to 6 numeric columns.
  4. Small query/filtering support is required, but no joins or cross queries.
  5. Is needed to execute some aggregate functions like "Count", "Sum", "Max", "Min" and "Avg".

Basically, I need something like Windows Azure Table Storage but with aggregate functions.

What would you recommend?

like image 452
vtortola Avatar asked May 26 '11 14:05

vtortola

People also ask

How do you store a large amount of data in a database?

You can just write it to the database, and when the volume exceeds the capacity of the DB to handle, you can shard the database (= have multiple subsets of the data sit on different database servers). Benefit: you can use a relational DB and don't have to learn anything new.

What type of data you should not store in a database select all that apply?

Finally, you shouldn't store credit card information in your database unless you absolutely need to. This includes credit card owner names, numbers, CVV numbers, and expiration dates.

What are aggregates in SQL?

What Is an Aggregate Function in SQL? An aggregate function in SQL performs a calculation on multiple values and returns a single value. SQL provides many aggregate functions that include avg, count, sum, min, max, etc.

3 Answers

RavenDB supports all you mentioned and more. Its scales very well, and supports aggregate functions via Map/Reduce queries and Linq queries. It also can run in-memory.

like image 87
synhershko Avatar answered Sep 22 '22 13:09

synhershko

MongoDB is also a good choice. It supports group (aggregate) queries on single-node installation. If you need to scale you can create sharded cluster and use map/reduce for aggregation needs - but Mongo map/reduce performance isn't top level when comapring to other solutions.

If you need big performance in map/reduce you can take look at Hadoop

like image 28
Daimon Avatar answered Sep 22 '22 13:09

Daimon

Why don't you use SQL Azure? It support Aggregate function and meet all of your needs, take a look at this table

like image 32
Nasser Hadjloo Avatar answered Sep 22 '22 13:09

Nasser Hadjloo
Sign in to Comment

Related questions

Does Google Cloud Spanner support stored procedures?
LMDB random writes really slow for large data (~1MB/s)
Schema for chat application using mongodb
Is 35k rows/s slow or fast for EdgeDB?
No suitable servers found (`serverSelectionTryOnce` set): [connection refused calling ismaster on '127 .0.0.1:27017']
How to download a live MySQL db into a local test db on demand, without SSH?
How to split Oracle sql statements for ADO.NET
Delphi: How to get the value of an output parameter of a stored procedure?
Using SUM so that NULL in columns make sum NULL
Java Exception Error - Sqlite preparedStatement.setBlob
Using email instead of login name in django
Wondering how Facebook does the "Mutual friends" feature
Better to commit inside or outside a loop?
storing binary data on cassandra just like MYSQL BLOB binary
Using two databases for an application in grails
programmatically access car dealers inventory
Writing dot net desktop application with MVC design pattern
Slow JOIN query with OR in WHERE statement
Python: Lazy DB from CSV data?
rake db:create generated "if you set the charset manually, make sure you have a matching collation" error

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!