Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to store 800 billion GPS markers in database [closed]

I need to store GPS tracks that users record into a database. The tracks will consist of a marker every 5 meter of movement for the purpose of drawing a line on a map. I am estimating 200 km tracks which means 40,000 lnlt markers. I estimate 50,000 users minimum and 20 pieces of 200 km tracks for each. That means at least 40 billion lnlt markers.

This needs to scale too, so for 1 million users I need capacity for 800 billion GPS markers.

Since each set of 40,000 markers belong to a single track, we are talking 1 - 20 million records/sets of GPS tracks.

Requirements: Users will request to view these tracks on top of a Google map in a mobile application.

Relations: I currently have 2 tables. Table one has:[trackid], [userid], [comment], [distance], [time], [top speed].

Table 2 has [trackid] [longitude] [latitude] and this is where all GPS markers are stored. What is an efficient way of storing this volume of GPS data while maintaining read performance?

New information:

Storing the GPS data in a KML file for the purpose of displaying them as a track on top of a Google map is a good solution that saves database space. Compressing the KML into a KMZ (basically a zipped KML wit a KMZ extension) greatly reduces file size further. KMZ loads much quicker than GPX and can be integrated with the Google Maps API as a KML layer. See this information from Google for further assistance. This seems to be the best solution so far for the intended requirement.

like image 272
Karl Avatar asked Oct 10 '12 18:10

Karl


1 Answers

The choice of a particular database, as always, is tied to how you want to store the information and how you want to use it. As such, without knowing the exact requirements of your project, as well as the relationships of the data, the best thing to do would be to do some reading on the topic to determine what particular product or storage model is best suited to you.

A good place to start is reading blogs that compare the performance and uses of the databases (see attached):

http://kkovacs.eu/cassandra-vs-mongodb-vs-couchdb-vs-redis

like image 183
jrd1 Avatar answered Oct 19 '22 00:10

jrd1