Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Best way to store a huge list with hashes in Javascript

I have a list with 10.000 entrys.

for example

myList = {};
myList[hashjh5j4h5j4h5j4]
myList[hashs54s5d4s5d4sd]
myList[hash5as465d45ad4d]
....

I dont use an array (0,1,2,3) because i can check very fast
-> if this hash exist or not.

if(typeof myObject[hashjh5j4h5j4h5j4] == 'undefined')
{
  alert('it is new'); 
}
else
{
  alert('old stuff'); 
}

But i am not sure, is this a good solution?
Is it maybe a problem to handle an object with 10.000 entries?

EDIT:
I try to build an rss feed reader which shows only new feeds. So i calculate an hash from the link (every news has an uniqe link) and store it in the object (mongoDB). BTW: 10.000 entrys is not the normal case (but it is possible)

like image 335
Peter Avatar asked Sep 23 '11 07:09

Peter


2 Answers

You can use the in operator:

if ('hashjh5j4h5j4h5j4' in myList) { .. }

However, this will also return true for members that are in the objects prototype chain:

Object.prototype.foo = function () {};
if ("foo" in myList) { /* will be true */ };

To fix this, you could use hasOwnProperty instead:

if (myList.hasOwnProperty('hashjh5j4h5j4h5j4')) { .. }

Whilst you yourself may not have added methods to Object.prototype, you cannot guarantee that other 3rd party libraries you use haven't; incidentally, extending Object.prototype is frowned upon, so you shouldn't really do it. Why?; because you shouldn't modify things you don't own.

like image 36
Rusty Fausak Avatar answered Oct 05 '22 03:10

Rusty Fausak


My advice:

  1. Use as small of a hash as possible for the task at hand. If you are dealing with hundreds of hashable strings, compared to billions, then your hash length can be relatively small.
  2. Store the hash as an integer, not a string, to avoid making it take less room than needed.
  3. Don't store as objects, just store them in a simple binary tree log2(keySize) deep.

Further thoughts:

  1. Can you come at this with a hybrid approach? Use hashes for recent feeds less than a month old, and don't bother showing items more than a month old. Store the hash and date together, and clean out old hashes each day?
like image 67
Sky Kelsey Avatar answered Oct 05 '22 04:10

Sky Kelsey