fast lookup in python

Question

I have entries (~88 millions in all) in the following format:

userid  age test    value
111 33  SODIUM  140
111 34  POTASSIUM   4.1
333 65.4    CHLORIDE    107
444 24  BICARBONATE 24

I need to create a fast lookup for the value in last column, given the first three items (ie. userid, age, test).

What is the best way to repeatedly lookup into this data? One that I am thinking of is to create a dictionary in which the keys are the tuple (userid, age, test) and values are value. In the past I used similar method and this method was very slow on much smaller data.

Answer 1

You should use a database, you have too much entries. It is the job of database to do lookup and indexes over so much data.