What would be faster ? Query mysql to see if the piece of information i need is there, OR load a python dictionary with all the information then just check if the id is there
If python is faster, then whats the best what to check if the id exists?
Im using python 2.4.3
Im searching for data which is tagged to a square on a board, im searching for x&y. Theres only one entry per square, the information wont change and it needs be recalled several times a second.
Thankyou !
I worked out it was python. I ran the code bellow, and mysql did it in 0.0003 of a second, but python did it in 0.000006 of a second and mysql had far far less to search and the test was run how the code would be running in real life. Which one had less overhead in terns of CPU and RAM i will never know but if speed is anything to go by python did much better.
And thankyou for your answers !
def speedtest():
global search
global data
qb = time.time()
search.execute("SELECT * FROM `nogo` where `1`='11' AND `2`='13&3'")
qa = search.fetchall()
print qa[0]
qc = time.time()
print "mysql"
print qb
print qc
print qc - qb
data = {}
for qa in range(15):
data[qa] = {}
for qb in range(300):
data[qa][str(qb)] = 'nogo'
qw = 5
qe = '50'
qb = time.time()
print data[qw][qe]
qc = time.time()
print "dictionary"
print qb
print qc
print qc - qb
Python can be used in database applications. One of the most popular databases is MySQL.
The fastest way to repeatedly lookup data with millions of entries in Python is using dictionaries. Because dictionaries are the built-in mapping type in Python thereby they are highly optimized.
Python dictionaries can be used when the data has a unique reference that can be associated with the value. As dictionaries are mutable, it is not a good idea to use dictionaries to store data that shouldn't be modified in the first place.
SQLite is probably the most straightforward database to connect to with a Python application since you don't need to install any external Python SQL modules to do so. By default, your Python installation contains a Python SQL library named sqlite3 that you can use to interact with an SQLite database.
Generally speaking, if you want information from a database, ask the database for what you need. MySQL (and other database engines) are designed to retrieve data as efficiently as possible.
Trying to write your own procedures for retrieving data is trying to outsmart the talented people who have already imbued MySQL with so much data processing power.
This isn't to say it's never appropriate to load data into Python, but you should be sure that a database query isn't the right way to go first.
Python ought to be much faster, but this largely depends on your specific scenario.
my_dict.has_key('foobar')
You may want to check out Bloom filters as well.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With