Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Efficient way to bulk insert with get_or_create() in Django (SQL, Python, Django)

Tags:

Is there a more efficient way for doing this?

for item in item_list:     e, new = Entry.objects.get_or_create(         field1 = item.field1,         field2 = item.field2,     ) 
like image 908
kemar Avatar asked Feb 12 '10 14:02

kemar


2 Answers

You can't do decent bulk insertions with get_or_create (or even create), and there's no API for doing this easily.

If your table is simple enough that creating rows with raw SQL isn't too much of a pain, it's not too hard; something like:

INSERT INTO site_entry (field1, field2) (          SELECT i.field1, i.field2          FROM (VALUES %s) AS i(field1, field2)          LEFT JOIN site_entry as existing                  ON (existing.field1 = i.field1 AND existing.field2 = i.field2)          WHERE existing.id IS NULL ) 

where %s is a string like ("field1, field2"), ("field3, field4"), ("field5, field6") that you'll have to create and escape properly yourself.

like image 72
Glenn Maynard Avatar answered Sep 23 '22 17:09

Glenn Maynard


Depends on what you are aiming at. You can use manage.py's loaddata function to load data in a appropriate format (JSON, XML, YAML,...).

See also this discussion.

like image 28
Felix Kling Avatar answered Sep 22 '22 17:09

Felix Kling