Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Fastest way to populate a Db [closed]

I have an excel spreadsheet with products and categories with a many to many relationship and i need to populate the db with it. I am searching for a quick way to enter this data in and have the join table have all the correct info. I have phpmyadmin but it seems to be a bit slow unless there is a shortcut. Another approach i was thinking was writing some queries that I could copy and paste but getting the join table to be correct might be more challenging. Unless someone can think of a query that i can create the product and catagory and have the join connection.

PRODUCTS

* PRODUCT_ID (primary key)
* PRODUCT_NAME

CATEGORIES

* CATEGORY_ID (primary key)
* CATEGORY_NAME

PRODUCT_CATEGORIES_MAP

* PRODUCT_ID (primary key, foreign key to PRODUCTS)
* CATEGORY_ID (primary key, foreign key to CATEGORIES)
* QUANTITY
like image 445
Matt Elhotiby Avatar asked Aug 26 '10 23:08

Matt Elhotiby


People also ask

What is the fastest way to populate a database?

If you are loading a freshly created table, the fastest method is to create the table, bulk load the table's data using COPY , then create any indexes needed for the table. Creating an index on pre-existing data is quicker than updating it incrementally as each row is loaded.

What is a common technique to speed up reads from databases?

Replication. Replication is an easy solution if reads are your problem and if a bit of time-delay of updates is not a big deal. Replication continuously copies the database to another machine. It speeds up reads and acts as a failover mechanism.


2 Answers

Most databases have some sort of optimized bulk insertion that is faster/less tedious than using INSERT statements - for MySQL, it's the LOAD DATA INFILE syntax.

Coming from a spreadsheet, it's easy to generate CSV ouptut so you can use:

LOAD DATA INFILE 'data.txt' INTO TABLE your_table
   FIELDS TERMINATED BY ',' ENCLOSED BY '"'
   LINES TERMINATED BY '\r\n'
   IGNORE 1 LINES;
like image 179
OMG Ponies Avatar answered Nov 08 '22 02:11

OMG Ponies


Export them as 3 CSV files and then use LOAD DATA INFILE to load them into the server.

like image 21
Ignacio Vazquez-Abrams Avatar answered Nov 08 '22 01:11

Ignacio Vazquez-Abrams