Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

MySQL performance of adding a column to a large table

Tags:

I have MySQL 5.5.37 with InnoDB installed locally with apt-get on Ubuntu 13.10. My machine is i7-3770 + 32Gb memory + SSD hard drive on my desktop. For a table "mytable" which contains only 1.5 million records the following DDL query takes more than 20 min (!):

ALTER TABLE mytable ADD some_column CHAR(1) NOT NULL DEFAULT 'N'; 

Is there a way to improve it? I checked

show processlist; 

and it was showing that it is copying my table for some reason. It is disturbingly inconvenient. Is there a way to turn off this copy? Are there other ways to improve performance of adding a column to a large table?

Other than that my DB is relatively small with only 1.3Gb dump size. Therefore it should (in theory) fit 100% in memory.

Are there settings which can help? Would migration to Precona change anything for me?

Add: I have

innodb_buffer_pool_size = 134217728 
like image 258
Artem Avatar asked Jul 09 '14 17:07

Artem


People also ask

Does number of columns affect performance in MySQL?

Does the number of columns on a table have impact on performance? Yes, more number of columns, more metadata is needed. We suggest to use least number of columns needed, ideally < 100 columns on a table if possible.

How big is too big for a MySQL table?

There are some fundamental limits on table sizes. You can't have more than 1000 columns. Your records can't be bigger than 8k each. These limits change depending on database engine.

Can we add column to the existing table in MySQL?

Adding and Dropping ColumnsUse ADD to add new columns to a table, and DROP to remove existing columns. DROP col_name is a MySQL extension to standard SQL. To add a column at a specific position within a table row, use FIRST or AFTER col_name . The default is to add the column last.


1 Answers

Are there other ways to improve performance of adding a column to a large table?

Short answer: no. You may add ENUM and SET values instantly, and you may add secondary indexes while locking only for writes, but altering table structure always requires a table copy.

Long answer: your real problem isn't really performance, but the lock time. It doesn't matter if it's slow, it only matters that other clients can't perform queries until your ALTER TABLE is finished. There are some options in that case:

  1. You may use the pt-online-schema-change, from Percona toolkit. Backup your data first! This is the easiest solution, but may not work in all cases.

  2. If you don't use foreign keys and it's slow because you have a lot of indexes, it might be faster for you to create a copy of the table with the changes you need but no secondary indexes, populate it with the data, and create all indexes with a single alter table at the end.

  3. If it's easy for you to create replicas, like if you're hosted at Amazon RDS, you may create a master-master replica, run the alter table there, let it get back in sync, and switch instances after finished.

UPDATE

As others mentioned, MySQL 8.0 INNODB added support for instant column adds. It's not a magical solution, it has limitations and side-effects -- it can only be the last column, the table must not have a full text index, etc -- but should help in many cases.

You can specify explicit ALGORITHM=INSTANT LOCK=NONE parameters, and if an instant schema change isn't possible, MySQL will fail with an error instead of falling back to INPLACE or COPY. Example:

ALTER TABLE mytable ADD COLUMN mycolumn varchar(36) DEFAULT NULL, ALGORITHM=INPLACE, LOCK=NONE; 

https://mysqlserverteam.com/mysql-8-0-innodb-now-supports-instant-add-column/

like image 96
Pedro Werneck Avatar answered Oct 20 '22 05:10

Pedro Werneck