Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Index creation taking forever on postgres

I was trying to create an index to an integer column using a btree index, but it was taking forever (more than 2 hours!). The table has 17.514.879 lines. I didn't expect it to take that long.

After almost 2.5 hours, the connection to the database just died. When I reconnected to it, the index was there, but I don't know how good is this index.

How can I be sure the index was not messed up by the connection lost?

like image 810
Alexander Rumanovsk Avatar asked Dec 28 '16 17:12

Alexander Rumanovsk


People also ask

How long does it take to build index Postgres?

The index creation takes about 2 hours, which is really not useful. Is there any way I could speed up the index creation? Probably it's not better to set the index before data import, as this will slow down the import?

How long does it take to create an index on a large table PostgreSQL?

I followed this guide to pick the parameters: https://www.cybertec-postgresql.com/en/postgresql-parallel-create-index-for-better-performance/. This sped up the index creation and it only took 35 hours. Shaving off 10 hours was a nice surprise. Thanks!

How long does it take to add an index?

If you are just adding the single index, it should take about 10 minutes. However, it will take 100 minutes or more if you don't have that index file in memory.

How can I make my index faster?

Go to Control Panel | Indexing Options to monitor the indexing. The DisableBackOff = 1 option makes the indexing go faster than the default value. You can continue to work on the computer but indexing will continue in the background and is less likely to pause when other programs are running.


1 Answers

How to Check Whether the Index is Fine

Connect to the database via psql and run \d table_name (where table_name is the name of your table). For example:

grn=# \d users
            Table "public.users"
 Column |          Type          | Modifiers
--------+------------------------+-----------
 name   | character varying(255) |
Indexes:
    "users_name_idx" btree (name)

You'll see indexes listed below the table schema. If the index is corrupt it'll be marked as so.

How to Create Indexes Without Locking the Whole Table

You can create an index in a way that doesn't lock the whole table but is even slower. To do so you need to add CONCURRENTLY to CREATE INDEX. For example:

CREATE INDEX CONCURRENTLY users_name_idx ON users(name);

How to Fix a Corrupt Index

If the index is corrupt you can either drop it and recreate CONCURRENTLY or use REINDEX INDEX index_name. For example:

REINDEX INDEX users_name_idx

will recreate the users_name_idx.

like image 173
Greg Navis Avatar answered Oct 20 '22 06:10

Greg Navis