Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Creation of GSI taking long time

I have a table with close to 2 billion rows already created in DynamoDB.

Due to a query requirement, I had to create a Global Secondary Index(GSI) in it. The process of GSI creation started 36 hours ago but still isn't completed. Portal shows Item Count to be around 100 million. So long way to go.

Questions:

  1. Why does it take such a long time when sufficient WCU and RCU are alotted( 30k in fact ).
  2. GSI partition key I've used is something whose values are repetitive, could that be the reason why GSI creation is taking more time (ideal scenario is that we select a partition key which doesn't repeat for items to span across multiple partitions).
  3. Is there a way to abort the creation of GSI while the process is on? it doesn't allow through AWS console.

Thanks.

like image 885
Dixit Gokhale Avatar asked Dec 21 '17 08:12

Dixit Gokhale


People also ask

How long does GSI take to create?

Wait until the global secondary index has been created. This takes approximately 5 minutes. Check output of the following command.

Can GSI be added later?

You can create a GSI on creation or you can add it later. Different partition key as well as different sort key. It gives a completely different view of the data.

Can GSI be deleted?

If you no longer need a global secondary index, you can delete it using the UpdateTable operation.

Can GSI be updated?

You can create a new GSI with a different name..


1 Answers

A GSI has its own WCUs and RCUs, distinct and separate to the primary index. Could this be because you dont have enough WCUs on your GSI?

If your global secondary index is taking too long to create (common when adding indexes on an existing large table), you can provision additional write capacity by following these steps:

Open the DynamoDB console.

From the navigation pane, choose Tables, and then select your table from the list.

Choose the Indexes tab.

Increase the write capacity of the index, and then choose Save.

After about a minute, check the OnlineIndexPercentageProgress metric from the Metrics tab to see if the creation of your global secondary index is progressing satisfactorily.

EDIT: Above from the AWS Knowledge Center

'OnlineIndexPercentageProgress' instructions:

Creation of your global secondary index will begin. You can monitor the progress on the Metrics tab:

Choose the Metrics tab.

Choose View all CloudWatch metrics.

In the CloudWatch console, choose DynamoDB. In the Search Metrics box, enter OnlineIndexPercentageProgress. Note: If the search returns an empty list, wait about a minute for metrics to populate.

Choose the name of the index to see the progress.

like image 111
F_SO_K Avatar answered Sep 21 '22 13:09

F_SO_K