Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

group_concat performance issue in MySQL

Tags:

sql

mysql

I added a group_concat to a query and killed the performance. The explain plans are identical before and after I added it, so I'm confused as to how to optimize this.

Here is a simplified version of the query:

SELECT @curRow := @curRow + 1 AS row_number,
docID,
docTypeID,
CASE WHEN COUNT(1) > 1
     THEN group_concat( makeID )
     -- THEN 'multiple makes found'
     ELSE MIN(makeID)
END AS makeID,
MIN(desc) AS desc
FROM simplified_mysql_table,
(SELECT @curRow := 0) r
GROUP BY docID, docTypeID,
CASE WHEN docTypeID = 1
     THEN 0
     ELSE row_number
END;

Note the CASE statement in the SELECT. The group_concat kills performance. If I comment that line and just output 'multiple makes found' it executes very quickly. Any idea what is causing this?

like image 452
Fook Avatar asked Nov 29 '12 17:11

Fook


People also ask

What does Group_concat do in MySQL?

The GROUP_CONCAT() function in MySQL is used to concatenate data from multiple rows into one field. This is an aggregate (GROUP BY) function which returns a String value, if the group contains at least one non-NULL value. Otherwise, it returns NULL.

Is there a length limit to Group_concat?

I'm using GROUP_CONCAT() in a MySQL query to convert multiple rows into a single string. However, the maximum length of the result of this function is 1024 characters.

Is concat slow in MySQL?

When running queries on a large table and using a function like CONCAT() in the WHERE clause, the queries can be much slower than when the function is not used. This is usually caused when the function is used on an indexed column.


2 Answers

In the original non-simplified version of this query we had a DISTINCT, which was completely unnecessary and causing the performance issue with group_concat. I'm not sure why it caused such a problem, but removing it fixed the performance issue.

like image 147
Fook Avatar answered Nov 05 '22 20:11

Fook


In MySQL, group_concat performance should not kill query performance. It is additional work involving strings, so some slow down is expected. But more like 10% rather than 10X. Can you quantify the difference in the query times?

Question: is MakeID a character string or integer? I wonder if a conversion from integer to string might affect the performance.

Second, what would the performance be for concat(min(MakeId), '-', max(MakedId)) isntead of the group_concat?

Third, does the real group_concat use DISTINCT or ORDER BY? These could slow things down, especially in a memory limited environment.

like image 20
Gordon Linoff Avatar answered Nov 05 '22 19:11

Gordon Linoff