My own storage engine crashes because of a too small sort_buffer

Question

I am working on my own storage engine for MySQL. So far this storage engine works reliable and correct - but only for small (~100 MB) tables... For big tables, I get a segmentation fault, when I try to execute a query with an order by, so something like this will lead to a segfault:

select * from item order by i_author;

So I compiled MySQL in debug mode, and saw, that the there is now an assertion failure in the merge_buffers function in filesort.cc:

/* The following will fire if there is not enough space in sort_buffer */
DBUG_ASSERT(maxcount!=0);

Honestly I have no idea what I can change in my storage engine to make this error disappear. It first looked like I have to change the configuration paramater sort_buffer_size - but even setting this thing higher than the size of the table does change anything with this error.

Does anyone, who know how to write MySQL storage engines have any idea how to solve this?

Answer 1

There is a similar issue reported with the falcon storage engine. The comment in the bug was,

It seems that the problem occurs in filesort when you have row estimates that are seriously wrong.

Most likely there is a bug in your storage engine somewhere, but that is a hard thing to debug through stack overflow.