Error while trying to retrieve expired object from memcached

Question

I'm trying to learn how to use the memcached plugin in MySQL 5.6. I used telnet to get and set values to memcached. When I try to get an expired value from memcache, it says "Connection closed by foreign host.". More details are below:

database name: test

table name: demo

my table description

+-------+---------------------+------+-----+---------+-------+
| Field | Type                | Null | Key | Default | Extra |
+-------+---------------------+------+-----+---------+-------+
| c1    | varchar(32)         | NO   | PRI | NULL    |       |
| c21   | varchar(1024)       | YES  |     | NULL    |       |
| c22   | varchar(1024)       | YES  |     | NULL    |       |
| c3    | int(11)             | YES  |     | NULL    |       |
| c4    | bigint(20) unsigned | YES  |     | NULL    |       |
| c5    | int(11)             | YES  |     | NULL    |       |
+-------+---------------------+------+-----+---------+-------+

my table contents

+----+------+------+------+------+------------+
| c1 | c21  | c22  | c3   | c4   | c5         |
+----+------+------+------+------+------------+
| a  | aaaa | aaaa |    0 |    1 |          0 |
| b  | aa   | aa   |    0 |    2 | 1390372227 |
+----+------+------+------+------+------------+

containers table

+------+-----------+-----------+-------------+---------------+-------+------------+--------------------+------------------------+
| name | db_schema | db_table  | key_columns | value_columns | flags | cas_column | expire_time_column | unique_idx_name_on_key |
+------+-----------+-----------+-------------+---------------+-------+------------+--------------------+------------------------+
| aaa  | test      | demo_test | c1          | c2            | c3    | c4         | c5                 | PRIMARY                |
| bb   | test      | demo      | c1          | c21,c22       | c3    | c4         | c5                 | PRIMARY                |
+------+-----------+-----------+-------------+---------------+-------+------------+--------------------+------------------------+

cache policies table

+--------------+-------------+-------------+---------------+--------------+
| policy_name  | get_policy  | set_policy  | delete_policy | flush_policy |
+--------------+-------------+-------------+---------------+--------------+
| cache_policy | innodb_only | innodb_only | innodb_only   | innodb_only  |
+--------------+-------------+-------------+---------------+--------------+

telnet connection

 ~ ⮀ telnet localhost 11211
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
get @@bb
VALUE @@bb 0 9
test/demo
END
get a
VALUE a 0 9
aaaa|aaaa
END
get c
END
get d
END
get b
Connection closed by foreign host.

There is no problem when I try to get a value not present in memcached, or when I try to get a value that's not expired. Also this happens only with tables having multiple value-columns( c21 and c22 here ). If I do the same for a table with only one value-column, there is no issue.

And after the connection is closed, if I try to query the database, the following error is shown:

mysql> select * from innodb_memcache.cache_policies;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id:    1
Current database: test

And sometimes it shows the following error:

*** glibc detected *** /opt/mysql/server-5.6/bin/mysqld: free(): invalid pointer: 0x00007f053c008827 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7f0596f39b96]
/opt/mysql/server-5.6/lib/plugin//innodb_engine.so(+0xb170)[0x7f0547df3170]
/opt/mysql/server-5.6/lib/plugin/libmemcached.so(+0x10bbb)[0x7f0569475bbb]
/opt/mysql/server-5.6/lib/plugin/libmemcached.so(+0x157b0)[0x7f056947a7b0]
/opt/mysql/server-5.6/lib/plugin/libmemcached.so(+0x16cec)[0x7f056947bcec]
/opt/mysql/server-5.6/lib/plugin/libmemcached.so(conn_parse_cmd+0x11)[0x7f056947c931]
/opt/mysql/server-5.6/lib/plugin/libmemcached.so(+0x9bdc)[0x7f056946ebdc]
/opt/mysql/server-5.6/lib/plugin/libmemcached.so(event_base_loop+0x286)[0x7f05694810d6]
/opt/mysql/server-5.6/lib/plugin/libmemcached.so(+0x19bb2)[0x7f056947ebb2]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7f05982dbe9a]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f0596faf3fd]
======= Memory map: ========
00400000-00f6f000 r-xp 00000000 08:03 21495897                           /opt/mysql/server-5.6/bin/mysqld
0116f000-0131a000 rw-p 00b6f000 08:03 21495897                           /opt/mysql/server-5.6/bin/mysqld
0131a000-0136d000 rw-p 00000000 00:00 0 
01a44000-021a0000 rw-p 00000000 00:00 0                                  [heap]
7f0534000000-7f0534021000 rw-p 00000000 00:00 0 
7f0534021000-7f0538000000 ---p 00000000 00:00 0 
7f053c000000-7f053c021000 rw-p 00000000 00:00 0 
7f053c021000-7f0540000000 ---p 00000000 00:00 0 
7f0540000000-7f05402dd000 rw-p 00000000 00:00 0 
7f05402dd000-7f0544000000 ---p 00000000 00:00 0 
7f05445e1000-7f05445e2000 ---p 00000000 00:00 0 
7f05445e2000-7f0544de2000 rw-p 00000000 00:00 0                          [stack:30166]
7f0544de2000-7f0544de3000 ---p 00000000 00:00 0 
7f0544de3000-7f05455e3000 rw-p 00000000 00:00 0                          [stack:30160]
7f05455e3000-7f05455e4000 ---p 00000000 00:00 0 
7f05455e4000-7f0545de4000 rw-p 00000000 00:00 0                          [stack:30161]
7f0545de4000-7f0545de5000 ---p 00000000 00:00 0 
7f0545de5000-7f05465e5000 rw-p 00000000 00:00 0                          [stack:30162]
7f05465e5000-7f05465e6000 ---p 00000000 00:00 0 
7f05465e6000-7f0546de6000 rw-p 00000000 00:00 0                          [stack:30163]
7f0546de6000-7f0546de7000 ---p 00000000 00:00 0 
7f0546de7000-7f05475e7000 rw-p 00000000 00:00 0                          [stack:30165]
7f05475e7000-7f05475e8000 ---p 00000000 00:00 0 
7f05475e8000-7f0547de8000 rw-p 00000000 00:00 0                          [stack:30164]
7f0547de8000-7f0547e00000 r-xp 00000000 08:03 21505464                   /opt/mysql/server-5.6/lib/plugin/innodb_engine.so
7f0547e00000-7f0547fff000 ---p 00018000 08:03 21505464                   /opt/mysql/server-5.6/lib/plugin/innodb_engine.so
7f0547fff000-7f0548000000 rw-p 00017000 08:03 21505464                   /opt/mysql/server-5.6/lib/plugin/innodb_engine.so
7f0548000000-7f0548022000 rw-p 00000000 00:00 0 
7f0548022000-7f054c000000 ---p 00000000 00:00 0 
7f054c000000-7f054c021000 rw-p 00000000 00:00 0 
7f054c021000-7f0550000000 ---p 00000000 00:00 0 
7f0550000000-7f0550021000 rw-p 00000000 00:00 0 
7f0550021000-7f0554000000 ---p 00000000 00:00 0 
7f0554000000-7f0554021000 rw-p 00000000 00:00 0 
7f0554021000-7f0558000000 ---p 00000000 00:00 0 
7f0558000000-7f0558021000 rw-p 00000000 00:00 0 
7f0558021000-7f055c000000 ---p 00000000 00:00 0 
7f055c000000-7f055c07a000 rw-p 00000000 00:00 0 
7f055c07a000-7f0560000000 ---p 00000000 00:00 0 
7f0560000000-7f0560021000 rw-p 00000000 00:00 0 
7f0560021000-7f0564000000 ---p 00000000 00:00 0 
7f0564041000-7f0564042000 ---p 00000000 00:00 0 
7f0564042000-7f0564842000 rw-p 00000000 00:00 0 
7f0564842000-7f0564843000 ---p 00000000 00:00 0 
7f0564843000-7f0565043000 rw-p 00000000 00:00 0                          [stack:19202]
7f0565043000-7f0565044000 ---p 00000000 00:00 0 
7f0565044000-7f0565844000 rw-p 00000000 00:00 0                          [stack:19201]
7f0565844000-7f0565845000 ---p 00000000 00:00 0 
7f0565845000-7f0566045000 rw-p 00000000 00:00 0                          [stack:19200]
7f0566045000-7f0566046000 ---p 00000000 00:00 0 
7f0566046000-7f0566846000 rw-p 00000000 00:00 0                          [stack:19199]
7f0566846000-7f0566847000 ---p 00000000 00:00 0 
7f0566847000-7f0567047000 rw-p 00000000 00:00 0                          [stack:19198]
7f0567047000-7f0567048000 ---p 00000000 00:00 0 
7f0567048000-7f0567848000 rw-p 00000000 00:00 0                          [stack:19197]
7f0567848000-7f0567849000 ---p 00000000 00:00 0 
7f0567849000-7f0568049000 rw-p 00000000 00:00 0                          [stack:19196]
7f0568049000-7f056804a000 ---p 00000000 00:00 0 
7f056804a000-7f056884a000 rw-p 00000000 00:00 0                          [stack:19195]
7f056884a000-7f056884b000 ---p 00000000 00:00 0 
7f056884b000-7f05692d8000 rw-p 00000000 00:00 0                          [stack:19194]
7f0569465000-7f056948f000 r-xp 00000000 08:03 21505461                   /opt/mysql/server-5.6/lib/plugin/libmemcached.so
7f056948f000-7f056968f000 ---p 0002a000 08:03 21505461                   /opt/mysql/server-5.6/lib/plugin/libmemcached.so
7f056968f000-7f0569691000 rw-p 0002a000 08:03 21505461                   /opt/mysql/server-5.6/lib/plugin/libmemcached.so
7f0569691000-7f0569694000 rw-p 00000000 00:00 0 
7f0569694000-7f0569695000 ---p 00000000 00:00 0 
7f0569695000-7f05696c5000 rw-p 00000000 00:00 0                          [stack:20139]
7f05696c5000-7f05696c6000 ---p 00000000 00:00 0 
7f05696c6000-7f05696f6000 rw-p 00000000 00:00 0                          [stack:19203]
7f05696f6000-7f05696f7000 ---p 00000000 00:00 0 
7f05696f7000-7f0569ef7000 rw-p 00000000 00:00 0                          [stack:19192]
7f0569ef7000-7f0569ef8000 ---p 00000000 00:00 0 
7f0569ef8000-7f056a6f8000 rw-p 00000000 00:00 0                          [stack:19191]
7f056a6f8000-7f056a6f9000 ---p 00000000 00:00 0 
7f056a6f9000-7f056aef9000 rw-p 00000000 00:00 0                          [stack:19190]
7f056aef9000-7f056aefa000 ---p 00000000 00:00 0 
7f056aefa000-7f056b6fa000 rw-p 00000000 00:00 0                          [stack:19189]
7f056b6fa000-7f056b6fb000 ---p 00000000 00:00 0 
7f056b6fb000-7f056befb000 rw-p 00000000 00:00 0                          [stack:19188]
7f056befb000-7f056befc000 ---p 00000000 00:00 0 
7f056befc000-7f056c6fc000 rw-p 00000000 00:00 0                          [stack:19187]
7f056c6fc000-7f056c6fd000 ---p 00000000 00:00 0 
7f056c6fd000-7f056cefd000 rw-p 00000000 00:00 0                          [stack:19186]
7f056cefd000-7f056cefe000 ---p 00000000 00:00 0 
7f056cefe000-7f056d6fe000 rw-p 00000000 00:00 0                          [stack:19185]

Answer 1

This is a bug, your connection is being closed because of the server crash log you posted.

This issue is known and will hopefully be fixed in a future release, in the mean time you could avoid using the table prefix (@@bb) as that is the key to the issue.