InnoDB: Error: space header page consists of zero bytes in data file ./ibdata1

Question

I had a perfectly fine wordpress setup, but today the server suddenly stopped loading my site. I logged in and restarted my centos 7 VPS. After the reboot it did not started the MariaDB. Here is what I found in my log:

141026 18:13:50 [Note] /usr/libexec/mysqld: Shutdown complete

141026 18:13:50 mysqld_safe mysqld from pid file /var/run/mariadb/mariadb.pid ended
141026 18:14:58 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
141026 18:14:58 InnoDB: The InnoDB memory heap is disabled
141026 18:14:58 InnoDB: Mutexes and rw_locks use GCC atomic builtins
141026 18:14:58 InnoDB: Compressed tables use zlib 1.2.7
141026 18:14:58 InnoDB: Using Linux native AIO
141026 18:14:58 InnoDB: Initializing buffer pool, size = 128.0M
141026 18:14:58 InnoDB: Completed initialization of buffer pool
InnoDB: Error: space header page consists of zero bytes in data file ./ibdata1
141026 18:14:58 InnoDB: Could not open or create data files.
141026 18:14:58 InnoDB: If you tried to add new data files, and it failed here,
141026 18:14:58 InnoDB: you should now edit innodb_data_file_path in my.cnf back
141026 18:14:58 InnoDB: to what it was, and remove the new ibdata files InnoDB created
141026 18:14:58 InnoDB: in this failed attempt. InnoDB only wrote those files full of
141026 18:14:58 InnoDB: zeros, but did not yet use them in any way. But be careful: do not
141026 18:14:58 InnoDB: remove old data files which contain your precious data!
141026 18:14:58 [ERROR] Plugin 'InnoDB' init function returned error.
141026 18:14:58 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
141026 18:14:58 [Note] Plugin 'FEEDBACK' is disabled.
141026 18:14:58 [ERROR] Unknown/unsupported storage engine: InnoDB
141026 18:14:58 [ERROR] Aborting

Can someone help me investigate what could be the problem? Is there a possibility now to backup my database without being able to start MariaDB?

Answer 1

Edit: copied my-innodb-heavy-4G.cnf to /etc/my.cnf.d and the server started fine. Probably caused by changing the binlog size.

---

It sounds like you changed the size of your InnoDB buffer, or it became corrupted. Possible that these changes were made some time ago and just took effect when you rebooted.

Definitely make a backup of your data in /var/lib/mysql first - preferably the entire directory.

cp -R /var/lib/mysql /var/lib/mysql-backup

With your data backed up, the best first step is to revert any changes that you made to the /etc/my.cnf file and restart.

If that doesn't do it, or you don't know what the size of your buffer used to be, the next possible option is to delete the InnoDB binlog files and let MariaDB/MySQL recreate them. The files are named /var/lib/mysql/ib_logfile{N} where {N} is a number.

rm -rf /var/lib/mysql/ib_logfile*

If that doesn't solve the problem try adding the following to your /etc/my.cnf (or whichever config file you use, backup first of course). It should go in the [innodb] section.

innodb_buffer_pool_size        = 256M
innodb_log_file_size           = 256M
innodb_thread_concurrency      = 16
innodb_flush_log_at_trx_commit = 2
innodb_flush_method            = normal

The trick is knowing what MariaDB is expecting. If you don't have success with the above try just the last line and set [innodb_flush_method].

Answer 2

If you encountered this problem when you installed MySQL initially, just remove ibdata and ib_logfile* files in your MySQL data directory, and then start MySQL again.

Good Luck!