mysqld Crashes with Signal 11 and Page Corrupt Error

Many MySQL users have reported encountering an issue where their MySQL Server crashes with the “mysqld got signal 11” error, along with some hexadecimal code. This is a run-time error that the operating system throws when there is a segmentation fault. You can see this error while using MySQL in Linux/Unix-based environments, like Ubuntu, CentOS, etc. It indicates that the mysqld service or MySQL Workbench failed when it tried to access an invalid memory segment. It can happen due to various reasons, such as:

• Bugs in the MySQL binary or linked libraries
• Misconfigured MySQL configuration file
• Corruption in system memory
• Hardware issues
• Corruption in InnoDB tables
• Corrupt index records or pages in the MySQL database
• Bugs in MySQL server plugins

In this article, we will discuss the troubleshooting methods to resolve the mysqld got signal 11 error in MySQL Server.

Methods to Resolve MySQL Server crashes with mysqld got Signal 11 Error

The mysqld got signal 11 error usually occurs due to bugs in the MySQL Server or its plugins, misconfigured configuration files, or corruption in data or indexes. You can follow the methods mentioned below to identify the root cause of the error and troubleshoot it effectively.

Method 1 – Use Stack Trace in the Error Log
The MySQL error log contains a stack trace that records complete details related to the mysqld crash. This trace helps identify the function calls that lead to the crash, detect failed operations, faulty MySQL plugins, bugs in code, and memory address issues in the binaries.
You can use the stack trace in the error log to find out the cause behind the mysqld crashes with signal 11. Here’s how to find the stack trace:

Note: Stack trace works only if you have not used the -fomit-frame-pointer option with GCC while compiling mysqld. To read the stack trace accurately, ensure MySQL is not compiled with this option.

• Locate the MySQL error log by following the below path:
/var/log/mysql/error.log
• In the error log, search for “Stack Trace”. It looks like:

mysqld got signal 11;
Attempting backtrace. You can use the following information
to find out where mysqld died. If you see no messages after
this, something went terribly wrong...

stack_bottom = 0x41fd0110 thread_stack = 0x40000
mysqld(my_print_stacktrace+0x32)[0x9da402]
mysqld(handle_segfault+0x28a)[0x6648e9]
/lib/libpthread.so.0[0x7f1a5af000f0]
/lib/libc.so.6(strcmp+0x2)[0x7f1a5a10f0f2]
mysqld(_Z21check_change_passwordP3THDPKcS2_Pcj+0x7c)[0x7412cb]
mysqld(_ZN16set_var_password5checkEP3THD+0xd0)[0x688354]

• Read and understand the information in the stack trace carefully. Next, use the addr2line utility to decode addresses from the stack trace into human-readable language. This utility is available in the binutils package on Linux. For example, to decode mysqld(_Z21check_change_passwordP3THDPKcS2_Pcj+0x7c)[0x7412cb] using this utility, run the following command:

$> addr2line -fie /usr/sbin/mysqld 0x7412cb
check_change_password
mysql-server/sql/sql_acl.cc:1234

It will display the check_change_password function behind the mysqld crash, along with the exact line number in the code. Once you have found the exact function name, then try to fix the issue. For example, as displayed above, there is an issue in the check_change_password function. So, locate the MySQL source code in the MySQL data directory to fix it.

Method 2 - Enable InnoDB Recovery Mode and Use the Drop and Reload Method

The MySQL "got signal 11" error can also occur if corruption is present in pages, indexes, or InnoDB tables (.ibd or ibdata1) files. In such a case, you can use innodb_force_recovery to enable the recovery mode. Enabling this option skips the crash recovery and allows MySQL to start. Next, repair the InnoDB tables using the Drop and Reload method. Here’s how to do so:

_Note: Before proceeding, create a backup of the database_

• Enable InnoDB Recovery Mode to access inaccessible MySQL tables. For this, change the my.cnf configuration file. The configuration file is located at /var/log/mysql/error.log.
Alternatively, you can run the below command to go to the configuration file:
sudo nano /etc/mysql/my.cnf
• Add the below line under the [mysqld] section:
innodb_force_recovery=1
• You can increase the recovery level from 1-6. However, setting the value to 4 or above can cause data loss.
• After changes, save the file and restart MySQL.

Now, dump and reload the tables using the below steps:

• First, dump all the tables in the MySQL database.
mysqldump -u [username] -p [database_name] [table_name] > dump.sql
mysql -u [username] -p [database_name] < dump.sql
• Next, drop and recreate the table.
DROP TABLE [table_name];
CREATE TABLE [table_name] (...);
• Then, rebuild the table using the ALTER statement.
sql
ALTER TABLE [table_name] ENGINE=InnoDB;
• Next, disable the InnoDB recovery mode by using this line of code:
#innodb_force_recovery=…
• Save the changes made to the configuration file and then start the MySQL Server.

Repair MySQL Database using a Professional MySQL Repair Tool

If corruption in the database is too high, you can use a professional MySQL repair tool, like Stellar Repair for MySQL. It can repair corrupted tables created in InnoDB and MyISAM storage engines, with complete integrity. It can restore all the data from .ibd and .ibdata1 files, including indexes, tablespaces, and keys, without any modifications. The tool is compatible with Linux and Windows operating systems. It helps you resolve all the corruption-related errors, including fatal errors that occur due to database corruption. It can repair databases created in MySQL 8.0.36 and earlier versions.

Conclusion

You may face a situation where the MySQL Server crashes with the “mysqld got signal 11” error. It can occur due to various reasons. If the error has occurred due to corruption in InnoDB tablespace or data files, you can try starting MySQL in recovery mode using the innodb_force_recovery option. If the corruption is severe, you can use an advanced MySQL repair tool, like Stellar Repair for MySQL. It can repair corrupt MySQL database files with ease. It can recover all the data from damaged or corrupted MySQL database files.