(notice) child pid XXXX exit signal Segmentation fault (11), possible coredump in /etc/apache2

Question

I keep getting the follow error in my Apache log:

[Wed Sep 18 17:59:20 2013] [notice] Apache/2.2.22 (Ubuntu) PHP/5.3.10-1ubuntu3.8 with Suhosin-Patch configured -- resuming normal operations
[Wed Sep 18 18:06:30 2013] [notice] child pid 7505 exit signal Segmentation fault (11), possible coredump in /etc/apache2
[Wed Sep 18 18:06:35 2013] [notice] child pid 7497 exit signal Segmentation fault (11), possible coredump in /etc/apache2
[Wed Sep 18 18:13:53 2013] [notice] child pid 7501 exit signal Segmentation fault (11), possible coredump in /etc/apache2
[Wed Sep 18 18:13:53 2013] [notice] child pid 7506 exit signal Segmentation fault (11), possible coredump in /etc/apache2
[Wed Sep 18 18:14:14 2013] [notice] child pid 8708 exit signal Segmentation fault (11), possible coredump in /etc/apache2

I've tried to backtrace it by doing the following:

user:~$ sudo gdb
user     8670  8571  0 18:12 pts/3    00:00:00 grep --color=auto httpd
user:~$ sudo gdb

(gdb) attach 8571
Attaching to process 8571
Reading symbols from /bin/bash...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libtinfo.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libtinfo.so.5 
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libdl.so.2
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib/x86_64-linux-gnu/libnss_compat.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libnss_compat.so.2
Reading symbols from /lib/x86_64-linux-gnu/libnsl.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libnsl.so.1
Reading symbols from /lib/x86_64-linux-gnu/libnss_nis.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libnss_nis.so.2
Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...(no debugging symbols      found)...done.
Loaded symbols for /lib/x86_64-linux-gnu/libnss_files.so.2
0x00007f553000244e in waitpid () from /lib/x86_64-linux-gnu/libc.so.6

(gdb) backtrace
#0  0x00007f553000244e in waitpid () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x0000000000441419 in ?? ()
#2  0x000000000044255c in wait_for ()
#3  0x0000000000432c88 in execute_command_internal ()
#4  0x00000000004352fe in execute_command ()
#5  0x000000000041e31d in reader_loop ()
#6  0x000000000041ca87 in main ()

(gdb) backtrace full
#0  0x00007f553000244e in waitpid () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#1  0x0000000000441419 in ?? ()
No symbol table info available.
#2  0x000000000044255c in wait_for ()
No symbol table info available.
#3  0x0000000000432c88 in execute_command_internal ()
No symbol table info available.
#4  0x00000000004352fe in execute_command ()
No symbol table info available.
#5  0x000000000041e31d in reader_loop ()
No symbol table info available.
#6  0x000000000041ca87 in main ()
No symbol table info available.`

I can't make head or tale of the problem.

I've also run gdb on apache as follows:

user:~$ sudo gdb apache2
Reading symbols from /usr/sbin/apache2...(no debugging symbols found)...done.

(gdb) run
Starting program: /usr/sbin/apache2
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
apache2: bad user name ${APACHE_RUN_USER}
[Inferior 1 (process 6925) exited with code 01]

I don't know if it's related to this issue but as soon as I installed gdb the following message is shown when I log in:

=> There were exceptions while processing one or more plugins. See
 /var/log/landscape/sysinfo.log for more information.

sysinfo.log contains the following:

for process_info in info.get_all_process_info():
File "/usr/lib/python2.7/dist-packages/landscape/lib/process.py", line 49, in get_all_process_info
process_info = self.get_process_info(process_id)
File "/usr/lib/python2.7/dist-packages/landscape/lib/process.py", line 85, in get_process_info
process_info["state"] = STATES[state]
KeyError: 't (tracing stop)'
2013-09-18 18:43:35,633 ERROR    Processes plugin raised an exception.
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/landscape/sysinfo/sysinfo.py", line 99, in run
result = plugin.run()
File "/usr/lib/python2.7/dist-packages/landscape/sysinfo/processes.py", line 18, in run
for process_info in info.get_all_process_info():
File "/usr/lib/python2.7/dist-packages/landscape/lib/process.py", line 49, in get_all_process_info
process_info = self.get_process_info(process_id)
File "/usr/lib/python2.7/dist-packages/landscape/lib/process.py", line 85, in get_process_info
process_info["state"] = STATES[state]
KeyError: 't (tracing stop)'

Some background.

I'm running a WordPress site off my VPS. The VPS is an Ubuntu based LAMP server with PERL and CURL installed. I use APC for caching but my segmentation faults occured before I installed APC. Finally I run my server throught Google PageSpeed Service so I have install the mod_remoteip mod for Apache 2.2 and have an X-Forwarded-For header in place.

ulimit core is unlimited. My phpinfo() file can be found here: http://tecne.ws/11v

Please assist. It would be much appreciated!

Answer 1

I had this issue and changed LogLevel warn to LogLevel debug in the apache config. On restart it seemed to be segfaulting just after mod_deflate.

Turning off mod_deflate in debian/ubuntu should just be sudo a2dismod deflate

Answer 2

This problem is often caused by an Apache module. As you can see, the other answers are about a module or another. The generic answer could be try to disable the last module(s) you enabled.

In my case, the module causing the problem is php7.3, on Ubuntu 18.04. PHP 7.2 works, but not PHP 7.3.

Answer 3

I solved this indirectly. I put Nginx in front of Apache and I no longer have segmentation errors. Having Nginx in front of Apache is a better set up in my opinion. Varnish Cache may have also solved the problem.