How is it possible that kill -9 for a process on Linux has no effect?

Question

I'm writing a plugin to highlight text strings automatically as you visit a web site. It's like the highlight search results but automatic and for many words; it could be used for people with allergies to make words really stand out, for example, when they browse a food site.

But I have problem. When I try to close an empty, fresh FF window, it somehow blocks the whole process. When I kill the process, all the windows vanish, but the Firefox process stays alive (parent PID is 1, doesn't listen to any signals, has lots of resources open, still eats CPU, but won't budge).

So two questions:

1. How is it even possible for a process not to listen to kill -9 (neither as user nor as root)?

2. Is there anything I can do but a reboot?

[EDIT] This is the offending process:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND digulla  16688  4.3  4.2 784476 345464 pts/14  D    Mar28  75:02 /opt/firefox-3.0/firefox-bin 

Same with ps -ef | grep firefox

UID        PID  PPID  C STIME TTY          TIME CMD digulla  16688     1  4 Mar28 pts/14   01:15:02 /opt/firefox-3.0/firefox-bin 

It's the only process left. As you can see, it's not a zombie, it's running! It doesn't listen to kill -9, no matter if I kill by PID or name! If I try to connect with strace, then the strace also hangs and can't be killed. There is no output, either. My guess is that FF hangs in some kernel routine but which?

[EDIT2] Based on feedback by sigjuice:

ps axopid,comm,wchan 

can show you in which kernel routine a process hangs. In my case, the offending plugin was the Beagle Indexer (openSUSE 11.1). After disabling the plugin, FF was a quick and happy fox again.

Answer 1

As noted in comments to the OP, a process status (STAT) of D indicates that the process is in an "uninterruptible sleep" state. In real-world terms, this generally means that it's waiting on I/O and can't/won't do anything - including dying - until that I/O operation completes.

Processes in a D state will normally only be there for a fraction of a second before the operation completes and they return to R/S. In my experience, if a process gets stuck in D, it's most often trying to communicate with an unreachable NFS or other remote filesystem, trying to access a failing hard drive, or making use of some piece of hardware by way of a flaky device driver. In such cases, the only way to recover and allow the process to die is to either get the fs/drive/hardware back up and running so the I/O can complete or to give up and reboot the system. In the specific case of NFS, the mount may also eventually time out and return from the I/O operation (with a failure code), but this is dependent on the mount options and it's very common for NFS mounts to be set to wait forever.

This is distinct from a zombie process, which will have a status of Z.

Answer 2

Double-check that the parent-id is really 1. If not, and this is firefox, first try sudo killall -9 firefox-bin. After that, try killing the specific process IDs individually with sudo killall -9 [process-id].

How is it even possible for a process not to listen to kill -9 (neiter as user nor as root)?

If a process has gone <defunct> and then becomes a zombie with a parent of 1, you can't kill it manually; only init can. Zombie processes are already dead and gone - they've lost the ability to be killed as they are no longer processes, only a process table entry and its associated exit code, waiting to be collected. You need to kill the parent, and you can't kill init for obvious reasons.

But see here for more general information. A reboot will kill everything, naturally.