Why lsof report a higher open files number than ulimit's "open file" output

Question

Tomcat runs on my workstation for several days, now it has no response, lsof command outputs lots of close_wait state connections, tomcat pid is 25422, however the ulimit command shows that the "open file" is 1024, how can this happen?

[root@localhost home]# lsof -p 25422 | wc -l
10309

[root@localhost home]# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 399360
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 399360
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Answer 1

For open files, we have soft / hard open files limit on linux os.

If the soft limit is reached, it will just expand the limit to higher limit but under Hard limit.

By checking the hard limit, you can simply run:

# ulimit -Hn

Here is also an article that may help you understand more:

Guide to limits.conf / ulimit /open file descriptors under linux