I have my inotify watch limit set to 1024 (I think the default is 128?). Despite that, yeoman, Guard and Dropbox constantly fail, and tell me to up my inotify limit. Before doing so, I'd like to know what's consuming all my watches (I have very few files in my Dropbox).
Is there some area of /proc or /sys, or some tool I can run, to find out what watches are currently registered?
Inotify Watch helps to keep track of the file changes under the directories on "watch" and report back to the application in a standard format using the API calls. We can monitor multiple file events under the watched directory using the API calls.
inotify (inode notify) is a Linux kernel subsystem created by John McCutchan, which monitors changes to the filesystem, and reports those changes to applications. It can be used to automatically update directory views, reload configuration files, log changes, backup, synchronize, and upload.
inotify filesystem options
sysctl fs.inotify
opened files
lsof | grep inotify | wc -l
Increase the values like this
sysctl -n -w fs.inotify.max_user_watches=16384
sysctl -n -w fs.inotify.max_user_instances=512
I already answered this in the same thread on Unix Stackexchange as was mentioned by @cincodenada, but thought I could repost my ready-made answer here, seeing that no one really has something that works:
I have a premade script, inotify-consumers
, that lists the top offenders for you (a newer version also lists the username owning the process, see below):
$ time inotify-consumers INOTIFY WATCHER COUNT PID CMD ---------------------------------------- 6688 27262 /home/dvlpr/apps/WebStorm-2018.3.4/WebStorm-183.5429.34/bin/fsnotifier64 411 27581 node /home/dvlpr/dev/kiwi-frontend/node_modules/.bin/webpack --config config/webpack.dev.js 79 1541 /usr/lib/gnome-settings-daemon/gsd-xsettings 30 1664 /usr/lib/gvfs/gvfsd-trash --spawner :1.22 /org/gtk/gvfs/exec_spaw/0 14 1630 /usr/bin/gnome-software --gapplication-service .... 7489 WATCHERS TOTAL COUNT real 0m0.099s user 0m0.042s sys 0m0.062s
Here you quickly see why the default limit of 8K watchers is too little on a development machine, as just WebStorm instance quickly maxes this when encountering a node_modules
folder with thousands of folders. Add a webpack watcher to guarantee problems ...
Even though it was much faster than the other alternatives when I made it initially, Simon Matter added some speed enhancements for heavily loaded Big Iron Linux (hundreds of cores) that sped it up immensely, taking it down from ten minutes (!) to 15 seconds on his monster rig.
inotify-consumers --help
😊 To get it on your machine, just copy the contents of the script and put it somewhere in your $PATH
, like /usr/local/bin
. Alternatively, if you trust this stranger on the net, you can avoid copying it and pipe it into bash over http:
$ curl -s https://raw.githubusercontent.com/fatso83/dotfiles/master/utils/scripts/inotify-consumers | bash INOTIFY WATCHER COUNT PID USER COMMAND -------------------------------------- 3044 3933 myuser node /usr/local/bin/tsserver 2965 3941 myuser /usr/local/bin/node /home/myuser/.config/coc/extensions/node_modules/coc-tsserver/bin/tsserverForkStart /hom 979 3954 myuser /usr/local/bin/node /home/myuser/.config/coc/extensions/node_modules/coc-tsserver/node_modules/typescript/li 1 7473 myuser /usr/local/bin/node --no-warnings /home/myuser/dev/dotfiles/common-setup/vim/dotvim/plugged/coc.nvim/build/i 1 3899 myuser /usr/local/bin/node --no-warnings /home/myuser/dev/dotfiles/common-setup/vim/dotvim/plugged/coc.nvim/build/i 6990 WATCHERS TOTAL COUNT
For reference, the main content of the script is simply this (inspired by this answer)
find /proc/*/fd \ -lname anon_inode:inotify \ -printf '%hinfo/%f\n' 2>/dev/null \ \ | xargs grep -c '^inotify' \ | sort -n -t: -k2 -r
In case you are wondering how to increase the limits
$ inotify-consumers --limits Current limits ------------- fs.inotify.max_user_instances = 128 fs.inotify.max_user_watches = 524288 Changing settings permanently ----------------------------- echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf sudo sysctl -p # re-read config
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With