Jstack and Not enough storage is available to process this command

Question

I'm trying to run jstack command on my java application. Application is rather big, running inside jboss AS occupying about 4gb of memory. OS is Windows Server 2003 Standard edition. Every time i get an error "Not enough storage is available to process this command". There is enough ram, 16gb, and disk space. So, any ideas?

Answer 1

I ran into this recently on Win2008r2 and thought I'd share my solution since it took a while to figure out. Rob's comment about psexec -s is what did it for me.

It appears that on Vista and later jstack doesn't work against services because of the user context. It has nothing to do with memory. I suspect this is the same reason people have seen this problem on 2003 via remote desktop, unless you use the /admin or /console switch on mstsc. As of Vista the tightened security is probably what broke it.

Starting my app from a cmd window worked fine, but that doesn't help me debug our standard install. Enabling the java debug port (for VisualVM, Eclipse or most any Java debugger) requires an app restart, so you lose the state you're probably trying to capture if you don't already have debugging enabled. Starting the service under my user credentials did not work - I was a little surprised at that. But psexec -s runs jstack from the system context, which worked like a charm. Oh, and you'll need to run psexec from an elevated cmd prompt, if UAC is on.

Answer 2

In the past I have seen this when the JVM is running as a Windows Service on Windows 2003.

First, check to see if this is an issue with the TMP directory.

Second, jstack (or the other utilities like jconsole) will not connect to the local process unless it is running in the same session. If the service is running as a specific user, you may be able to connect by logging into the same session. If you are using Remote Desktop, you can connect using "mstsc /admin" (used to be /console) and try to run jstack again. Definitely check to make sure the TMP directory is set properly if this doesn't fix the problem.

If the service is running as LocalSystem, the above procedure probably will not help much. I don't know if there is a way to log into the same session as LocalSystem.

Some other alternatives may be to set the process up for remote monitoring and use jvisualvm (from the server itself or another machine) to connect over a port and do a thread dump.