I need to monitor the amount of memory consumed by threads spawned by my application. The idea is to take corrective actions, if a greedy thread consumes too much of memory. I have referred to How much memory does my java thread take?. One of the suggestions on that link is to use getThreadAllocatedBytes
in ThreadMXBean.
I experimented with getThreadAllocatedBytes
with the following job.
List<Long> primes = new ArrayList<Long>();
long i = 0;
while (true) {
primes.add(++i);
if ((i % 10) == 0) {
primes.clear();
System.runFinalization();
System.gc();
}
}
I run this job on four threads for considerable time. Though the job does not accumulate memory continuously, the values returned by getThreadAllocatedBytes
keeps increasing and does not go down even once. This implies that getThreadAllocatedBytes
does not return the actual amount of memory on heap used by the thread. It returns the total amount of memory allocated on the heap for the thread since it was started. My platform details are as follows:
Linux PG85213.egi.ericsson.com 3.5.0-030500-generic #201207211835 SMP Sat Jul 21 22:35:55 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
java version "1.7.0_45"
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)
Is the above behavior desired behavior of getThreadAllocatedBytes
?
If so, is there no way to find effective memory on heap used by a thread.
Am listing the complete program for reference:
package workbench;
import java.lang.management.ManagementFactory;
import com.sun.management.ThreadMXBean;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.Executors;
import java.util.logging.Level;
import java.util.logging.Logger;
public class AnotherWorkBench {
private static final CountDownLatch latch = new CountDownLatch(4);
static final List<Long> threadIds = Collections.synchronizedList(new ArrayList<Long>());
private void dummyJob() {
List<Long> primes = new ArrayList<Long>();
long i = 0;
while (true) {
primes.add(++i);
if ((i % 10) == 0) {
primes.clear();
//introduce sleep to prevent process hogging
try {
Thread.currentThread().sleep(2000);
} catch (InterruptedException ex) {
Logger.getLogger(AnotherWorkBench.class.getName()).log(Level.SEVERE, null, ex);
}
System.runFinalization();
System.gc();
}
}
}
private void runDummyJobs() {
Runnable dummyJob = new Runnable() {
@Override
public void run() {
threadIds.add(Thread.currentThread().getId());
latch.countDown();
dummyJob();
}
};
Runnable memoryMonitorJob = new Runnable() {
@Override
public void run() {
System.out.println(Thread.currentThread().getName() + " : Monitor thread started");
ThreadMXBean threadMxBean = (ThreadMXBean) ManagementFactory.getThreadMXBean();
threadMxBean.setThreadAllocatedMemoryEnabled(true);
while (true) {
for (Long threadId : threadIds) {
System.out.println(Thread.currentThread().getName() + " : Thread ID : " + threadId + " : memory = " + threadMxBean.getThreadAllocatedBytes(threadId) + " bytes");
}
//wait between subsequent scans
try {
System.out.println(Thread.currentThread().getName() + " : secondary sleep");
Thread.currentThread().sleep(5000);
System.out.println(Thread.currentThread().getName() + " : out of secondary sleep");
} catch (InterruptedException ex) {
Logger.getLogger(WorkBench.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
};
Executors.newSingleThreadExecutor().submit(dummyJob);
Executors.newSingleThreadExecutor().submit(dummyJob);
Executors.newSingleThreadExecutor().submit(dummyJob);
Executors.newSingleThreadExecutor().submit(dummyJob);
try {
latch.await();
} catch (InterruptedException ex) {
Logger.getLogger(AnotherWorkBench.class.getName()).log(Level.SEVERE, null, ex);
}
Executors.newSingleThreadExecutor().submit(memoryMonitorJob);
}
/**
* @param args the command line arguments
*/
public static void main(String[] args) {
new AnotherWorkBench().runDummyJobs();
}
}
We talked about the two types of memory available to a process or a thread, the stack and the heap. It is important to distinguish between these two types of process memory because each thread will have its own stack, but all the threads in a process will share the heap.
Be mindful of thread use and stack size. The default option -Xss512k means that each thread will use 512kb of memory. The JVM default without this option is 1MB.
1 thread = 1,000,000 bytes.
In other words, two threads can read from the same memory and access the same global variables. i.e. memory is shared because it's the same address space. Separate processes can't do that, they are independent of each other.
To my knowledge, there is no reliable way to do this at runtime. And as pointed out in the source question, the heap is a shared resource and thus the heap size of a single thread does not make sense as it will overlap with objects references from other threads.
That said, when I do want to know the 'retained' size of a single thread, and yes retained size is a different but similar metric to the one that you asked for, then I do it by taking a heap dump and then using MAT (http://www.eclipse.org/mat/).
I have known people to use Java Agents to instrument the allocation of objects and then to use a weak reference to monitor when it gets GC'd. However the performance impact of doing this is high. Very high.
You may be best off using a heuristic at runtime and unit testing to ensure that memory stays within bounds. For example, you could use JMX to monitor the heap sizes and when you see the old gen growing then you can raise an alert. Using getThreadAllocatedBytes to calculate rate of allocation could also be useful.
Good run time monitoring tools: appdynamics, newrelic, visualvm and yourkit
For offline memory analysis, mat and jclarity are very good.
A very useful tool to help one spot whether there is a leak, or at least is running different to expectations is to print a count of how many instances of each class are currently on the heap: jcmd <pid> GC.class_histogram.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With