Throttling CPU/Memory usage of a Thread in Java?

Question

I'm writing an application that will have multiple threads running, and want to throttle the CPU/memory usage of those threads.

There is a similar question for C++, but I want to try and avoid using C++ and JNI if possible. I realize this might not be possible using a higher level language, but I'm curious to see if anyone has any ideas.

EDIT: Added a bounty; I'd like some really good, well thought out ideas on this.

EDIT 2: The situation I need this for is executing other people's code on my server. Basically it is completely arbitrary code, with the only guarantee being that there will be a main method on the class file. Currently, multiple completely disparate classes, which are loaded in at runtime, are executing concurrently as separate threads.

The way it's written, it would be a pain to refactor to create separate processes for each class that gets executed. If that's the only good way to limit memory usage via the VM arguments, then so be it. But I'd like to know if there's a way to do it with threads. Even as a separate process, I'd like to be able to somehow limit its CPU usage, since as I mentioned earlier, several of these will be executing at once. I don't want an infinite loop to hog up all the resources.

EDIT 3: An easy way to approximate object size is with java's Instrumentation classes; specifically, the getObjectSize method. Note that there is some special setup needed to use this tool.

Answer 1

If I understand your problem, one way would be to adaptively sleep the threads, similarly as video playback is done in Java. If you know you want 50% core utilization, the your algorithm should sleep approximately 0.5 seconds - potentially distributed within a second (e.g. 0.25 sec computation, 0.25 sec sleep, e.t.c.). Here is an example from my video player.

long starttime = 0; // variable declared //... // for the first time, remember the timestamp if (frameCount == 0) {     starttime = System.currentTimeMillis(); } // the next timestamp we want to wake up starttime += (1000.0 / fps); // Wait until the desired next time arrives using nanosecond // accuracy timer (wait(time) isn't accurate enough on most platforms)  LockSupport.parkNanos((long)(Math.max(0,      starttime - System.currentTimeMillis()) * 1000000)); 

This code will sleep based on the frames/second value.

To throttle the memory usage, you could wrap your object creation into a factory method, and use some kind of semaphore with a limited permits as bytes to limit the total estimated object size (you need to estimate the size of various objects to ration the semaphore).

package concur;  import java.util.Random; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; import java.util.concurrent.Semaphore; import java.util.concurrent.TimeUnit;  public class MemoryLimited {     private static Semaphore semaphore = new Semaphore(1024 * 1024, true);     // acquire method to get a size length array     public static byte[] createArray(int size) throws InterruptedException {         // ask the semaphore for the amount of memory         semaphore.acquire(size);         // if we get here we got the requested memory reserved         return new byte[size];     }     public static void releaseArray(byte[] array) {         // we don't need the memory of array, release         semaphore.release(array.length);     }     // allocation size, if N > 1M then there will be mutual exclusion     static final int N = 600000;     // the test program     public static void main(String[] args) {         // create 2 threaded executor for the demonstration         ExecutorService exec = Executors.newFixedThreadPool(2);         // what we want to run for allocation testion         Runnable run = new Runnable() {             @Override             public void run() {                 Random rnd = new Random();                 // do it 10 times to be sure we get the desired effect                 for (int i = 0; i < 10; i++) {                     try {                         // sleep randomly to achieve thread interleaving                         TimeUnit.MILLISECONDS.sleep(rnd.nextInt(100) * 10);                         // ask for N bytes of memory                         byte[] array = createArray(N);                         // print current memory occupation log                         System.out.printf("%s %d: %s (%d)%n",                             Thread.currentThread().getName(),                             System.currentTimeMillis(), array,                             semaphore.availablePermits());                         // wait some more for the next thread interleaving                         TimeUnit.MILLISECONDS.sleep(rnd.nextInt(100) * 10);                         // release memory, no longer needed                         releaseArray(array);                     } catch (InterruptedException e) {                         e.printStackTrace();                     }                 }             }         };         // run first task         exec.submit(run);         // run second task         exec.submit(run);         // let the executor exit when it has finished processing the runnables         exec.shutdown();     } } 

Answer 2

Care of Java Forums. Basically timing your execution and then waiting when your taking too much time. As is mentioned in the original thread, running this in a separate thread and interrupting the work thread will give more accurate results, as will averaging values over time.

import java.lang.management.*;  ThreadMXBean TMB = ManagementFactory.getThreadMXBean(); long time = new Date().getTime() * 1000000; long cput = 0; double cpuperc = -1;  while(true){  if( TMB.isThreadCpuTimeSupported() ){     if(new Date().getTime() * 1000000 - time > 1000000000){ //Reset once per second         time = new Date().getTime() * 1000000;         cput = TMB.getCurrentThreadCpuTime();     }      if(!TMB.isThreadCpuTimeEnabled()){         TMB.setThreadCpuTimeEnabled(true);     }      if(new Date().getTime() * 1000000 - time != 0)         cpuperc = (TMB.getCurrentThreadCpuTime() - cput) / (new Date().getTime() *  1000000.0 - time) * 100.0;                       } //If cpu usage is greater then 50% if(cpuperc > 50.0){      //sleep for a little bit.      continue; } //Do cpu intensive stuff }