How to calculate run-time for a multi-threaded program?

Question

I am trying to test the performance (in terms of execution time) for my webcrawler but I am having trouble timing it due to multi-threading taking place.

My main class:

class WebCrawlerTest {
    //methods and variables etc
    WebCrawlerTest(List<String> websites){
    //
    }

    if(!started){
        startTime = System.currentTimeMillis();
        executor = Executors.newFixedThreadPool(32); //this is the value I'm tweaking
        started=true;
    }       
    for(String site : websites){
        executor.submit(webProcessor = new AllWebsiteProcessorTest(site, deepSearch));
    }                               
    executor.shutdown();
    //tried grabbing end time here with no luck

AllWebsiteProcessorTest class:

class AllWebsiteProcessorTest implements Runnable{
    //methods and var etc

    AllWebsiteProcessorTest(String site, boolean deepSearch) {
    }
    public void run() {         
        scanSingleWebsite(websites);
        for(String email:emails){
            System.out.print(email + ", ");
        }

private void scanSingleWebsite(String website){
    try {
        String url = website;
        Document document = Jsoup.connect(url).get();
        grabEmails(document.toString());
    }catch (Exception e) {}

With another class (with a main method), I create an instance of WebCrawlerTest and then pass in an array of websites. The crawler works fine but I can't seem to figure out how to time it.

I can get the start time (System.getCurrentTime...();), but the problem is the end time. I've tried adding the end time like this:

//another class 
public static void main(.....){
    long start = getCurrent....();
    WebCrawlerTest w = new WebCrawlerTest(listOfSites, true);
    long end = getCurrent....();
}

Which doesn't work. I also tried adding the end after executor.shutdown(), which again doesn't work (instantly triggered). How do I grab the time for the final completed thread?

Answer 1

After shutting down your executors pool

executor.shutdown();
//tried grabbing end time here with no luck

You can simply

executor.awaitTermination(TimeUnit, value)

This call will block untill all tasks are completed. Take the time, subtract T0 from it and voila, we have execution time.

shutdown() method just assures that no new tasks will be accepted into excution queue. Tasks already in the queue will be performed (shutdownNow() drops pending tasks). To wait for all currently running tasks to complete, you have to awaitTermination().