Benchmarking Java, Groovy, Jython and Python

Question

I'm trying to benchmark a Monte Carlo calculation of PI (3.14159) throwing darts. I've implemented my code in Java, Groovy, BeanShell, Julia, Jython and Python (Python2 implemented in C).

Here is my original Java code "MonteCarloPI.java":

import java.util.Random; 

public class MonteCarloPI {
     public static void main(String[] args)
       {
         int nThrows = 0;
         int nSuccess = 0;
         double x, y;
         long then = System.nanoTime();
         int events=(int)1e8;
         Random r = new Random(); 
         for (int i = 0; i < events; i++) {
            x = r.nextFloat();      // Throw a dart
            y = r.nextFloat();
            nThrows++;
            if ( x*x + y*y <= 1 )  nSuccess++;
       }
 double itime = ((System.nanoTime() - then)/1e9);
 System.out.println("Time for calculations (sec): " + itime+"\n");
 System.out.println("Pi = " + 4*(double)nSuccess/(double)nThrows +"\n");
      }
}

Here is my Groovy code in the file "MonteCarloPI.groovy":

import java.util.Random

int nThrows = 0
int nSuccess = 0
double x, y
long then = System.nanoTime()
int events=1e8
r = new Random()
for (int i = 0; i < events; i++) {
            x = r.nextFloat()      // Throw a dart
            y = r.nextFloat()
            nThrows++
            if ( x*x + y*y <= 1 )  nSuccess++
}
itime = ((System.nanoTime() - then)/1e9)
System.out.println("Time for calculations (sec): " + itime+"\n")
System.out.println("Pi = " + 4*(double)nSuccess/(double)nThrows +"\n")
       

Alternatively, I've removed the definitions such as "float" and "int" (i.e. loose types). This checks the performance using "loose" types.

I've renamed "MonteCarloPI.groovy" to a BeanShell script file "MonteCarloPI.bsh" (BeanShell has a very similar syntax as Groovy)

In the case of the standard Python language, the code "MonteCarloPI_CPython.py" looks as this:

import random,time

nThrows,nSuccess = 0,0
then = time.time()
events=int(1e8)
for i in xrange(events):
   x,y = random.random(),random.random();   # Throw a dart                   
   nThrows +=1
   if ( x*x + y*y <= 1 ):  nSuccess+=1
itime = time.time() - then
print ("Time: ",itime,"sec Pi = ",4*nSuccess/float(nThrows))

This code is executed either in CPython 2.7.18 (Python implemented in C) or Jython 2.7.2 (Java implementation). For Python 3.8.3 ("Python3"), replace "xrange" with "range".

I also implemented the same algorithm in JRuby (MonteCarloPI.rb):

require "java"
java_import java.lang.System;
java_import java.util.Random;

nThrows = 0; nSuccess = 0;
xthen = System.nanoTime();
events=1e8;
r = Random.new();
for i  in 0 .. events do
  x = r.nextFloat();      #  Throw a dart
  y = r.nextFloat();
  nThrows +=1
   if ( x*x + y*y <= 1 )
                nSuccess += 1
  end
end
itime = (System.nanoTime() - xthen)/1e9;
xpi=(4.0*nSuccess)/nThrows
puts "Time for calculations (sec):  #{itime}"
puts "Pi = #{xpi}"

Here the code using Julia:

using Random
nThrows = 0
nSuccess = 0
events=1e8
then = time()
for j in 0:events
        x = rand();      #  Throw a dart
        y = rand();
        global  nThrows += 1;
        if  x*x + y*y <= 1
                 global nSuccess += 1;
        end
end
itime = time() - then
println( "Time for calculations (sec):", itime, " sec")
println( "Pi = ", 4.0*nSuccess/float(nThrows) )

I ran "MonteCarloPI.java", "MonteCarloPI.groovy", "MonteCarloPI.py", "MonteCarloPI.bsh" and MonteCarloPI.rb inside DataMelt editor. The julia code was processed using locally installed julia-1.5.0/bin.

Here is the benchmark results on Intel(R) Core(TM) i5-4690K CPU @ 3.50GHz (ubuntu 20.04, 8 GB memory), with 2048 MB allocated for JDK9 when running Groovy, Jython, BeanShell code:

Java   code:   1.7 sec Pi = 3.14176584  -> executed in DataMelt/JDK9
Groovy code:   2.1 sec Pi = 3.14144832  -> executed in DataMelt/JDK9
Groovy code:   18 sec Pi = 3.14141132  -> same but with "loose" types 
Julia code:    15 sec Pi = 3.14156104  -> executed in julia-1.5.0
Python code:   24 sec Pi = 3.14188036  -> executed in CPython 2.7.18
Python code:   30 sec Pi = 3.14188230  -> executed in CPython 3.2.8
Python code:    3 sec Pi = 3.14188036  -> executed using PyPy
Jython code:   24 sec Pi = 3.14187860  -> executed in DataMelt/JDK9
JRuby  code:   25 sec Pi = 3.14187860  -> executed in DataMelt/JDK9
BeanShell code: takes forever?!       -> executed in DataMelt/JDK9

As you can see, Java and Groovy calculations take about the same time (about 2 sec). With loose types in Groovy, the execution takes 9 times slower. Python is a factor 12 slower than Java and Groovy. Python3 is even slower. JRuby is as slow as Python. PyPy is rather fast (but slower than Java/Groovy). But BeanShell cannot do this calculation at all (takes forever, and my computer never stops processing this file).

Any wisdom on this?

Answer 1

Nice work. Interesting comparison you have got there. As a python developer I would like to add some additional view on Python.

I assume it is slower mostly because of dynamic typing. Another reason is that you are computing scalar values (i.e. using for loop and computing one number at a time). One of the advantages of Python is a vector computing using NumPy library (this allows to compute multiple numbers at the same time). So, here is my implementation of the algorithm. Note: I am using python 3.6.

import numpy as np
import time

start = time.time()

events = int(1e8)
nThrows, nSuccess = 0, 0

x, y = np.random.uniform(size=(2, events))
nSuccess = (x*x + y*y <= 1).sum()
nThrows = events
pi = 4*nSuccess/float(nThrows)

stop = time.time()
print('Time: {}, Pi = {}'.format(stop-start, pi))

Here are benchmark results on my i7 x64 computer (Windows 10):

Python (original code):      42.6s  Pi = 3.1414672
Python (my optimized code):  4.7s   Pi = 3.1417642

As you can see, the original python code run on my computer is slower than the python code on your computer. So, the optimized version might be even faster than Java or Groovy.

Hope this helps.

Answer 2

For the Julia code, you are including compilation time in your benchmark. The other thing to note is that you are using global variables, which is well known to kill performance. With your baseline version, the execution time on my machine is 17.7 seconds. Moving everything into a function, I get down to 0.83 seconds. Removing compilation time from this gets me down to 713.625 ms. The final version of my code is this (note you counted one too many in your loop).

using Random
using BenchmarkTools

function benchmark()
    nThrows = 0
    nSuccess = 0
    events = 100_000_000
    for j in 1:events
            x = rand()      #  Throw a dart
            y = rand()
            nThrows += 1
            if  x^2 + y^2 <= 1
                nSuccess += 1
            end
    end
    4.0*nSuccess/nThrows
end

pi = @btime benchmark()
println( "Pi = ",  pi)

Note that further improvements are possible. It could be beneficial to allocate an array of random numbers outside of the loop rather than calling rand twice per iteration. You can find other performance tips here: https://docs.julialang.org/en/v1/manual/performance-tips/