Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Multithreaded ruby program only uses 100% cpu [duplicate]

I'm using ruby-head and Debian wheezy x64. When I run a multithreaded ruby script, htop shows that it's using multiple cores visually with the bars at the top, and that it's using 100% CPU on the process list but it's only using 100% of the capacity of one core. I assume it's possible to have multiple cores running at 100% and this number seems too convenient to be bottle-necked by either the program logic or another hardware aspect. Is the OS limiting the amount of available instructions I'm using, if so how do I stop this?

EDIT more info:

When I mean visually using multiple cores e.g.: 47% core 1, 29% core 2, and 24% core 3. These percentages are constantly shifting up and down and to different sets of cores, but always collectively add up to 100%-102%. More than 3(/8 total) cores are being used, but any cores other than the three most burdened only utilize 2% or less capacity. I guess I should also mention this is a linode VPS.

EDIT:

Well it looks like I was reading promises that 2.0 would feature true parallel threads, and not actual release information. Time to switch to Jruby...

like image 202
gtech Avatar asked Dec 09 '22 12:12

gtech


2 Answers

You failed to mention which Ruby implementation you are using. Not all Ruby implementations are capable of scheduling Ruby threads to multiple CPUs.

In particular:

  • MRI implements Ruby threads as green threads inside the interpreter and schedules them itself; it cannot schedule more than one thread at a time and it cannot schedule them to multiple CPUs
  • YARV implements Ruby threads as native OS threads (POSIX threads or Windows threads) and lets the OS schedule them, however it puts a Giant VM Lock (GVL) around them, so that only one Ruby thread can be running at any given time
  • Rubinius implements Ruby threads as native OS threads (POSIX threads or Windows threads) and lets the OS schedule them, however it puts a Global Interpreter Lock (GIL) around them, so that only one Ruby thread can be running at any given time; Rubinius 2.0 is going to have fine-grained locks so that multiple Ruby threads can run at any given time
  • JRuby implements Ruby threads as JVM threads and uses fine-grained locking so that multiple threads can be running; however, whether or not those threads are scheduled to multiple CPUs depends on the JVM being used, some allow this, some don't
  • IronRuby implements Ruby threads as CLI threads and uses fine-grained locking so that multiple threads can be running; however, whether or not those threads are scheduled to multiple CPUs depends on the VES being used, some allow this, some don't
  • MacRuby implements Ruby threads as native OS threads and uses fine-grained locking so that multiple threads can be running on multiple CPUs at the same time

I don't know enough about Topaz, Cardinal, MagLev, MRuby and all the others.

like image 135
Jörg W Mittag Avatar answered Dec 11 '22 11:12

Jörg W Mittag


MRI implements Ruby Threads as Green Threads within its interpreter. Unfortunately, it doesn't allow those threads to be scheduled in parallel, they can only run one thread at a time.

See similar question here

like image 38
rudolph9 Avatar answered Dec 11 '22 11:12

rudolph9