Ruby only creating 3 threads at a time

Question

I am trying to run 500 clients that send some request to the server simultaneously for load testing purpose. The client is a ruby program again. Sounds trivial. But I am facing weird problem with ruby threads. My code looks like this -

n = 10

n.times do
  Thread.new do
    `calc`
  end
end

The code is a sample. I'm just trying to run calc command from command line (If you are trying this code on platform other than windows please replace it with some command that works on your command line or shell). This will later be replaced by 'ruby my_client.rb' and also the value of n will be set to 500 (or whatever).

The problem that I am facing here is that regardless the number of threads I want to create, only 3 threads are created at a time. That is only 3 calc windows are opened simultaneously. The remaining threads just wait in a queue waiting for termination of these 3 threads. May be it has something to do with blocking and non-blocking calls. But I tried Java-equivalent of the same program and it worked perfectly. Its an old saying that threads in ruby are not recommended. Is it true that this is a problem with Ruby's threads or am I doing something wrong?

Answer 1

The problem you're observing is specific for GUI applications. It gets much better when you run command-line stuff inside workers.

With example below I can run 200 wget instances just fine, which is probably enough for your load testing goals.

n = 200

threads = []
(1..n).each do |i|
  threads << Thread.new do
    puts `wget google.com` # forgive me google
    sleep 10
    puts "#{i} done"
  end
end

threads.each do |t| # wait until all workers are done
  t.join
end

You could probably get many more workers if you switch from wget to Ruby code for fetching web pages. And still, you should remember that Ruby threads scales only that far. Don't expect many thousands or parallel threads to work fine -- try subprocesses or continuation-based approach instead.