Perl threads: How to make a producer?

Question

I have a function that is very slow to run. I need input from that function in the main part of the program. So I would like to do something similar to the UNIX command yes, that produces as much input as is read, but only a little more than is needed. Unlike yes I do not want the values from STDIN but I want the values in a Perl queue.

In other words: This problem is not about selecting on file-handles, but on queues maintained by threads.

I imagine the meta code will look similar to this:

my $DataQueue = Thread::Queue->new();

my @producers;
my $no_of_threads = 10;
for (1..$no_of_threads) {
  push @producers, threads->create(\&producer);
}

for(<>) {
  # This should block until there is a value to dequeue
  # Maybe dequeue blocks by default - then this part is not a problem
  my $val = $DataQueue->dequeue();
  do_something($_,$val);
}
# We are done: The producers are no longer needed
kill @producers;

sub producer {
   while(1) {
      # How do I wait until the queue length is smaller than number of threads?
      wait_until(length of $DataQueue < $no_of_threads);
      $DataQueue->enqueue(compute_slow_value());
   }
}

But is there a more elegant way of doing this? I am especially unsure of how to do the wait_until part in an efficient way.

Answer 1

Something like this will probably work:

my $DataQueue = Thread::Queue->new();

my @producers;
my $no_of_threads = 10;
for (1..$no_of_threads) {
  push @producers, threads->create(\&producer);
}
$DataQueue->limit = 2 * $no_of_threads;

for(<>) {
  # This blocks until $DataQueue->pending > 0
  my $val = $DataQueue->dequeue();
  do_something($_,$val);
}
# We are done: The producers are no longer needed
kill @producers;

sub producer {
   while(1) {
      # enqueue will block until $DataQueue->pending < $DataQueue->limit
      $DataQueue->enqueue(compute_slow_value());
   }
}