Using AnyEvent run_cmd in Mojolicious, I keep getting this error: "AnyEvent::CondVar: recursive blocking wait attempted"

Question

In a Mojolicious app, I'm trying to convert ODT files into HTML when a link is clicked. I convert the files by using "soffice", a shell command. Converting the files takes some time. I send status messages to the user to notify him of the progress. I send those status update messages by writing to a Mojo::Log object. Then, I subscribe to this log object in an EventSource route.

Then I loop through the files and use AnyEvent::Util run_cmd to execute the external "soffice" program.

for my $file (@{ $filelist }) {
   my $output_dir = './output_dir';
   my $cmd = "soffice --headless --convert-to html --outdir '$output_dir' '$file'";
   my $cv = AnyEvent->condvar;
   my $w;
   $w = run_cmd($cmd, 
                '>'  => sub { my $out = shift;
                              &WriteToLog({ status => "cmd output '$out'..." });
                              undef $w;
                              $cv->send;
                 },

                '2>' => sub { my $err = shift;
                              &WriteToLog({ status => "ERROR '$err'..." });
                              undef $w;
                              $cv->send;
                 }
            );

   $cv->recv;
}

Pretty much copied and pasted from the main AnyEvent tutorials. If there are only few files to convert (around 2 or 3), then all goes well. The status messages sent through the EventSource connection appears on the client browser. Then after all the files have been converted, the web page is rendered.

If more files are to be processed, a few files get converted then the error message in the thread title occurs.

The routing for the route containing the code above is this:

my $initdocs = $r->under->to('docroute#initdocs');
$initdocs->get('/showdocs')->to('docroute#showdocs');

The code above is in the "initdocs" route.

Any help is appreciated. Thanks in advance.

Answer 1

I think what you are trying to do is to call the soffice (blocking) process without blocking the rest of the server thread. I'm no expert on AE but I don't think that's what run_cmd does. It is closer to what fork_call does though. Perhaps what you want to do is something more like this:

#!/usr/bin/env perl

use Mojolicious::Lite;
use EV;
use AnyEvent::Util 'fork_call';

any '/' => sub {
  my $c = shift;
  $c->render_later;
  fork_call { `sleep 5 && echo 'hi'` } sub {
    my $data = shift;
    $c->render( text => $data );
  };
};

app->start;

In my example I just make a simple blocking call, but you could as easily call out to soffice.

Now since you say you may have to convert several different files before returning to the client, you may want to use the excellent Mojo::IOLoop::Delay to manage the processes.

#!/usr/bin/env perl

use Mojolicious::Lite;
use EV;
use AnyEvent::Util 'fork_call';

my @jobs = (
  q{sleep 5 && echo 'hi'},
  q{sleep 5 && echo 'bye'},
);

any '/' => sub {
  my $c = shift;
  $c->render_later;
  my $delay = Mojo::IOLoop->delay;
  $delay->on( finish => sub { 
    shift; $c->render(text => join '', @_ );
  });
  fork_call { `$_` } $delay->begin(0) for @jobs;
};

app->start;

Once again, I am just capturing the output and sending it to the render call, but notice that it waits for all the jobs to finish before returning. Something closer to your real use-case might be:

#!/usr/bin/env perl

use Mojolicious::Lite;
use EV;
use AnyEvent::Util 'fork_call';
use Capture::Tiny 'capture';

any '/' => sub {
  my $c = shift;
  my $files = $c->every_param('file');
  $c->render_later;
  my $delay = Mojo::IOLoop->delay;
  $delay->on( finish => sub { 
    shift; $c->render( json => \@_ );
  });
  my $output_dir = './output_dir';
  for my $file (@$files) {
    my $cmd = "soffice --headless --convert-to html --outdir '$output_dir' '$file'";
    fork_call { [ capture { system $cmd } ] } $delay->begin(0);
  }
};

app->start;

This runs soffice on each file name passed as a param to the route (/?file=myfile&file=otherfile). Then the stdout, stderr and return code are (well should be, I haven't run this one obviously) rendered as json to the client (you could as easily log it).

Answer 2

Creating single Thread Server with AnyEvent

AnyEvent recursive Blocking..

If you are using AnyEvent, you usually have to deal with CondVars. There are two things you can do with a CondVar: Either you register a callback which will be called when the CondVar is triggered, or you call recv and it will block until the CondVar is triggered. In a route of your Mojo::Controller, you probably want to block until you got all the data you want to display to your user.

Take the following (made-up) example which uses a CondVar:

untested:

get '/' => sub {
    ...
    my $cv = AnyEvent->condvar;
    my $timer = AnyEvent->timer(after => 1, cb => sub { $cv->send(1) });
    my $result = $cv->recv;
    ...
};

You will get a runtime error stating "AnyEvent::CondVar: recursive blocking wait detected". Maybe this is because Morbo also uses a CondVar as exit_guard, to run infinitely long (blocking on a CondVar is an easy way to run the main loop).

My approach would be to use a specific eventloop, such as EV, and call EV->loop instead of blocking on a CondVar:

EV->loop