PHP running multiple scripts concurrently

Question

I have an array with object server like this:

Array
(
    [0](
                (
                    [id] => 1
                    [version] => 1
                    [server_addr] => 192.168.5.210
                    [server_name] => server1
                )
        )
    [1](
                (
                    [id] => 2
                    [server_addr] => 192.168.5.211
                    [server_name] => server2
                )
        )
)

By running the code below, I'm able to get the desired output

foreach ($model as $server) {
        $cpu_usage = shell_exec('sudo path/to/total_cpu_usage.sh '.$server->server_addr);
        $memory_usage = shell_exec('sudo path/to/total_memory_usage.sh '.$server->server_addr);
        $disk_space = shell_exec('sudo path/to/disk_space.sh '.$server->server_addr);
        $inode_space = shell_exec('sudo path/to/inode_space.sh '.$server->server_addr);
        $network = shell_exec('sudo path/to/network.sh '.$server->server_addr);
        exec('sudo path/to/process.sh '.$server->server_addr, $processString);
        $processArray = array();
        foreach ($processString as $i) {
          $row = explode(" ", preg_replace('/\s+/', ' ', $i));
          array_push($processArray,$row);
        }
        $datetime = shell_exec('sudo path/to/datetime.sh '.$server->server_addr);
        echo $cpu_usage;
        echo $mem_usage;
        echo $disk_space;
        ......
}

My scripts are similar like:

#!/bin/bash
if [ "$1" == "" ]
then
        echo "To start monitor, please provide the server ip:"
        read IP
else
        IP=$1
fi

ssh root@$IP "date"

But the whole process took like 10 sec for 5 servers compared to 1 server for less than 2 sec. Why is that? Is there anyway to lessen the time? My guess is that the exec command was waiting for the output to be assign to the variable before going to next loop? I tried to google a little bit but most of the answer are for without returning any output at all... I need the output though

Answer 1

You can run your scripts simultaneously with popen() and grab the output later with fread().

//execute
foreach ($model as $server) {
    $server->handles = [
        popen('sudo path/to/total_cpu_usage.sh '.$server->server_addr, 'r'),
        popen('sudo path/to/total_memory_usage.sh '.$server->server_addr, 'r'),
        popen('sudo path/to/disk_space.sh '.$server->server_addr, 'r'),
        popen('sudo path/to/inode_space.sh '.$server->server_addr, 'r'),
        popen('sudo path/to/network.sh '.$server->server_addr, 'r'),
    ];
}

//grab and store the output, then close the handles
foreach ($model as $server) {
    $server->cpu_usage = fread($server->handles[0], 4096);
    $server->mem_usage = fread($server->handles[1], 4096);
    $server->disk_space = fread($server->handles[2], 4096);
    $server->inode_space = fread($server->handles[3], 4096);
    $server->network = fread($server->handles[4], 4096);

    foreach($server->handles as $h) pclose($h);
}

//print everything
print_r($model);

I tested a similar code to execute 5 scripts that sleep for 2 seconds and the whole thing took only 2.12 seconds instead of 10.49 seconds with shell_exec().

Update 1: Big thanks to Markus AO for pointing out an optimization potential.

Update 2: Modified the code to remove the possibility of overwrite. The results are now inside $model.

This can also show which server refused the connection, in case that issue about sshd is affecting you.

Answer 2

All you need to do is add an > /dev/null & at the end on Linux, you wont get the output though, but it will run as a background ( async ) process.

shell_exec('sudo path/to/datetime.sh '.$server->server_addr.'  > /dev/null &');

see also this Background process script from my GitHub, ( it has windows compatible background processes )

https://github.com/ArtisticPhoenix/MISC/blob/master/BgProcess.php

Cheers!