Queue in laravel with redis

Question

I'm having trouble with an application where i do a lot of api calls in the backend. Because of this the application gets a timeout error.

Someone suggested I should use Queues. I have tried to do so with redis. The function goes in the job and uses the handler but I expected the page to load with the data I give it without the data from the api while the api-call is going through on the background. Instead it just goes through like when i hadn't used the queue. I tried to follow a tutorial to do this but they weren't doing exactly the same and I couldn't adjust it so it would work for me.

For info about what i do in the job. I get comments out a csv and the I use the numbers in the comments to call to the api and i get a json of like 8-10 fields. I need to call the api about 650 times so it takes o long when i want to save the data to the database. I use one insert at a time to use "caching" so i wont do the same call twice.

This is the controller where I call on the job.

class ImportController extends Controller
{
public function checkErrors(Request $request)
{
    $this->checkAgainstDocuments($csv_id);
    $supplierErrorIds=$this->checkSupplierErrors($parameters, $company , $csv_id);
    $timesheetErrors=TsData::whereIn('id', $supplierErrorIds)->sortable()->paginate(20);
    return view('show_errors', compact('timesheetErrors', 'company', 'csv_id'));
}

public function checkAgainstDocuments($csv_id)
{
    GetFromDocumentsAPI::dispatch($csv_id)->delay(now()->addMinutes(10));
}
}

This is the job I use:

class GetFromDocumentsAPI implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

protected $csv_id;
/**
 * Create a new job instance.
 *
 * @return void
 */
public function __construct($csv_id)
{
    //
    $this->csv_id=$csv_id;
}

/**
 * Execute the job.
 *
 * @return void
 */
public function handle()
{
    $comments = TsData::select('id', 'comments')->where('csv_id', $this->csv_id)->get()->toArray();

    $commentIDs = array();

    foreach ($comments as $comment) {
        preg_match_all('/(\d{5,})/', $comment['comments'], $out, PREG_PATTERN_ORDER);
        foreach ($out as $item) {
            $commentIDs[$comment['id']] = $item;
        }
    }

    $commentIDs = array_filter($commentIDs);

    $apiKey=config('app.apiKey');

    $documentsResponse = array();

    Issue::truncate();

    $arrayTest=[];

    foreach ($commentIDs as $key => $commentID) {
        foreach ($commentID as $item) {
            $issue = Issue::where('id', $item)->first();

            if ($issue === null) {
                try {
                    $url = file_get_contents('https://documents.calibrate.be/issues/' . $item . '.json?key=' . $apiKey);
                    $json = json_decode($url, true);
                } catch (Exception $e) {
                    echo 'Caught exception: ', $e->getMessage(), "\n";
                }

            

$issue = Issue::Create(['id'=>$item, 'name'=>$json['issue']['subject'], 'projectId'=>$json['issue']['project']['id'], 'priority'=>$json['issue']['priority']['id']]);

        }
    }
}

}

config/queue.php

'default' => env('QUEUE_DRIVER', 'redis'),

'redis' => [
        'driver' => 'redis',
        'connection' => 'default',
        'queue' => env('REDIS_QUEUE', 'default'),
        'retry_after' => 90,
        'block_for' => null,
    ],

Answer 1

By default, Laravel uses the sync driver for handling queues.

Make sure that the QUEUE_CONNECTION config variable is set to either database or redis or any other service. The same can be set in the .env file as well as the config/queue.php file.

To use database, do

1. Run php artisan queue:table and php artisan migrate. This will create all the required tables to run.
2. Make sure you have the queue worker running in the background. You can do this in the console by running php artisan queue:work

To use redis , do

1. Install sudo apt-get install redis-server and start redis server $ sudo systemctl enable redis-server.service. (For Linux based systems)
2. Configure and set the redis specific variables in .env or config/queue.php

P.S. Run the php artisan config:clear command after making the changes so that the changes are reflected in the cache.