Laravel 5 losing sessions and .env configuration values in AJAX-intensive applications

Question

I am using Laravel 5 (to be specific, "laravel/framework" version is "v5.0.27"), with session driver = 'file'.

I'm developing on Windows 7 64 bit machine.

I noticed that sometimes (once a week or so) I get unexpectedly and randomly logged out. Sometimes this happens even immediately after I log in. I have added log messages to my auth logic code, but the log code was not triggered. Laravel behaved as if it has completely lost the session file.

Another, more serious issue, was that sometimes after debugging sessions (using xdebug and Netbeans) Laravel started losing also other files - .env settings, some debugbar JS files etc. The error log had messages like:

[2015-07-08 13:05:31] local.ERROR: exception 'ErrorException' with message 'mcrypt_encrypt(): Key of size 7 not supported by this algorithm. Only keys of sizes 16, 24 or 32 supported' in D:\myproject\vendor\laravel\framework\src\Illuminate\Encryption\Encrypter.php:81
[2015-07-08 13:05:31] local.ERROR: exception 'PDOException' with message 'SQLSTATE[HY000] [1044] Access denied for user ''@'localhost' to database 'forge'' in D:\myproject\vendor\laravel\framework\src\Illuminate\Database\Connectors\Connector.php:47

This clearly signals that .env file was not read by Laravel, so it is using default settings:

'database'  => env('DB_DATABASE', 'forge'),
'key' => env('APP_KEY', 'somekey'),

Losing files happened rarely, maybe once a month or so, and it always happened after debugging sessions. I always had to restart Apache to make it work again.

To stress-test the system and reproduce the issues reliably, I used a quick hack in my Angular controller:

setInterval(function(){
    $scope.getGridPagedDataAsync();
}, 500);

It is just a basic data request from Angular to Laravel.

And that was it - now I could reproduce the session losing and .env losing in 3 minutes or less.

I have developed AJAX-intensive web applications earlier on the same PC with the same Apache+PHP, but without Laravel, without .env, and I hadn't noticed such issues before.

While debugging through code, I found out that Laravel is not using PHP built-in sessions at all, but has implemented their own files-based session. Obviously, it does not provide the same reliability as default PHP sessions and I'm not sure why.

Of course, in real life scenarios my app won't be that AJAX-intensive, but in my experiences on some occasions it is enough with just two simultaneous AJAX requests to lose the session.

I have seen some related bug reports on Laravel for various session issues. I haven't yet seen anything about dot-env, though, but it seems suffering from the same issue.

My guess is that Laravel does not use file locks and waiting, thus if a file cannot be read for some reason (maybe locked by some parallel process or Apache) then Laravel just gives up and returns whatever it can.

Is there any good solution to this? Maybe it is specific to Windows and the problems will go away on a Linux machine?

Curious, why Laravel (or Symfony) developers haven't fixed their session file driver yet. I know that locking/waiting would slow it down, but it would be great to at least have some option to turn on "reliable sessions".

Meanwhile I'll try to step through Laravel code and see if I can invent some "quick&dirty" fix, but it would be much better to have some reliable and "best practices" solution.

Update about .env

The issue turned to be not related to locking files. I found the Laravel bug report for .env issue, which lead me to a linked report for Dotenv project which, in turn, says that it is a core PHP issue. What disturbs me is that Dotenv devs say that Dotenv was never meant to be used for production, but Laravel seems to rely upon Dotenv.

In https://github.com/laravel/framework/pull/8187 there seems to be a solution which should work in one direction but some commenter says that in their case the issue was the opposite. Someone called crynobone gave a clever code snippet to try:

$value = array_get($_ENV, $key, getenv($key));

There appeared another suggestion to use "makeMutable()" on both Dotenv and Laravel Githubs, but commenters report that this might break tests.

So I tried the crynobone's code but it did not work for me. While debugging, I found out that in my case when things break down for concurrent requests, the $key cannot be found nor in getenv(), nor in $_ENV and not even in $_SERVER. The only thing that worked (quick&dirty experminet) was to add:

static::$cached[$name] = $value;

to Dotenv class and then in helpers.php env() method I see that:

Dotenv::$cached[$key]

is always good, even when $_ENV and getenv both give nothing.

Although Dotenv was not meant for production, I don't want to change our deployment and configuration workflow.

Next I'll have to investigate the session issues...

---

Addendum

Related Laravel bug reports (some even from version 4. and it seems, not fixed): https://github.com/laravel/framework/issues/4576

https://github.com/laravel/framework/issues/5416

https://github.com/laravel/framework/issues/8172

and an old article which sheds some light on what's going on (at least with session issues): http://thwartedefforts.org/2006/11/11/race-conditions-with-ajax-and-php-sessions/

Answer 1

After two days of intensive debugging I have some workarounds which might be useful to others:

Here is my patch for Dotenv 1.1.0 and Laravel 5.0.27 to fix .env issues: https://gist.github.com/progmars/db5b8e2331e8723dd637

And here is my workaround patch to make session issues much less frequent (or fix them completely, if you don't write to session yourself on every request): https://gist.github.com/progmars/960b848170ff4ecd580a

I tested them with Laravel 5.0.27 and Dotenv 1.1.0.

Also recently recreated patches for Laravel 5.1.1 and Dotenv 1.1.1: https://gist.github.com/progmars/e750f46c8c12e21a10ea https://gist.github.com/progmars/76598c982179bc335ebb

Make sure you add

'metadata_update_threshold' => 1,

to your config/session.php for this patch to become effective.

All the patches should be applied to vendor folder each time it gets recreated.

Also you might want to separate the session patch out because you update session.php just once, but the other parts of the patch should be applied to vendor folder each time it gets recreated before deployment.

Be warned: "It works on my machine". I really hope Laravel and Dotenv developers will come up with something better, but meanwhile I can live with these fixes.

Answer 2

My personal opinion that using .env to configure Laravel is a bad decision. Having .php files that contained key:value style of configuration was much better.

However, the problem you are experiencing is not PHP's fault, nor Apache's - it's most likely Windows issue.

A few other things: Apache contains a module that allows PHP binary to be integrated into Apache's process or thread, called mod_php - the issue with this is that PHP is not only slow, but getting another binary integrated into an existing one is super tricky and things might be missed. PHP also must be built with thread-safety in this case. If it's not, then weird bugs can (and will) occur.

To circumvent the problem of tricky integration of one program into another, we can avoid this completely and we can have .php served over FastCGI protocol. This means that the web server (Apache or Nginx) will take the HTTP request and pass it to another "web" server. In our case, this will be PHP FastCGI Process Manager or PHP-FPM.

PHP-FPM is preferred way of serving .php pages - not only because it's faster (much, much faster than integrating via mod_php), but you can easily scale your HTTP frontend and have multiple machines serve .php pages, allowing you to easily horizontally scale your HTTP frontend.

However, PHP-FPM is something called a supervisor process and it relies on process control. As far as I'm aware, Windows do not support process control in the way *nix does, therefore php-fpm is unavailable for Windows (in case I am wrong here, please correct me).

What does all of this mean for you? It means that you should use software that's designed to play nicely with what you want to do. This is the logic that should be followed:

• A web server accepts HTTP requests (Apache or Nginx)
• Web server validates the request, parses the raw HTTP request, determines whether the request is too big and if everything goes well in this case, it proxies the request to php-fpm.
• php-fpm processes the request (in your case it boots up Laravel) and returns the HTML which the web server shows to the user

Now, this process while great, comes with a few issues and one huge problem here is how PHP deals with sessions. A default PHP session is a file stored somewhere on the server. This means that if you have 2 physical machines serving your php-fpm, you're going to have problems with sessions. This is where Laravel does something great - it lets you use encrypted cookie based sessions. It comes with limitations (you can't store resources in those sessions and you have a size limit), but a correctly built app wouldn't store too much info in a session in the first place. There are, of course, multiple ways of dealing with sessions, but in my opinion the encrypted cookie is super, super trivial to use and powerful. When such a cookie is used, it's the client who carries the session information and any machine that contains decryption key can read this session, which means that you can easily scale your setup to multiple servers - all they have to do is have access to same decryption key (it's the APP_KEY in the .env). Basically you need to copy the same Laravel installation to machines that you wish to serve your project.

The way I would deal with the issue that you have while developing is the following:

• Use a virtual machine (let's say Oracle Virtualbox)
• Install Ubuntu 14.04
• Map a network drive to your Windows host machine (using Samba)
• Use your preferred way of editing PHP files, but they would be stored on the mapped drive
• Boot an nginx or Apache, along with php-fpm on the VM to serve your project

Now what you gain via this process is this: you don't pollute your Windows machine with a program that listens on ports 80 / 443, when you're done working you can just shut the VM down without losing work, you can also easily simulate how your website would behave on an actual production machine and you wouldn't have surprises such as "it works on my dev machine but it doesn't work on my production machine" because you'd have the same software for both purposes.

These are my opinions, they are not all cold-facts and what I wrote here should be taken with a grain of salt. If you think what I wrote might help you, then please try to approach the problem that way. If not, well, no hard feelings and I wish you good luck with your projects.