Capistrano Symlinks Being Cached?

Question

I've been setting up PHP deployments with Capistrano on CentOS 6 and have run into an interesting issue. The way capistrano works, it sets up folders like this:

• /var/www/myapp.com/
  • current (symlink to latest release in /releases)
  • shared
  • releases
    • 20130826172737
    • 20130826172114

When I look at the "current" symlink, it points to the most recent release. At first, when opening my web app, everything worked fine. After deploying a new release, the current folder correctly points to the new release, but the web application tries to load files from the old release (which has been deleted in a Capistrano cleanup process). Also, the virtual host is configured to point at /var/www/myapp.com/current/Public.

Are symlinks cached in any way?

The specific PHP code that fails (which initializes my framework) is this:

require_once dirname(dirname(__FILE__)) . '/App/App.php';
App\App::run();

That is in index.php currently located at /var/www/app.com/current/Public/index.php.

My Apache error logs show:

PHP Fatal error: require_once(): Failed opening required '/var/www/myapp.com/releases/20130826172237/App/App.php' (include_path='.:/usr/share/pear:/usr/share/php') in /var/www/myapp.com/releases/20130826172237/Public/index.php

And the current symlink shows:

current -> /var/www/zverse/releases/20130826172641

Obviously 20130826172641 != 20130826172237 which the latter was the previous version.

Any ideas or areas I can look at?

Answer 1

I can't verify this, but it seems that there is some unpredictable behaviour with Apache following / caching the old location of symlinks:

• Is there a way to mimic symlink behavior with an apache configuration?
• Case Against Using Symlinks For Code Promotion

The only thing that would absolutely clear up this issue was to cycle Apache, which we would prefer not to do on every deployment. -- Mike Brittain

He suggests moving the whole directory, instead of updating the symlink.

Answer 2

Have you checked the realpath_cache_size and realpath_cache_ttl directives? By default, php > 5.1 caches the real paths of symlinked files for 120 seconds. This will cause problems with capistrano deployments. The main problems are caching - that even if you clear your cache, your old php files will continue to be served for two minutes, repopulating it with old data - and interaction between php and static files. Static files are served directly by Apache, so will be updated immediately. The php code will still be from the previous release for two minutes after deploying though, so it will be expecting the old versions of any changed static files. That's especially a problem if you use a cache breaking procedure that changes the names of those files; in that case the php code won't be able to find the files it's expecting at all.

Anyway, there are two solutions. The first is to set realpath_cache_size to 0 in php.ini. (Note: setting realpath_cache_ttl to 0 does not disable the cache.) Or, if you want to keep it enabled, you should be able to use the clearstatcache function to clear the realpath cache immediately after deploying your symlink, using a capistrano hook. Be aware though, if you're using mod_php, the php cli and apache runtimes are separate, so you would need to call that function using a php script invoked by apache, similarly to what's done for clearing the APC cache here. I haven't tested that though, as I didn't notice a significant performance impact from simply disabling the cache.