PHP Timezone database is corrupt error

Question

I have a wordpress website which suddenly stopped working today. When I look at the logs I see and error:

[error] [client 50.78.108.177] PHP Fatal error: strtotime(): Timezone database is corrupt - this should never happen!

After reading up on google one person said that they discovered a permissions problem in /usr/share/zoneinfo. I tried changing the permissions to 777, 775, 770 and I still keep on getting the same error. I am running php PHP 5.3.2 on Ubuntu 10.04.3 LTS. Any suggestions or recommendations would be helpful.If all else fails I'm going to try downgrading to an earlier version of php but I wanted to try other things before doing that.

thanks, Timnit

Update
just in case it helps: the error points to strtotime in the function below

function mysql2date( $dateformatstring, $mysqlstring, $translate = true ) {
    $m = $mysqlstring;
    if ( empty( $m ) )
            return false;

    if ( 'G' == $dateformatstring )
            return strtotime( $m . ' +0000' );

    $i = strtotime( $m );

    if ( 'U' == $dateformatstring )
            return $i;

    if ( $translate )
            return date_i18n( $dateformatstring, $i );
    else
            return date( $dateformatstring, $i );
}

Update#2:
for now I have fixed the problem by simply having the function above return false; without performing anything. However I still haven't figured out the root cause of the problem.

update#3:

var_dump($dateformatstring)

string(5) "d.m.y" string(1) "m" string(5) "d.m.y" string(1) "m" string(5) "d.m.y" string(1) "m"

var_dump($mysqlstring)

string(19) "2011-10-20 05:35:01" string(19) "2011-10-20 05:35:01" string(19) "2011-10-20 05:25:22" string(19) "2011-10-20 05:25:22" string(19) "2011-10-19 05:10:06" string(19) "2011-10-19 05:10:06"

update#4:
there is another code snippet that is generating the error log below:

PHP Fatal error: date(): Timezone database is corrupt - this should never happen! in /srv/www/motionthink.com/public_html/wp-admin/includes/class-wp-filesystem-direct.php on line 346, referer: wp_root_directory/wp-admin/plugins.php?plugin_status=upgrade

309         function dirlist($path, $include_hidden = true, $recursive = false) {
  310                 if ( $this->is_file($path) ) {
  311                         $limit_file = basename($path);
  312                         $path = dirname($path);
  313                 } else {
  314                         $limit_file = false;
  315                 }
  316 
  317                 if ( ! $this->is_dir($path) )
  318                         return false;
  319 
  320                 $dir = @dir($path);
  321                 if ( ! $dir )
  322                         return false;
  323 
  324                 $ret = array();
  325 
  326                 while (false !== ($entry = $dir->read()) ) {
  327                         $struc = array();
  328                         $struc['name'] = $entry;
  329 
  330                         if ( '.' == $struc['name'] || '..' == $struc['name'] )
  331                                 continue;
  332 
  333                         if ( ! $include_hidden && '.' == $struc['name'][0] )
  334                                 continue;
  335 
  336                         if ( $limit_file && $struc['name'] != $limit_file)
  337                                 continue;
  338 
  339                         $struc['perms']         = $this->gethchmod($path.'/'.$entry);
  340                         $struc['permsn']  = $this->getnumchmodfromh($struc['perms']);
  341                         $struc['number']        = false;
  342                         $struc['owner']         = $this->owner($path.'/'.$entry);
  343                         $struc['group']         = $this->group($path.'/'.$entry);
  344                         $struc['size']          = $this->size($path.'/'.$entry);
  345                         $struc['lastmodunix']= $this->mtime($path.'/'.$entry);
  346                         $struc['lastmod']   = date('M j',$struc['lastmodunix']);
  347                         $struc['time']          = date('h:i:s',$struc['lastmodunix']);
  348                  $struc['type']          = $this->is_dir($path.'/'.$entry) ?   'd:'f';
  349 

Update#5:
doing a php -i | fgrep -i date returns

Build Date => Dec 13 2011 18:43:02

date
date/time support => enabled
date.default_latitude => 31.7667 => 31.7667
date.default_longitude => 35.2333 => 35.2333
date.sunrise_zenith => 90.583333 => 90.583333
date.sunset_zenith => 90.583333 => 90.583333
date.timezone => no value => no value

then I edited the php.ini file to set the timezone to "America/Los Angeles" and got this output

date/time support => enabled
date.default_latitude => 31.7667 => 31.7667
date.default_longitude => 35.2333 => 35.2333
date.sunrise_zenith => 90.583333 => 90.583333
date.sunset_zenith => 90.583333 => 90.583333
date.timezone => America/Los_Angeles => America/Los_Angeles

I then restarted apache2. I still get the error

Answer 1

This issue can also occur when using php-fpm in chroot mode, the solution in this case to be to create something like /usr/share/zoneinfo/Europe in your chroot dir then copy your TZ file in to it e.g. London

Answer 2

Root cause: one of the zoneinfo files could not be opened.

also caused by: too many open files.

I had the same problem today on Ubuntu 14.04.01-LTS "Trusty Tahr", and tried the other answers with no benefit. Permissions were OK, the files were there, the content was as expected.

At last I resolved to run the script from within a command line harness, so that I could try with strace. And this was the result:

openat(AT_FDCWD, "/usr/share/zoneinfo/", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = -1 EMFILE (Too many open files)
open("/usr/share/zoneinfo/zone.tab", O_RDONLY) = -1 EMFILE (Too many open files)
stat("/usr/share/zoneinfo/Europe/Rome", {st_mode=S_IFREG|0644, st_size=2652, ...}) = 0
open("/usr/share/zoneinfo/Europe/Rome", O_RDONLY) = -1 EMFILE (Too many open files)
write(1, "\nFatal error: Unknown: Timezone "..., 104) = 104

What is happening

When PHP "accesses the zoneinfo database" it actually tries to open a directory and some files. If some of these operations fail, the "zoneinfo corrupt" message appears, but it simply means that the PHP process could not open those files:

• they were not there (chroot jail, zoneinfo install error)
• they were not there, nor should they be: "Europe/Roem" is not a valid timezone but a typo.
• they were there, but with wrong permissions.
• they were there, but the process isn't authorized (SELinux, AppArmor, ...)
• they were there, but the fopen operation is temporarily not working

My case was the last one: the real problem was that the script was opening too many temporary files, and leaving them open while running. There is a limit on how many files can be opened at the same time, and the zoneinfo file was the proverbial last straw. A quick fix temporarily resolved the problem while I bounced the "too many files" problem to the developer responsible.

Actually I suspect that this also points to PHP continuously opening and closing the zoneinfo database instead of caching it, but it's an investigation for another day.

Intermittent error The "number of open files" thingy is per process, not per PHP script. So there are two (at least) scenarios which could lead to a hard-to-diagnose, possibly intermittent/nonreproducible error:

• a slow resource leak by some long-running process, e.g. under Racket.
• a resource hogging by another script or subroutine running in the same process, and possibly not even related to PHP at all.

A PHP script that, right or wrong, allocates 800 files could work okay until it meets another subprocess that has allocated 224 files. The limit of 1024 open files per process is reached and in that case the process fails with a mysterious error (which only refers, cryptically at that, to the very last symptom in a long chain of concurrent causes).

Apache: too many web sites.

Apache running with mod_php5 will cause files accessed by PHP to be opened by the Apache process. But the Apache process also keeps its log files open, and every process has a handle to every log file.

So if you have 200 web sites, each with an independent access_log, say /var/www/somesite/logs/access_log, each process will start with some 210 handles already taken for housekeeping, leaving some 800 free for PHP to use.

This can lead to a situation where the development server (with one site) works, and the production server (with 200 sites installed) does not, if the script needs to allocate 900 temporary files at once.

Dirty diagnostics (on Unix/Linux): glob /proc/self/fd and count() the result. Ugly as sin, but it gives a ballpark figure on how many file descriptors are actually open.

Quick and dirty fix (on Unix/Linux): increase the fdlimit on per-process open files, bringing it to 1024 (of course you need to be root). It's more a matter for Server Fault.