Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Tracking Memory Usage in PHP

I'm trying to track the memory usage of a script that processes URLs. The basic idea is to check that there's a reasonable buffer before adding another URL to a cURL multi handler. I'm using a 'rolling cURL' concept that processes a URLs data as the multi handler is running. This means I can keep N connections active by adding a new URL from a pool each time an existing URL processes and is removed.

I've used memory_get_usage() with some positive results. Adding the real_usage flag helped (not really clear on the difference between 'system' memory and 'emalloc' memory, but system shows larger numbers). memory_get_usage() does ramp up as URLs are added then down as the URL set is depleted. However, I just exceeded the 32M limit with my last memory check being ~18M.

I poll the memory usage each time cURL multi signals a request has returned. Since multiple requests may return at the same time, there's a chance a bunch of URLs returned data at the same time and actually jumped the memory usage that 14M. However, if memory_get_usage() is accurate, I guess that's what's happening.

[Update: Should have run more tests before asking I guess, increased php's memory limit (but left the 'safe' amount the same in the script) and the memory usage as reported did jump from below my self imposed limit of 25M to over 32M. Then, as expected slowly ramped down as URLs where not added. But I'll leave the question up: Is this the right way to do this?]

Can I trust memory_get_usage() in this way? Are there better alternative methods for getting memory usage (I've seen some scripts parse the output of shell commands)?

like image 368
Tim Lytle Avatar asked Feb 18 '10 17:02

Tim Lytle


People also ask

How do I monitor PHP memory usage?

The memory_get_usage function can be used to track the memory usage. The 'malloc' function is not used for every block required, instead a big chunk of system memory is allocated and the environment variable is changed and managed internally. The above mentioned memory usage can be tracked using memory_get_usage().

How much memory does each PHP process consume?

When I set the PHP memory limit to e.g. 128 MB for each of these daemons, the processes will only get killed once they reach 128 MB according to PHP's own measurements. However, according to ps , the processes will be using around 200 MB each by that time.

How can I get CPU and memory usage in PHP?

To get the current memory usage, we can use the memory_get_usage() function, and to get the highest amount of memory used at any point, we can use the memory_get_peak_usage() function.

How can I limit my PHP memory?

You can check it from your php. ini file. Execute php -i | grep "php. ini" on command line and check memory_limit in your Loaded Configuration File.


2 Answers

real_usage works this way:

Zend's memory manager does not use system malloc for every block it needs. Instead, it allocates a big block of system memory (in increments of 256K, can be changed by setting environment variable ZEND_MM_SEG_SIZE) and manages it internally. So, there are two kinds of memory usage:

  1. How much memory the engine took from the OS ("real usage")
  2. How much of this memory was actually used by the application ("internal usage")

Either one of these can be returned by memory_get_usage(). Which one is more useful for you depends on what you are looking into. If you're looking into optimizing your code in specific parts, "internal" might be more useful for you. If you're tracking memory usage globally, "real" would be of more use. memory_limit limits the "real" number, so as soon as all blocks that are permitted by the limit are taken from the system and the memory manager can't allocate a requested block, there the allocation fails. Note that "internal" usage in this case might be less than the limit, but the allocation still could fail because of fragmentation.

Also, if you are using some external memory tracking tool, you can set this environment variable USE_ZEND_ALLOC=0 which would disable the above mechanism and make the engine always use malloc(). This would have much worse performance but allows you to use malloc-tracking tools.

See also an article about this memory manager, it has some code examples too.

like image 70
StasM Avatar answered Oct 05 '22 02:10

StasM


I also assume memory_get_usage() is safe but I guess you can compare both methods and decide for yourself, here is a function that parses the system calls:

function Memory_Usage($decimals = 2) {     $result = 0;      if (function_exists('memory_get_usage'))     {         $result = memory_get_usage() / 1024;     }      else     {         if (function_exists('exec'))         {             $output = array();              if (substr(strtoupper(PHP_OS), 0, 3) == 'WIN')             {                 exec('tasklist /FI "PID eq ' . getmypid() . '" /FO LIST', $output);                  $result = preg_replace('/[\D]/', '', $output[5]);             }              else             {                 exec('ps -eo%mem,rss,pid | grep ' . getmypid(), $output);                  $output = explode('  ', $output[0]);                  $result = $output[1];             }         }     }      return number_format(intval($result) / 1024, $decimals, '.', ''); } 
like image 41
Alix Axel Avatar answered Oct 05 '22 02:10

Alix Axel