Can I use a module, and later unload it shrinking the optree?

Question

Disclaimer I'm not sure I'm using the right terms. It may not be the optree responsible for the bloat mentioned below: it may be the symbols loaded by DynaLoader that are not freed.

Is it possible to use a module, like POSIX.pm, unload it and lessen (shrink, or prune) the optree without either

1. Rexecing perl
2. Forking

Things I've tried,

1. Class::Unload->unload('POSIX');
2. Symbol::delete_package('POSIX');
3. no POSIX;

Here is an easy test create a file test.pl

$|++;
use Symbol;
use Class::Unload;
use POSIX;

print "GOT POSIX";
sleep(3);

no POSIX;
Class::Unload->unload('POSIX');
Symbol::delete_package('POSIX');
print "unloaded";

sleep(3);

Shell command

perl ./test.pl & watch -n1 'ps -C perl -o "cmd rss";'

You may or may not be able to see the RSS size increase (POSIX may load before watch spawns ps). But, I want to see it shrink back down.

Tracking down what exactly POSIX.pm does I see it uses XSLoader which uses DynaLoader.

Doing some quick comparative checks in /proc/$$/smaps I've determined that using POSIX.pm causes a heap allocation that represents the difference in space. The first allocation on the heap, is massively bigger when using POSIX.pm:

56122fe4c000-561230040000 rw-p 00000000 00:00 0                          [heap]
Size:               2000 kB
Rss:                1956 kB
Pss:                1956 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:      1956 kB
Referenced:         1956 kB
Anonymous:          1956 kB
AnonHugePages:         0 kB
ShmemPmdMapped:        0 kB
Shared_Hugetlb:        0 kB
Private_Hugetlb:       0 kB
Swap:                  0 kB
SwapPss:               0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB
VmFlags: rd wr mr mw me ac sd

vs

560c9f6ba000-560c9f6fc000 rw-p 00000000 00:00 0                          [heap]
Size:                264 kB
Rss:                 220 kB
Pss:                 220 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:       220 kB
Referenced:          220 kB
Anonymous:           220 kB
AnonHugePages:         0 kB
ShmemPmdMapped:        0 kB
Shared_Hugetlb:        0 kB
Private_Hugetlb:       0 kB
Swap:                  0 kB
SwapPss:               0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB
Locked:                0 kB
VmFlags: rd wr mr mw me ac sd

I've confirmed a few things, nuking the namespace does not drop the open file handle to POSIX.so and Fnctl.so -- I determined this with lsof. That is in itself somewhat concerning. I would think it would make sense to allocate the handle on the callee's package. XSLoader also obscures that you can release that file handle -- a feature available in DynaLoader.

Further, it seems that in libc / dlfcn.h I have

dlclose()

The function dlclose() decrements the reference count on the dynamically loaded shared object referred to by handle. If the reference count drops to zero, then the object is unloaded. All shared objects that were automatically loaded when dlopen() was invoked on the object referred to by handle are recursively closed in the same manner.

A successful return from dlclose() does not guarantee that the symbols associated with handle are removed from the caller's address space. In addition to references resulting from explicit dlopen() calls, a shared object may have been implicitly loaded (and reference counted) because of dependencies in other shared objects. Only when all references have been released can the shared object be removed from the address space.

So I'm guessing that may be suspect, DynaLoader::dl_unload_file is calling dlclose and it does seems to work.

foreach my $dlref ( @DynaLoader::dl_librefs ) {
  print DynaLoader::dl_unload_file($dlref);
}

After I nuked all files loaded with DynaLoader and XSLoader by doing the above the RSS still did not drop.

Answer 1

Generally speaking, no. The gritty details are that almost no one shrinks their own memory because almost everyone uses the C library malloc (and friends) call to allocate memory, either directly or indirectly. And there is no (standard) way to tell the C library to deallocate memory (to send it back to the OS). Perl is no different here - once malloced and freed, the C library upon which Perl depends keeps the memory for future use so that if you need to reuse the memory, no expensive kernel calls are required (specifically, brk), and it can simply be reused. In fact, this is what your unload scenarios are doing - when you come back and reuse that next 2MB in the rest of your server process, you'll be re-using the memory, not calling brk, and you'll be that much faster.

It is possible to do if you take over memory allocation ownership and call brk yourself, but it's rarely worth it. And getting perl to use that allocator would require some code changes to perl and recompilation. Probably not what you want to do.

Other options are to either bite the bullet, load POSIX prior to forking off any servers (which should leave all of that in shared copy-on-write memory, and thus only take up 2MB of memory for 5k servers), or fork, load POSIX in the child, do the dirty work, exit the child, and continue in the parent. This seems to be relatively slow to me.

Answer 2

Yes, you can.

But there are dragons, and practically not.

SV's and OP's are allocated in arenas. OP's hold pointers to their data, SV's. Those OP's and SV's can be freed via undef, with the malloc'ed parts being immediately freed and the arenas (~70 OPs) freed when all OPs therein are freed.

Then you have the globals which can be easily freed also by walking the namespace. But beware of not destroying data for which references from somewhere else still exist, and it's DESTROY handler cannot deal with that. There's a lot of unsafe DESTROY code out there, because nobody does this.

And sometimes to be deleted globals are referenced from somewhere else, so it will not be freed, just the refcount drops.

And then you have the external XS code, for which you have to call dl_unload_file().

In your case use POSIX creates tons of imports into the main:: namespace, GV aliases of all the imported functions. They need to be deleted also. use POSIX (); will skip the import, so will need MUCH less memory, and chances are it can be deleted fully.

To see what is not really undef'd see

#!/usr/bin/perl
$|++;
my $s = shift // 3;
sub rss { `ps -o "comm,rss,vsize" | grep perl` }
print "BEGIN ",scalar keys %main::," ",rss;
require Symbol;
#require Class::Unload;
require POSIX;

print "GOT POSIX ",scalar keys %main::," ",rss;
sleep($s);

POSIX->import;
print "IMPORT POSIX ",scalar keys %main::," ",rss;
sleep($s);

POSIX->unimport;
#Class::Unload->unload('POSIX');
Symbol::delete_package('POSIX');

for (keys %main::) {
  #print "$_\n";
  undef ${$_} unless /^(STD|!|0|1|2|\]|_)/;
  undef &{$_} unless /rss/;
  undef @{$_};
  # clear the GV
  undef *{$_} unless /^(STD...?|rss|main::|DynaLoader::|_|!)$/;
  # delete the GV
  delete $main::{$_} unless /^(STD...?|rss|main::|DynaLoader::|_|!)$/;
}
#Symbol::delete_package('main::'); # needs a patched Symbol
print "unloaded ",scalar keys %main::," ",rss;

sleep($s);

DynaLoader::dl_unload_file($_) for @DynaLoader::dl_librefs;
undef *DynaLoader::;
print "unload XS ",scalar keys %main::," ",rss;
#print "  $_\n" for keys %main::;
print "POSIX::$_\n" for keys %POSIX::;
print "freed ",scalar keys %main::," ",rss;
sleep($s);

result,

=>
  BEGIN 45 /usr/src/perl/bl   3192  2451188
  GOT POSIX 70 /usr/src/perl/bl   6112  2468844
  IMPORT POSIX 645 /usr/src/perl/bl   6928  2468844
  unloaded 8 /usr/src/perl/bl   7120  2468844
  unload XS 8 /usr/src/perl/bl   7040  2468596
  freed 8 /usr/src/perl/bl   7048  2468596

which shows that

1. Symbol is unreliable deleting readonly, protected symbols, and
2. the global symbols (in main::) are not freed by undef, but by deleting the stash entry.
3. Do not import POSIX and such old import-heavy modules, rather use the full name. Purging these is hard.
4. You cannot free SVs only OPs, memory will mostly increase, not shrink.

SV head and body arena's are never freed, they are just reused. So you can only shrink the optree, not the data.

The SV's behind the symbol are just set to TEMP if undef'd, so its memory is never freed, and the symbol itself (the GV) is only cleared with undef. OPs are deleted by undef'ing the CV, but system malloc rarely frees it, only if a full page is freed and with glibc with a call to malloc_trim(0), and perl's memory is too sprinkled out. It's still a linked list with not much compaction after all.

The rss goes a bit down from unload XS to freed, but it still higher than after the initial import.

My watcher is watch -n1 'ps -o "comm,rss,vsize" |grep perl;' because this works also on BSD/darwin.

I wrote a Internals::gc() for cperl to actually walk all arenas and free the empty ones, but it's pretty unstable and not recommended, as the VM can only "properly" deal with those free SVs during global destruction, not at run-time. See https://github.com/perl11/cperl/issues/336