How does one enable the OCI8 PHP extension, using the Oracle Instant Client, in Ubuntu 18.04 LTS with PHP-FPM and NGINX?

Question

I'm using the latest PHP packages available from https://launchpad.net/~ondrej/+archive/ubuntu/php .

When I build and install the OCI8 extension, everything appears to be in order, but despite enabling the extension in the PHP-FPM configuration, its presence is not reflected in the output from phpinfo().

The following Gist details the exact process that I'm using to configure, build, and install the OCI8 PHP extension:

https://gist.github.com/cbj4074/fa761f60b6f8db431539d76ebfba828e

The very same process and configuration work perfectly well on Ubuntu 16.04 LTS, so it seems that there is some fundamental difference on Ubuntu 18.04 LTS, whether with the operating system or the PHP packages in question.

As a bit of important (and I suspect relevant to this issue) background information, on Ubuntu 18.04 LTS, the extension fails to load in the CLI environment out-of-the-box, with the error:

PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/20160303/oci8.so' - libmql1.so: cannot open shared object file: No such file or directory in Unknown on line 0

I resolved the issue like so:

# echo 'LD_LIBRARY_PATH="/opt/oracle/instantclient_12_2"' >> /etc/environment

I thought that perhaps adding the LD_LIBRARY_PATH to the PHP-FPM environment configuration might resolve the equivalent issue there:

# echo "env['LD_LIBRARY_PATH'] = /opt/oracle/instantclient_12_2" >> /etc/php/7.2/fpm/pool.d/www.conf
# systemctl restart php7.2-fpm

This does indeed cause the LD_LIBRARY_PATH value, as specified, to be reflected in both the Environment section of phpinfo() (when rendered via PHP-FPM + NGINX and requested from a browser) and the PHP Variables section, as $_SERVER['LD_LIBRARY_PATH'].

Oddly, even with PHP-FPM's logging set to debug, I don't see any trace of the libmql1.so error that I experience with the CLI. The OCI8 extension simply fails to load, silently. display_startup_errors = On in PHP-FPM's effective php.ini, too.

I elected to see if the OCI8 extension works in Apache, on the same server, and it does, provided I add export LD_LIBRARY_PATH=/opt/oracle/instantclient_12_2 to /etc/apache2/envvars; in its absense, Apache complains on startup:

PHP Warning: PHP Startup: Unable to load dynamic library 'oci8.so' (tried: /usr/lib/php/20170718/oci8.so (libmql1.so: cannot open shared object file: No such file or directory), /usr/lib/php/20170718/oci8.so.so (/usr/lib/php/20170718/oci8.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0

None of this business with the LD_LIBRARY_PATH is necessary on Ubuntu 16.04 LTS, and based on my observations herein and the comments regarding https://stackoverflow.com/a/45242468/1772379 , that changed in Ubuntu 17.10 and Ubuntu 18.04 LTS.

Has anybody else tried this, on Ubuntu 18.04 LTS, specifically?

I've tried this on two different Vagrant VMs, laravel/homestead box 6.0.0, and ubuntu/bionic64 box v20180509.0.0, and the behavior is the same in both.

Any other ideas would be most appreciated!

EDIT 1:

I asked about this issue on the package maintainer's GitHub tracker and he suggested that the problem stems from failing to set an appropriate RPATH at compile time.

I explain in my reply that I am setting an appropriate value, but the issue remains closed.

I do notice an interesting detail, however, which is that the compiled extension on Ubuntu 18.04 uses RUNPATH (and not RPATH, which is used in Ubuntu 16.04). If PHP-FPM ignores RUNPATH, and looks only for RPATH, it would explain this behavior.

EDIT 2:

This still-open report looks like an excellent candidate for having introduced the observed behavior:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=859732

(discovered through comments on use RPATH but not RUNPATH? )

EDIT 3:

On a commenter's advice, I reexamined updating the ld configuration before building the extension and that resolved the issue! I had tried this before, but must have overlooked something between build attempts:

# echo /opt/oracle/instantclient_12_2 > /etc/ld.so.conf.d/oracle-instantclient.conf
# ldconfig

I still don't know why LD_LIBRARY_PATH doesn't work as it should in this instance, but adding the Instant Client library path to the linker configuration seems a better approach besides.

EDIT 4:

I stated in my previous edit that modifying the ldconfig constitutes a better approach, but came to realize (on a commenter's good advice) that doing so can cause undesirable library conflicts, because the effects are system-wide.

In hindsight, it makes sense to minimize the "collateral damage" from runtime library linkage modifications by limiting them to the execution environment via the LD_LIBRARY_PATH. Accordingly, I am motivated to determine why this does not work on Ubuntu 18.04 LTS.

I feel that I have established definitively that the PHP-FPM daemon ignores LD_LIBRARY_PATH on Ubuntu (and has since at least Ubuntu 16.04 LTS; see Comments for explanation).

The ld.so(8) manpage states (in relation to the order in which runtime library paths are searched):

Using the environment variable LD_LIBRARY_PATH (unless the executable is being run in secure-execution mode; see below). [sic] in which case it is ignored.

As yet, I cannot think of any other reason for which the path would be ignored. Of secure-execution mode, the same document says:

 Secure-execution mode
       For  security reasons, the effects of some environment variables are voided or modified if the dynamic linker determines that the binary
       should be run in secure-execution mode.  (For details, see the discussion of individual environment variables below.)  A binary is  exe‐
       cuted  in  secure-execution  mode if the AT_SECURE entry in the auxiliary vector (see getauxval(3)) has a nonzero value.  This entry may
       have a nonzero value for various reasons, including:

       *  The process's real and effective user IDs differ, or the real and effective group IDs differ.  This typically occurs as a  result  of
          executing a set-user-ID or set-group-ID program.

       *  A process with a non-root user ID executed a binary that conferred capabilities to the process.

       *  A nonzero value may have been set by a Linux Security Module.

Firstly, Secure-Execution Mode seems not to be in effect, as the PHP executables don't exhibit this flag (AT_SECURE is 0):

LD_SHOW_AUXV=1 /usr/sbin/php-fpm7.1 -daemonize --fpm-config /etc/php/7.1/fpm/php-fpm.conf
AT_SYSINFO_EHDR: 0x7ffc569e1000
AT_HWCAP:        178bfbff
AT_PAGESZ:       4096
AT_CLKTCK:       100
AT_PHDR:         0x55ceab0c4040
AT_PHENT:        56
AT_PHNUM:        9
AT_BASE:         0x7f823c77f000
AT_FLAGS:        0x0
AT_ENTRY:        0x55ceab19e360
AT_UID:          0
AT_EUID:         0
AT_GID:          0
AT_EGID:         0
AT_SECURE:       0
AT_RANDOM:       0x7ffc56962349
AT_HWCAP2:       0x0
AT_EXECFN:       /usr/sbin/php-fpm7.1
AT_PLATFORM:     x86_64

It occurred to me that the child FPM pool processes might exhibit different AT_SECURE values, but the output is identical for the PHP-FPM daemon itself, as well as any child processes. The parent and the children all have the following values:

# od -t d8 /proc/851/auxv
0000000                   33      140722944548864
0000020                   16            395049983
0000040                    6                 4096
0000060                   17                  100
0000100                    3       93903778242624
0000120                    4                   56
0000140                    5                    9
0000160                    7      140365152313344
0000200                    8                    0
0000220                    9       93903779136352
0000240                   11                    0
0000260                   12                    0
0000300                   13                    0
0000320                   14                    0
0000340                   23                    0
0000360                   25      140722944193929
0000400                   26                    0
0000420                   31      140722944196579
0000440                   15      140722944193945
0000460                    0                    0

Secondly, none of these reasons seem to apply, given the following:

1) There is no indication that PHP-FPM or its child processes have real and effective user or group IDs that differ (thanks to https://unix.stackexchange.com/a/202359 for this command):

# ps -e -o user= -o ruser= | awk '$1 != $2'
systemd+ systemd-timesync
systemd+ systemd-resolve
beansta+ beanstalkd
message+ messagebus
daemon   root
systemd+ systemd-network

# ps -e -o group= -o rgroup= | awk '$1 != $2'
systemd+ systemd-timesync
systemd+ systemd-resolve
beansta+ beanstalkd
message+ messagebus
daemon   root
systemd+ systemd-network

2) The binaries in question do not have any capabilities (the following commands produce no output):

# getcap /usr/lib/php/20170718/oci8.so
# getcap -r /opt/oracle/instantclient_12_2/

3) I have ensured that AppArmor is disabled (it doesn't have a policy that should affect PHP-FPM, anyway):

# systemctl disable apparmor
Synchronizing state of apparmor.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable apparmor
# reboot
# aa-status
apparmor module is loaded.
0 profiles are loaded.
0 profiles are in enforce mode.
0 profiles are in complain mode.
0 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

So, why does PHP-FPM ignore LD_LIBRARY_PATH, if not for any of the aforementioned reasons?

EDIT 5 (Solution):

An astute commenter, @vinc17 , points-out that on systems running systemd, environment variables, such as LD_LIBRARY_PATH, are not necessarily propagated to processes that are started via a systemd Unit.

In other words, PHP-FPM isn't "ignoring" LD_LIBRARY_PATH, but rather, it is not being conveyed to the process. And attempts to set LD_LIBRARY_PATH within the PHP-FPM configuration are futile, because it's too late to do anything useful with the value.

On this advice, it occurred to me to set LD_LIBRARY_PATH in the systemd context, namely, in the Unit file(s) that start the PHP-FPM daemon(s), in which case PHP-FPM loads the OCI8 extension successfully.

Needless to say, we want to avoid editing the package maintainer's file (to avoid conflicts during future upgrades), so we extend it instead:

# mkdir /etc/systemd/system/php7.1-fpm.service.d
# touch /etc/systemd/system/php7.1-fpm.service.d/environment.conf

To this file we add the following:

[Service]
Environment=LD_LIBRARY_PATH=/opt/oracle/instantclient_12_2

And to make the change effective:

# systemctl daemon-reload
# systemctl restart php7.1-fpm

For a more complete example, which addresses multiple co-installed PHP versions, please see my post at https://github.com/oerdnj/deb.sury.org/issues/865#issuecomment-395441936 .

Answer 1

First, Debian bug 859732 is a completely different issue (I would even say an opposite issue): for this bug, several versions of the library are present in the search path (one in some directory specified by LD_LIBRARY_PATH and a different one in some directory specified by the run path), but the wrong one is chosen by the dynamic linker.

In your case, the problem is that the requested library is not found anywhere in the search path. Note also that in your case, it is PHP that seems to try to open the library (via dlopen?), since the message starts with "PHP Warning:". However, it seems that the mechanisms are the same as with usual dynamic linking.

After installing the library, what you need is at least one of:

• Nothing special if the library has been installed in a directory that is searched by default. Since you get an error, this is not your case.
• Providing the directory in a run path, which must be specified at compile time of the software that will need the library. The problem is that under Linux, this is not done in standard by the build tools, and it may be complex to do it right without breaking other things. However, in the context of dlopen, the software (here, PHP) may have set up what one can call a "plugin search path", where you can put your libraries.
• Providing the directory in LD_LIBRARY_PATH. This is what you tried, but your LD_LIBRARY_PATH seems incorrect. Libraries are usually installed in subdirectories named lib (or lib32 or lib64 in specific cases). So, export LD_LIBRARY_PATH=/opt/oracle/instantclient_12_2 seems wrong. Search for the full pathname of the library oci8.so, and just take the directory part of this pathname for LD_LIBRARY_PATH.

Note: strace may be useful to see what directories are considered to search for the libraries. EDIT: ldd and objdump -p are other useful tools to find what's going on with the search paths.

EDIT 2: Another point to note when choosing to use a run path is that indirect library dependencies are found when RPATH is used, but not when RUNPATH is used (so, in this latter case, all dependencies also need to have a run path if they depend on other libraries so that all libraries can be found without resorting to LD_LIBRARY_PATH). This is documented in recent versions of the ld.so(8) man page:

Using the directories specified in the DT_RUNPATH dynamic section attribute of the binary if present. Such directories are searched only to find those objects required by DT_NEEDED (direct dependencies) entries and do not apply to those objects' children, which must themselves have their own DT_RUNPATH entries. This is unlike DT_RPATH, which is applied to searches for all children in the dependency tree.

This is probably why, without using LD_LIBRARY_PATH, this was working with 16.04 (where RPATH is used) but not with 18.04 (where RUNPATH is used).