cURL not working (Error #77) for SSL connections on CentOS for non-root users

Question

Just recently my server has stopped working for curl requests to https:// addresses for my web server. Having dug around a little it appears that it's a problem with the user the webserver is running.

If I SSH onto the server as root & call

curl -I -v https://google.com 

... I get the following response...

* About to connect() to google.com port 443 (#0) *   Trying 173.194.67.113... connected * Connected to google.com (173.194.67.113) port 443 (#0) * Initializing NSS with certpath: sql:/etc/pki/nssdb *   CAfile: /etc/pki/tls/certs/ca-bundle.crt   CApath: none * SSL connection using SSL_RSA_WITH_RC4_128_SHA * Server certificate: *       subject: CN=*.google.com,O=Google Inc,L=Mountain View,ST=California,C=US *       start date: May 22 15:50:20 2013 GMT *       expire date: Oct 31 23:59:59 2013 GMT *       common name: *.google.com *       issuer: CN=Google Internet Authority,O=Google Inc,C=US > HEAD / HTTP/1.1 > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 > Host: google.com > Accept: */* 

However, if I log in as any of the cPanel accounts (also used when running via the web server) I get the following...

* About to connect() to google.com port 443 (#0) *   Trying 173.194.67.101... connected * Connected to google.com (173.194.67.101) port 443 (#0) * Initializing NSS with certpath: none * NSS error -5978 * Closing connection #0 * Problem with the SSL CA cert (path? access rights?) curl: (77) Problem with the SSL CA cert (path? access rights?) 

I've not been able to find a definitive answer to the problem, & my hosting company are refusing to help as it's "Out of support" even though it was working fine last week!

I did find mention on http://curl.haxx.se/docs/sslcerts.html that

"If libcurl was built with NSS support, then depending on the OS distribution, it is probably required to take some additional steps to use the system-wide CA cert db. RedHat ships with an additional module, libnsspem.so, which enables NSS to read the OpenSSL PEM CA bundle. This library is missing in OpenSuSE, and without it, NSS can only work with its own internal formats. NSS also has a new database format: https://wiki.mozilla.org/NSS_Shared_DB"

... but I can find no information on how I get this working system-wide on my CentOS server.

Info

curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.14.0.0 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp  Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz  

Can anyone shed some light on why this might have suddenly changed, or better still how to fix it?

Thanks

Answer 1

I just had a similar problem with Error#77 on CentOS7. I was missing the softlink /etc/pki/tls/certs/ca-bundle.crt that is installed with the ca-certificates RPM.

'curl' was attempting to open this path to get the Certificate Authorities. I discovered with:

strace curl https://example.com 

and saw clearly that the open failed on that link.

My fix was:

yum reinstall ca-certificates 

That should setup everything again. If you have private CAs for Corporate or self-signed use make sure they are in /etc/pki/ca-trust/source/anchors so that they are re-added.

Answer 2

If you recently reached here as I did when searching for the same error in vain you may find it to be an update to NSS causing failure on CentOS. Test by running yum update and see if you get errors, curl also creates this error. Solution is simple enough just install NSS manually.

Read on...

If you're like me it threw up an error similar to this:

curl: (77) Problem with the SSL CA cert (path? access rights?) 

This took some time to solve but found that it wasn't the CA cert because by recreating them and checking all the configuration I had ruled it out. It could have been libcurl so I went in search of updates.

As mentioned I recreated CA certs. You can do this also but it may be a waste of time. http://wiki.centos.org/HowTos/Https

The next step (probably should of been my first) was to check that everything was up-to-date by simply running yum.

$ yum update $ yum upgrade 

This gave me an affirmative answer that there was a bigger problem at play: Downloading Packages: error: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: BAD Problem opening package nss-softokn-freebl-3.14.3–19.el6_6.x86_64.rpm I started reading about Certificate Verification with NSS and how this new update may be related to my problems. So yum is broken. This is because nss-softokn-* needs nss-softokn-freebl-* need each other to function. The problem is they don't check each others version for compatibility and in some cases it ends up breaking yum. Lets go fix things:

$ wget http://mirrors.linode.com/centos/6.6/updates/x86_64/Packages/nsssoftokn-freebl-3.14.3-19.el6_6.x86_64.rpm $ rpm -Uvh nss-softokn-freebl-3.14.3–19.el6_6.x86_64.rpm $ yum update 

You should of course download from your nearest mirror and check for the correct version / OS etc. We basically download and install the update from the rpm to fix yum. As @grumpysysadmin pointed out you can shorten the commands down. @cwgtex contributed that you should install the upgrade using the RPM command making the process even simplier.

To fix things with wordpress you need to restart your http server.

$ service httpd restart 

Try again and success!