Openssl: certificate verification fails when CApath argument is used in SSL_CTX_load_verify_locations API

Question

I am trying to establish a TLS connection to a server machine. I have created the root CA certificate and the server certificate using openssl CLI commands. I created the server certificate with common name same as its IP address. The common name of the root CA certificate is the FQDN of the server.

I am using openssl library APIs to establish connection to the server. I am using the API

int SSL_CTX_load_verify_locations(SSL_CTX *ctx, const char *CAfile, const char *CApath)

for setting the CA file look up path.

Everything works fine when I use the CAfile argument to specify the path of my CA file, leaving the CApath argument as NULL.

But if I use the CApath argument to specify the path to the directory containing the CA files, leaving the CAfile argument as NULL, the connection fails due to certificate verification error.

When I captured the packets using wireshark, I found that my client code is sending TLS response "Unknown CA" for the "server hello" from the server. I have used the same CA certificate file that I have used in the successful connection.

From my observation while exploring the openssl library source code, I infer that, in my case, the CA files are not being considered valid and hence not being loaded by the library APIs due to some unknown reason.

Can someone tell me the reason and, if possible, a solution for this issue?

Answer 1

I'm posting answer for my own question, because I spent a lot of time for finding a solution for this problem since I didn't get too much information regarding this issue from the internet. And I hope this will help someone else facing the similar issue.

If the CApath is not null, the directory pointed by CApath should contain valid CA certificates. And the most important thing is that the CA file name should be the subject name hash value.

Either the CA files can be renamed to their subject name hash values or softlinks to the CA files can be created with the name same as the CA subject name hash of the CA files.

c_rehash utility can be used to create the necessary links in the CApath. The syntax of this command is quite simple.

c_rehash <CApath>

c_rehash utility may not be available in all linux distros. In that case

openssl x509 -in <CA file name> -noout -subject_hash

can be used to generate the subject name hash(e.g. e5d93f80). Just append ".0" to this value and create a softlink with this name(e5d93f80.0) to the CA file. If there are more than one CA files with same the subject name hash value, their extensions should be different(e.g. e5d93f80.1). The search is performed in the ordering of the extension number.

opensssl has introduced this technique to reduce the the CA file look up time. Otherwise openssl may have to read all files in the CApath to find the matching CA file.