OpenSSL and Trusted System Certifcates

Question

So I already know how to specify locations for trusted certificates using SSL_CTX_load_verify_locations(). Now the documentation says the following:

SSL_CTX_load_verify_locations() specifies the locations for ctx, at which CA certificates for verification purposes are located. The certificates available via CAfile and CApath are trusted.

And also says:

When looking up CA certificates, the OpenSSL library will first search the certificates in CAfile, then those in CApath.

That's fine. But nothing is mentioned about the trusted system certificates residing in the OPENSSLDIR.

1. Are system certificates checked after both CAfile and CApath fail?
2. Does a call to SSL_CTX_set_default_verify_paths() override SSL_CTX_load_verify_locations()? Or do they work side by side, i.e., both trusted system certificates and the ones specified by CAfile and CApath?
3. If the certificates are manually added to the certificate store using SSL_CTX_get_cert_store(), i.e., SSL_CTX_load_verify_locations() is not called at all, what happens in that case? Are only the store certificates checked? Anyway to disable/enable checking trusted system certificates in that case?

Answer 1

Alright .. So I took a stab at it and found out what I needed to know.
Setup:
- Simple client and server with peer verification enabled on both sides
- I created two CAs. Let's call them SS (self-signed) and TR (trusted).
- SS was used to create a client (SS_C) and server (SS_S) certificates.
- TR was used to create a client (TR_C) and server (TR_S) certificates.
- The TR CA was hashed and added to the default CA directory.

Foolproof Test:
- openssl verify of SS_C and SS_S fail unless I specify the CAfile
- openssl verify of TR_C and TR_S succeed
Result as expected

Client/Server Basic Test:
- No verification paths calls: SS fails -- TR fails
- Call to SSL_CTX_set_default_verify_paths: SS fails -- TR succeeds
- Call to SSL_CTX_load_verify_locations with SS as CAfile: SS succeeds -- TR fails
Result as expected

---

Now, let's get to the more interesting stuff.
Calls to both SSL_CTX_set_default_verify_paths and SSL_CTX_load_verify_locations:
In this case, calls to SSL_CTX_load_verify_locations were always made with SS as the CAfile.
- TR succeeds -- irrespective of the order of the calls
- SS succeeds -- irrespective of the order of the calls
Interesting result -- at least to me
Now, I can expect that using the certificate store will work as well

Using the X509_STORE instead of SSL_CTX_load_verify_locations:
In this case, I created a byte array of the SS CA, got the context's certificate store, and added the SS CA to it.
- Only get context's store and add SS CA to it: SS succeeds -- TR fails
- Only get context's store (to test whether it adds the default trusted certificates): SS fails -- TR fails
- Call to SSL_CTX_set_default_verify_paths + get context's store and add SS CA to it: SS succeeds -- TR succeeds
Great .. it indeed works