Error: "invalid use of incomplete type ‘RSA {aka struct rsa_st}" in OpenSSL 1.1.0

Question

I have old code that was written to link against an old version of openssl. Part of this code loads a key from a PEM file, and tries to understand whether this key is a private or public key, by using the following code:

if( (prv->p==0 || prv->q==0) ) {
    // This is not a private key!
    throw error("No private key for decryption");
}

With the latest version of openssl, this (justifiably) doesn't compile:

crypto.cpp: In function ‘key* decrypt_header(file_t, RSA*)’:
crypto.cpp:158:13: error: invalid use of incomplete type ‘RSA {aka struct rsa_st}’
     if( (prv->p==0 || prv->q==0) ) {
             ^~

I understand that direct access to the struct's private members was replaced with a function, but I am having a hard time figuring out which function that is.

Answer 1

crypto.cpp:158:13: error: invalid use of incomplete type ‘RSA {aka struct rsa_st}’
     if( (prv->p==0 || prv->q==0) ) {
             ^~

As you are aware, OpenSSL 1.1.0 changed the visibility of a lot of struct members. You can no longer access the members directly. Instead, you have to use getter and setter functions.

Try RSA_get0_factors. The get0 means the reference counts are not incremented. Do not BN_free them.

void RSA_get0_factors(const RSA *r, const BIGNUM **p, const BIGNUM **q);

---

If the code supports multiple versions of OpenSSL, then you will need a guard because RSA_get0_factors is for OpenSSL 1.1.0 and above. Maybe something like the following. Also see OPENSSL_VERSION_NUMBER man page.

#include <openssl/opensslv.h>

#if OPENSSL_VERSION_NUMBER < 0x10100000L

    /* OpenSSL 1.0.2 and below (old code) */

#else

    /* OpenSSL 1.1.0 and above (new code) */

#endif

Answer 2

After 1.1.1 OpenSSL Supports getter what return each parameter like this.

const BIGNUM *RSA_get0_n(const RSA *d);
const BIGNUM *RSA_get0_e(const RSA *d);
const BIGNUM *RSA_get0_d(const RSA *d);
const BIGNUM *RSA_get0_p(const RSA *d);
const BIGNUM *RSA_get0_q(const RSA *d);
const BIGNUM *RSA_get0_dmp1(const RSA *r);
const BIGNUM *RSA_get0_dmq1(const RSA *r);
const BIGNUM *RSA_get0_iqmp(const RSA *r);

So if you don't need to considerate below version of OpenSSL less than 1.1.1 these code will make simple code. AND other structures support kind of this getter too. You can find more information of functions this. https://www.openssl.org/docs/man1.1.1/man3/

Answer 3

#if OPENSSL_VERSION_NUMBER < 0x10100005L
static void RSA_get0_key(const RSA *r, const BIGNUM **n, const BIGNUM **e, const BIGNUM **d, const BIGNUM **p, const BIGNUM **q)
{
    if(n != NULL)
        *n = r->n;

    if(e != NULL)
        *e = r->e;

    if(d != NULL)
        *d = r->d;

    if(p != NULL)
        *p = r->p;

    if(q != NULL)
        *q = r->q;
}
#endif

const BIGNUM *bn_p;
const BIGNUM *bn_q;

RSA_get0_key(key, NULL, NULL, NULL, &bn_p, &bn_q);
/*   if( (prv->p==0 || prv->q==0) ) { */
if( (prv_p==0 || prv_q==0) ) {