What is the correct way to free output buffer after call to openssl::i2d_X509?

Question

I want to encode X509 structure into DER bytes. Following source code example from the openssl (version > 0.9.7) man page I need to do (if I would i2d_X509 to allocate memory on its own):

int len;
unsigned char *buf;

buf = NULL;

len = i2d_X509(x, &buf);

if (len < 0)
    /* error */

However, it is not completely clear (but I assume it is needed to call OPENSSL_free) from the documentation what is the right way to free memory after I am done with buf.

What is the correct way to free buf?

Answer 1

Short answer: OPENSSL_free must be used to free buf.

Long answer: IMPLEMENT_ASN1_FUNCTIONS macro is expanded to definition of i2d_X509 function. The example below demonstrates that, put following source code into a source.c:

#include <openssl/asn1t.h>
IMPLEMENT_ASN1_FUNCTIONS(X509)

After execution of gcc -E source.c the macro is expanded to:

X509 *d2i_X509(X509 **a, const unsigned char **in, long len) { return (X509 *)ASN1_item_d2i((ASN1_VALUE **)a, in, len, (&(X509_it))); }
int i2d_X509(X509 *a, unsigned char **out) { return ASN1_item_i2d((ASN1_VALUE *)a, out, (&(X509_it))); }
X509 *X509_new(void) { return (X509 *)ASN1_item_new((&(X509_it))); } 
void X509_free(X509 *a) { ASN1_item_free((ASN1_VALUE *)a, (&(X509_it))); }

The point of interest is definition of i2d_X509, in turn that function calls ASN1_item_i2d. As per source code of openssl, ASN1_item_i2d is a function defined in tasn_enc.c file:

static int asn1_item_flags_i2d(ASN1_VALUE *val, unsigned char **out,
                               const ASN1_ITEM *it, int flags)
{
    if (out && !*out) {
        unsigned char *p, *buf;
        int len;
        len = ASN1_item_ex_i2d(&val, NULL, it, -1, flags);
        if (len <= 0)
            return len;
        buf = OPENSSL_malloc(len);
        if (buf == NULL)
            return -1;
        p = buf;
        ASN1_item_ex_i2d(&val, &p, it, -1, flags);
        *out = buf;
        return len;
    }

    return ASN1_item_ex_i2d(&val, out, it, -1, flags);
}

The branch if (out && !*out) is used in a case described in the original question (buf is NULL). So, internally, openssl allocates memory for the buf using OPENSSL_malloc, and as a consequence OPENSSL_free must be used to deallocate memory.

Note: I looked at the source code of openssl available on the GH at the current time.