How to handle OpenSSL and not get hurt and what does that library call really do?

X509_STORE_CTX_get_chain() vs. X509_STORE_CTX_get1_chain()

There exists X509_STORE_CTX_get_chain(X509_STORE_CTX *ctx) and X509_STORE_CTX_get1_chain(X509_STORE_CTX *ctx). The difference is that the X509_STORE_CTX_get_chain() will only return a reference to the certificate chain (type STACK_OF (X509) *) from within the X509_STORE_CTX structure and the X509_STORE_CTX_get1_chain() will make a duplicate certificate chain that will need to be free()'d.

Proper memory liberation of a STACK_OF (X509) *

Freeing a STACK_OF (X509) * is not to be done with just free(). That will create a memory leak. Also a X509_free (X509) * and using a STACK_OF (X509) * as input will create a memory leak. The proper way to free a STACK_OF (X509) * is to use sk_X509_pop_free(st, free_func), where for free_func you should use X509_free. Example: sk_X509_pop_free(chain, X509_free). Warning: do not get confused with sk_X509_free() which will accept the STACK_OF (<type>) *, but does not pop the stack to free all the individual certificates of the chain (which will be equal to an instant memory leakage for any certificate chain longer then one certificate).

Getting the oneline notation of a Subject DN and Issuer DN (properly)

To get the issuer DN and subject DN you can use constructions like:

char * cert_DN   = X509_NAME_oneline (X509_get_subject_name (cert), NULL, 0);
char * issuer_DN = X509_NAME_oneline (X509_get_issuer_name (cert), NULL, 0);

or

char cert_DN[255];
char issuer_DN[255];
X509_NAME_oneline (X509_get_subject_name (cert), cert_DN, 255);
X509_NAME_oneline (X509_get_issuer_name (cert), issuer_DN, 255);

In the first construction you'll need to free both the cert_DN and the issuer_DN. In the second example a static buffer is used and filled by the X509_NAME_oneline() routines.

Verification depth extensions and supporting unlimited proxy certificate delegations

OpenSSL uses a default depth of 9 (don't ask why, it just is).

To cope with Subordinate CAs we have to extend the verification depth to be able to hold the certificate chain (could contain a lot of delegations) and all the CA certificate, which might not be added to the certificate chain itself but would still be lingering in the X509 CA directory lookup functions.

After setting up a X509_STORE object, by adding the lookup functions for the CA directory, adding the verification callback function (for proper proxy support), and setting the proper flags... you can create a X509_STORE_CTX object from it with X509_STORE_CTX_new(). Then you can initialize the X509_STORE_CTX object with the certificate chain and the call to X509_STORE_CTX_init(). After setting the purpose to the proper setting and the extra flag if the OpenSSL version is higher then 0.9.8, we can finally get to the chase and upgrade the verification depth to something more usefull.

Our solution (for now) is to upgrade the depth to be the number of certificates in the chain, plus the OpenSSL default verification depth.

Use X509_STORE_CTX_set_depth() to fit the certificate chain, sub-CAs and root CA into it. We've chosen to check for the certificate chain's depth, plus the OpenSSL of 9. The X509_STORE_CTX_set_depth() is a macro in OpenSSL 0.9.7a and it's implemented as a wrapper function in OpenSSL 0.9.8a.

/* Alter verification depth to fit the certificate chain, sub-CAs and root CA */
X509_STORE_CTX_set_depth (verify_ctx, depth + VERIFICATION_CHAIN_DEPTH_EXTENSION);

Path length constraints

The OpenSSL X509 structure has a field called ex_pathlen which holds the parsed pathlen value of the certificate. It means that beyond this point the depth can not exceed the Path Length value.

Code example:

X509 * certificate;
certificate->ex_pathlen

X509_V_ERR_PATH_LENGTH_EXCEEDED

This error gets triggered when a CA certificate indicates to have a path length constraint set and the certificate chain to evaluate violates the set limitation. When set in a CA certificate this means that you can only have pathlen sub-CA.

Example OK:

Root CA (no path len constaint)
      \_ Sub CA (pathlen = 1)
                   \_ Sub Sub CA (no path len constraint)
                                      \_ EEC (client or host certificate)

Example in error:

Root CA (no path len constaint)
      \_ Sub CA (pathlen = 0)
                   \_ Sub Sub CA (no path len constraint)
                                      \_ EEC (client or host certificate)