How to handle OpenSSL and not get hurt

Stuff you wished to know before you every needed to touch the OpenSSL library

On the web OpenSSL is poorly documented and even the book Network Security with OpenSSL doesn't touch the advanced inner workings of the library. There are more gems in the documentation of the OpenSSL source tarballs.

Use the source! When you're looking for anything particular or specialized and you really wish to know how OpenSSL is handling this: Use the Source!

What does that library call really do?

There exists X509_STORE_CTX_get_chain(X509_STORE_CTX *ctx) and X509_STORE_CTX_get1_chain(X509_STORE_CTX *ctx). The difference is that the X509_STORE_CTX_get_chain() will only return a reference to the certificate chain (type STACK_OF (X509) *) from within the X509_STORE_CTX structure and the X509_STORE_CTX_get1_chain() will make a duplicate certificate chain that will need to be free()'d.

Freeing a STACK_OF (X509) * is not to be done with just free(). That will create a memory leak. Also a X509_free (X509) * and using a STACK_OF (X509) * as input will create a memory leak. The proper way to free a STACK_OF (X509) * is to use sk_X509_pop_free(st, free_func), where for free_func you should use X509_free. Example: sk_X509_pop_free(chain, X509_free). Warning: do not get confused with sk_X509_free() which will accept the STACK_OF (<type>) *, but does not pop the stack to free all the individual certificates of the chain (which will be equal to an instant memory leakage for any certificate chain longer then one certificate).

To get the issuer DN and subject DN you can use constructions like:

char * cert_DN   = X509_NAME_oneline (X509_get_subject_name (cert), NULL, 0);
char * issuer_DN = X509_NAME_oneline (X509_get_issuer_name (cert), NULL, 0);

or

char cert_DN[255];
char issuer_DN[255];
X509_NAME_oneline (X509_get_subject_name (cert), cert_DN, 255);
X509_NAME_oneline (X509_get_issuer_name (cert), issuer_DN, 255);

In the first construction you'll need to free both the cert_DN and the issuer_DN. In the second example a static buffer is used and filled by the X509_NAME_oneline() routines.

Interesting OpenSSL CLI need-to-knows

s_client foo

example: openssl s_client -connect 127.0.0.1:13050 -msg -nbio -ssl3 -CApath ~/dvl/ca/ -cert ~/dvl/ca/newcert.pem -key ~/dvl/ca/newkey.pem

-CAfile vs. -CApath

Using the -CAfile <specific CA file> will send this certificate over the wire to the server-side. This will typically fail the verification of the certificate chain at the server-side, because it is not allowed to transfer the self-signed certificates. The trust-anchors should be installed at the service, not transfered by the client (for obvious reasons). The -CAfile <file> will also be used for the verification of the server-side certificate, but it's safer to use the -CApath <path to one or more CA certificates> option.

How to calculate the hash used by CA files

OpenSSL CLI and the OpenSSL library functions will search in a default path and/or a given path to the needed (installed) CA files when it needs to verify a certificate chain. By convention a client (and server) will never provide the (final) CA certificate to the connected peer. The trust in the peer certificate (chain) has to be completed by adding the CA certificate(s) to the chain for it to verify completely.

This means that the OpenSSL CLI tool and/or library functions need to search one or more (stated) paths for the use CA files by the peers. OpenSSL will search in the -CApath directory by the hash of the used CA.

Run the following command:

openssl x509 -hash -noout -in cacert.pem
0e52ca4f

Copy or rename the cacert.pem file to 0e52ca4f.0. The .0 indicates that it is the root CA. A .1 extension indicates a subordinate CA. The .1 doesn't always work because of differences in the OpenSSL implementations between its versions. Also Java libraries will handle the .1 each in a different way.