|
|
| (32 intermediate revisions by 2 users not shown) |
| Line 1: |
Line 1: |
| − | == Stuff you wished to know before you every needed to touch the OpenSSL library == | + | == Intro == |
| | + | |
| | + | === Stuff you wished to know before you every needed to touch the OpenSSL library === |
| | | | |
| | This page is constructed as a personal '''braindump''' to be able to share some point of reference with those involved with OpenSSL. On the web OpenSSL is poorly documented and even the book '''Network Security with OpenSSL''' doesn't touch the advanced inner workings of the library. The library and its CLI tools are a part of our everyday Grid life and working with it, as intimately as developing callback functions and home-brew proxy certificate verification routines, has let me research how stuff works deeply from within the rabbit hole of the library itself. | | This page is constructed as a personal '''braindump''' to be able to share some point of reference with those involved with OpenSSL. On the web OpenSSL is poorly documented and even the book '''Network Security with OpenSSL''' doesn't touch the advanced inner workings of the library. The library and its CLI tools are a part of our everyday Grid life and working with it, as intimately as developing callback functions and home-brew proxy certificate verification routines, has let me research how stuff works deeply from within the rabbit hole of the library itself. |
| | | | |
| − | | + | === Use the source! === |
| − | '''I'd like to invite everybody who reads this to contribute tips, tricks, {code,wiki}-patches, need-to-knows, pitfalls, quirks, interesting routines, &c to this page directly or for external to Nikhef people through my email address.'''
| |
| − | | |
| − | | |
| − | You can contact me via email: okoeroo apestaartje nikhef punt nl.
| |
| − | | |
| − | == Tip #1 == | |
| | | | |
| | When in true doubt: Use the source! When you're '''looking for anything particular or specialized''' and you really wish to know how OpenSSL is handling this: '''Use the Source!'''. There are more gems in the documentation of the OpenSSL source tarballs and it's easily grep-able. | | When in true doubt: Use the source! When you're '''looking for anything particular or specialized''' and you really wish to know how OpenSSL is handling this: '''Use the Source!'''. There are more gems in the documentation of the OpenSSL source tarballs and it's easily grep-able. |
| | | | |
| | + | == Down the rabbit hole... == |
| | | | |
| − | == Background information == | + | === Background information === |
| | | | |
| | This section will have all kinds of background information regarding OpenSSL, Proxy certificates, CAs, formatting details and other (hopefully) useful tidbits. | | This section will have all kinds of background information regarding OpenSSL, Proxy certificates, CAs, formatting details and other (hopefully) useful tidbits. |
| | | | |
| | + | Go to: [[How to handle OpenSSL and not get hurt background information]] |
| | | | |
| − | === OpenSSL oneline format === | + | === What does that library call really do? === |
| | | | |
| − | OpenSSL uses a more readable format then RFC2253 (aka X500Name). The format is a defacto for wide variety of tools, services and infrastructures. To the best of my knowledge it was Globus which adopted the format for the '''grid-mapfile'''. This file authorizes users based on their certificate's subject DN and specifies a one-to-one mapping between a subject DN and a local Unix system account. In the time IBM has adopted it for their Grid middleware as many others have followed in the footsteps of Globus.
| + | So, how does that library call REALLY work and what am I expect to do? To free, or not to free, that is the question... |
| | | | |
| − | To see the OpenSSL oneline format at work, you'll have to dive into the command line and try it to see both the Subject DN and the Issuer DN:
| + | Go to: [[How to handle OpenSSL and not get hurt and what does that library call really do?]] |
| | | | |
| − | Vision:~ okoeroo$ openssl x509 -noout -in usercert.pem -subject
| + | Also added some OCSP related information. |
| − | subject= /O=dutchgrid/O=users/O=nikhef/CN=Oscar Koeroo
| |
| − | issuer= /C=NL/O=NIKHEF/CN=NIKHEF medium-security certification auth
| |
| | | | |
| | + | === Interesting OpenSSL CLI need-to-knows === |
| | | | |
| − | The output shows DN for both the issuer and the subject. In the oneline format the Relative Distinguished Name (RDN). RDNs are separated by slash-signs. Although a DN MUST be read as is, in the oneline-format the left-most RDN is the most significant RDN when taking the build-up of its sub-components into account. This differs per printable string format.
| + | This page hold all kinds of OpenSSL CLI tool tips and tricks. |
| | | | |
| − | Each RDN is a attribute and multi-valued pair. The attribute name is a registered OID. Usually only one value is associated to each attribute. The value is a UTF8String in the ASN.1 sequence, though commonly only ASCII characters are used because a lot of middleware is unable to handle UTF8 characters. The oneline notation describes the binding between an OID and a value with a equal-sign. The separator between multiple values is the plus-sign.
| + | Go to: [[How to handle OpenSSL and not get hurt using the CLI]]. |
| | | | |
| | + | == Contact and contribute == |
| | | | |
| − | Less information can be found here: http://www.openssl.org/docs/apps/x509.html#item_oneline
| + | I'd like to invite everybody who reads this to contribute tips, tricks, {code,wiki}-patches, need-to-knows, pitfalls, quirks, interesting routines, &c to this page directly or for external to Nikhef people through my email address. |
| | | | |
| − | === The Distinguished Name in detail ===
| + | You can contact me via email: okoeroo apestaartje nikhef punt nl. |
| − | | |
| − | The DN is a sequence of Relative Distinguished Names (RDNs), described in RFC2253 (though superseeded by another one that I need to lookup). The [http://www.igtf.net/ International Grid Trust Federation (IGTF)], its PMAs ([http://www.eugridpma.org/ European Policy Management Authority (EUGridPMA)], [http://www.apgridpma.org/ Asia Pacific Grid Policy Management Authority (APGridPMA)] and [http://www.tagpma.org/ The Americas Grid Policy Management Authority] (TAGPMA)) and [http://www.tacar.org/ TERENA Academic CA Repository (TACAR)] have scoped the usage of the available RDNs in CA due to the fact that different printable string representations of the same RDN Object Identifiers (OID) have been used or that the same printable string has been associated to different registered OIDs. Which makes the practical usage between inter-operating middleware which can be based in Java, C, Python, Perl, OpenSSL, Bouncycastle, GnuTLS and homebrew solutions impossible.
| |
| − | | |
| − | The DN is build up based upon the openssl.cnf file used by the CA during the signing process. Each CA SHOULD use a private namespace and declare the namespace publicly. CAs, like those in the [http://www.igtf.net/ International Grid Trust Federation (IGTF)], will not overlap in their namespaces. Commercial CAs tend to have a declared name space of '''.*''', hence they do not limit their signing namespace by any shape or form. This makes certificates from commercial CAs inherently non-unique and therefore not very usable in a global Grid environment without additional policies in place.
| |
| − | | |
| − | === The general 'how do (proxy) certificates work' and 'what is a proxy certificate' ===
| |
| − | | |
| − | When (identity) delegation are used in a world full of X.509 certificates it means that you are using proxy certificates. Proxy certificates come in various shapes, forms and types. Some of the basics properties are consistent:
| |
| − | | |
| − | * It's an X.509 certificate
| |
| − | * Has a public and private key pair.
| |
| − | * The first delegated proxy certificate is signed by an End-Entity Certificate (EEC). An EEC means your personal certificate.
| |
| − | * Proxy certificates can signed and follow other proxy certificates
| |
| − | | |
| − | A certificate chain with one delegation, will be constructed like this:
| |
| − | # CA certificate
| |
| − | # EEC (personal, server or service certificate)
| |
| − | # proxy certificate
| |
| − | | |
| − | | |
| − | In short, the construction steps (certificate "Genesis") to a proxy certificate:
| |
| − | * The CA operator creates a key pair and a signing request for the root CA certificate
| |
| − | * The CA operator (self-)signs its own request with its own private key
| |
| − | * A user creates a key pair, a signing request and send that off to a CA operator
| |
| − | * The CA operator performs identity vetting on the user and its signing request
| |
| − | * The CA operator creates the certificate from the signing request, signs it with the CA's private key and returns a brand new certificate to the user
| |
| − | * The user can use {grid,voms}-proxy-init to generate a new keypair, signing request for the proxy certificate and sign the proxy certificate with his/her private key.
| |
| − | | |
| − | Adding another proxy certificate layer is a repeat of the final step, but instead of using the user's certificate and private key, the certificate and private key of the latest created proxy certificate are used to create the next delegation and sign that proxy certificate.
| |
| − | | |
| − | | |
| − | A more complex certificate chain:
| |
| − | # CA certificate
| |
| − | # intermediate or sub-CA certificate
| |
| − | # yet another intermediate or sub-CA certificate
| |
| − | # EEC (personal, server or service certificate)
| |
| − | # proxy certificate
| |
| − | # another proxy certificate
| |
| − | | |
| − | == What does that library call really do? ==
| |
| − | | |
| − | === X509_STORE_CTX_get_chain() vs. X509_STORE_CTX_get1_chain() ===
| |
| − | There exists '''X509_STORE_CTX_get_chain(X509_STORE_CTX *ctx)''' and '''X509_STORE_CTX_get1_chain(X509_STORE_CTX *ctx)'''. The difference is that the '''X509_STORE_CTX_get_chain()''' will only return a reference to the certificate chain (type '''STACK_OF (X509) *''') from within the X509_STORE_CTX structure and the '''X509_STORE_CTX_get1_chain()''' will make a duplicate certificate chain that will need to be '''free()''''d.
| |
| − | | |
| − | === Proper memory liberation of a STACK_OF (X509) * ===
| |
| − | Freeing a '''STACK_OF (X509) *''' is not to be done with just '''free()'''. That will create a memory leak. Also a '''X509_free (X509) *''' and using a '''STACK_OF (X509) *''' as input will create a memory leak. The proper way to free a '''STACK_OF (X509) *''' is to use '''sk_X509_pop_free(st, free_func)''', where for '''free_func''' you should use '''X509_free'''. Example: '''sk_X509_pop_free(chain, X509_free)'''.
| |
| − | Warning: do not get confused with '''sk_X509_free()''' which will accept the '''STACK_OF (<type>) *''', but does not ''pop'' the stack to free all the individual certificates of the chain (which will be equal to an instant memory leakage for any certificate chain longer then one certificate).
| |
| − | | |
| − | === Getting the oneline notation of a Subject DN and Issuer DN (properly) ===
| |
| − | To get the '''issuer DN''' and '''subject DN''' you can use constructions like:
| |
| − | char * cert_DN = X509_NAME_oneline (X509_get_subject_name (cert), NULL, 0);
| |
| − | char * issuer_DN = X509_NAME_oneline (X509_get_issuer_name (cert), NULL, 0);
| |
| − | or
| |
| − | char cert_DN[255];
| |
| − | char issuer_DN[255];
| |
| − | X509_NAME_oneline (X509_get_subject_name (cert), cert_DN, 255);
| |
| − | X509_NAME_oneline (X509_get_issuer_name (cert), issuer_DN, 255);
| |
| − | | |
| − | In the first construction you'll need to free both the cert_DN and the issuer_DN. In the second example a static buffer is used and filled by the X509_NAME_oneline() routines.
| |
| − | | |
| − | === Verification depth extensions and supporting unlimited proxy certificate delegations ===
| |
| − | | |
| − | OpenSSL uses a default depth of 9 (don't ask why, it just is).
| |
| − | | |
| − | To cope with Subordinate CAs we have to extend the verification depth to be able to hold the certificate chain (could contain a lot of delegations) and all the CA certificate, which might not be added to the certificate chain itself but would still be lingering in the X509 CA directory lookup functions.
| |
| − | | |
| − | == Interesting OpenSSL CLI need-to-knows ==
| |
| − | | |
| − | === s_client foo ===
| |
| − | | |
| − | example: openssl s_client -connect 127.0.0.1:13050 -msg -nbio -ssl3 -CApath ~/dvl/ca/ -cert ~/dvl/ca/newcert.pem -key ~/dvl/ca/newkey.pem
| |
| − | | |
| − | | |
| − | ==== -CAfile vs. -CApath ====
| |
| − | | |
| − | Using the -CAfile <specific CA file> will send this certificate over the wire to the server-side. This will typically fail the verification of the certificate chain at the server-side, because it is not allowed to transfer the self-signed certificates. The trust-anchors should be installed at the service, not transfered by the client (for obvious reasons). The -CAfile <file> will also be used for the verification of the server-side certificate, but it's safer to use the -CApath <path to one or more CA certificates> option.
| |
| − | | |
| − | | |
| − | ==== How to calculate the hash value used by CA file names ====
| |
| − | | |
| − | OpenSSL CLI and the OpenSSL library functions will search in a default path and/or a given path to the needed (installed) CA files when it needs to verify a certificate chain. By convention a client (and server) will never provide the (final) CA certificate to the connected peer. The trust in the peer certificate (chain) has to be completed by adding the CA certificate(s) to the chain for it to verify completely.
| |
| − | | |
| − | This means that the OpenSSL CLI tool and/or library functions need to search one or more (stated) paths for the use CA files by the peers. OpenSSL will search in the -CApath directory by the hash of the used CA.
| |
| − | | |
| − | Run the following command:
| |
| − | openssl x509 -hash -noout -in cacert.pem
| |
| − | 0e52ca4f
| |
| − | | |
| − | Copy or rename the cacert.pem file to '''0e52ca4f.0'''. The '''.0''' indicates that it is the root CA. A '''.1''' extension indicates a subordinate CA. The '''.1''' doesn't always work because of differences in the OpenSSL implementations between its versions. Also Java libraries will handle the '''.1''' each in a different way.
| |
| − | | |
| − | | |
| − | | |
| − | | |
| − | ==== Using proxy certificates and s_client ====
| |
| − | | |
| − | Setting up a mutually authenticated SSL connection means that you'll receive the server-side certificate for your (automated) verification and that your (client-side) credentials are to be passed over to the server-side for authentication (in that order).
| |
| − | | |
| − | To perform a proper verification of the certificate chain, the server-side must be capable of constructing the complete certificate chain and through a challenge-response mechanism the private key used must fit the final certificate. When using a complex certificate chain where only the CA certificates are distributed to the service nodes a client must send its EEC and proxy certificates to the server-side.
| |
| − | | |
| − | To send the EEC and the proxy to the server-side with OpenSSL s_client you can use -CAfile to fill the gap in the certificate chain:
| |
| − | openssl s_client -connect 127.0.0.1:13050 \
| |
| − | -debug -state -nbio \
| |
| − | -CApath /etc/grid-security/certificates/
| |
| − | '''-CAfile $HOME/.globus/usercert.pem''' \
| |
| − | -cert /tmp/x509up_u501 \
| |
| − | -key /tmp/x509up_u501
| |
| − | | |
| − | The '''-cert''' is used for the proxy certificate, '''-key''' for the private key (which is in the same file as the proxy certificate), the '''-CAfile''' is used for your EEC (your personal certificate) and the '''-CApath''' refers to the directory filled with root CA certificates. The -debug, -state and -nbio are not relevant here.
| |
| − | | |
| − | | |
| − | === Downloading the host, service or user certificate from an OpenSSL session ===
| |
| − | | |
| − | Copy and execute the following little script. Rename the SERVER and PORT variable and you will end up with the certificate of the service in PEM format on file.
| |
| − | | |
| − | #!/bin/sh
| |
| − | OPENSSL=openssl
| |
| − | SED=sed
| |
| − | CAT=cat
| |
| − | TIMEOUT=1
| |
| − | SERVER=kuiken.nikhef.nl
| |
| − | PORT=8443
| |
| − | CERTDIR=/etc/grid-security/certificates/
| |
| − | PROXY_FILE=/tmp/x509up_u501
| |
| − | TMP_FILE=/tmp/cert_fetch.tmp
| |
| − | OUTPUTFILE=certificate_${SERVER}_${PORT}.pem
| |
| − | $OPENSSL s_client -connect ${SERVER}:${PORT} -CApath ${CERTDIR} -key ${PROXY_FILE} -CAfile ${PROXY_FILE} -cert ${PROXY_FILE} -showcerts > ${TMP_FILE} &
| |
| − | PID=$!
| |
| − | sleep $TIMEOUT
| |
| − | kill -TERM $PID 2>/dev/null
| |
| − | $CAT $TMP_FILE | $SED -ne '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' > $OUTPUTFILE
| |
| − | echo The host certificate is written in: $OUTPUTFILE
| |
| − | exit 0
| |
Intro
Stuff you wished to know before you every needed to touch the OpenSSL library
This page is constructed as a personal braindump to be able to share some point of reference with those involved with OpenSSL. On the web OpenSSL is poorly documented and even the book Network Security with OpenSSL doesn't touch the advanced inner workings of the library. The library and its CLI tools are a part of our everyday Grid life and working with it, as intimately as developing callback functions and home-brew proxy certificate verification routines, has let me research how stuff works deeply from within the rabbit hole of the library itself.
Use the source!
When in true doubt: Use the source! When you're looking for anything particular or specialized and you really wish to know how OpenSSL is handling this: Use the Source!. There are more gems in the documentation of the OpenSSL source tarballs and it's easily grep-able.
Down the rabbit hole...
Background information
This section will have all kinds of background information regarding OpenSSL, Proxy certificates, CAs, formatting details and other (hopefully) useful tidbits.
Go to: How to handle OpenSSL and not get hurt background information
What does that library call really do?
So, how does that library call REALLY work and what am I expect to do? To free, or not to free, that is the question...
Go to: How to handle OpenSSL and not get hurt and what does that library call really do?
Also added some OCSP related information.
Interesting OpenSSL CLI need-to-knows
This page hold all kinds of OpenSSL CLI tool tips and tricks.
Go to: How to handle OpenSSL and not get hurt using the CLI.
Contact and contribute
I'd like to invite everybody who reads this to contribute tips, tricks, {code,wiki}-patches, need-to-knows, pitfalls, quirks, interesting routines, &c to this page directly or for external to Nikhef people through my email address.
You can contact me via email: okoeroo apestaartje nikhef punt nl.
