How to handle OpenSSL and not get hurt using the CLI
Back to top How to handle OpenSSL and not get hurt
Standard OpenSSL stuff
I'm not going to copy/paste everything in here which is supplied in a nice and orderly fashion elsewhere.
- Using the OpenSSL CLI tools: http://www.madboa.com/geek/openssl/
- Openssl configuration file foo: http://wiki.cacert.org/VhostTaskForce
s_client foo
example: openssl s_client -connect 127.0.0.1:13050 -msg -nbio -ssl3 -CApath ~/dvl/ca/ -cert ~/dvl/ca/newcert.pem -key ~/dvl/ca/newkey.pem
-CAfile vs. -CApath
Using the -CAfile <specific CA file> will send this certificate over the wire to the server-side. This will typically fail the verification of the certificate chain at the server-side, because it is not allowed to transfer the self-signed certificates. The trust-anchors should be installed at the service, not transfered by the client (for obvious reasons). The -CAfile <file> will also be used for the verification of the server-side certificate, but it's safer to use the -CApath <path to one or more CA certificates> option.
How to calculate the hash value used by CA file names
OpenSSL CLI and the OpenSSL library functions will search in a default path and/or a given path to the needed (installed) CA files when it needs to verify a certificate chain. By convention a client (and server) will never provide the (final) CA certificate to the connected peer. The trust in the peer certificate (chain) has to be completed by adding the CA certificate(s) to the chain for it to verify completely.
This means that the OpenSSL CLI tool and/or library functions need to search one or more (stated) paths for the use CA files by the peers. OpenSSL will search in the -CApath directory by the hash of the used CA.
Run the following command:
openssl x509 -hash -noout -in cacert.pem 0e52ca4f
Copy or rename the cacert.pem file to 0e52ca4f.0. The .0 indicates that it is the root CA. A .1 extension indicates a subordinate CA. The .1 doesn't always work because of differences in the OpenSSL implementations between its versions. Also Java libraries will handle the .1 each in a different way.
Using proxy certificates and s_client
Setting up a mutually authenticated SSL connection means that you'll receive the server-side certificate for your (automated) verification and that your (client-side) credentials are to be passed over to the server-side for authentication (in that order).
To perform a proper verification of the certificate chain, the server-side must be capable of constructing the complete certificate chain and through a challenge-response mechanism the private key used must fit the final certificate. When using a complex certificate chain where only the CA certificates are distributed to the service nodes a client must send its EEC and proxy certificates to the server-side.
To send the EEC and the proxy to the server-side with OpenSSL s_client you can use -CAfile to fill the gap in the certificate chain:
openssl s_client -connect 127.0.0.1:13050 \
-debug -state -nbio \
-CApath /etc/grid-security/certificates/ \
-CAfile $HOME/.globus/usercert.pem \
-cert /tmp/x509up_u501 \
-key /tmp/x509up_u501
The -cert is used for the proxy certificate, -key for the private key (which is in the same file as the proxy certificate), the -CAfile is used for your EEC (your personal certificate) and the -CApath refers to the directory filled with root CA certificates. The -debug, -state and -nbio are not relevant here.
Downloading the host, service or user certificate from an OpenSSL session
Copy and execute the following little script. Rename the SERVER and PORT variable and you will end up with the certificate of the service in PEM format on file.
#!/bin/sh
OPENSSL=openssl
SED=sed
CAT=cat
TIMEOUT=1
SERVER=kuiken.nikhef.nl
PORT=8443
CERTDIR=/etc/grid-security/certificates/
PROXY_FILE=/tmp/x509up_u501
TMP_FILE=/tmp/cert_fetch.tmp
OUTPUTFILE=certificate_${SERVER}_${PORT}.pem
$OPENSSL s_client -connect ${SERVER}:${PORT} -CApath ${CERTDIR} -key ${PROXY_FILE} -CAfile ${PROXY_FILE} -cert ${PROXY_FILE} -showcerts > ${TMP_FILE} &
PID=$!
sleep $TIMEOUT
kill -TERM $PID 2>/dev/null
$CAT $TMP_FILE | $SED -ne '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' > $OUTPUTFILE
echo The host certificate is written in: $OUTPUTFILE
exit 0
File Creation
Creating a CA
First create a random file, as random input seed. WARNING! I'm using /dev/urandom which is a speedy short cut. Do NOT use in production CAs!
SEED="private/.rand"
dd bs=4096 count=2048 if=/dev/urandom of=${SEED}
Create a new CA with sha1 for 5 years:
PRIVATE_KEY="cakey.pem"
PUBLIC_CA_CERT="cacert.pem"
DAYS=`echo "(365 + 1) * 5" | bc`
openssl req -new -x509 -sha1 -keyout ${PRIVATE_KEY} -rand ${SEED} -out ${PUBLIC_CA_CERT} -days ${DAYS}
Creating a CRL file
To create a CRL file you'll need the OpenSSL config file used by your CA to generate the CRL file:
CRL_OUT="output.r0"
CONFIG_FILE="config.file"
openssl ca -gencrl -config ${CONFIG_FILE} -out ${CRL_OUT}
File Conversion
A couple of file handling and format conversion examples.
Base64 to S/MIME p7s file to certificate chain
Grab the mail source and between the Content markers copy (only) the base64 to file and decode the base64:
openssl base64 -d -in raw.mail.content.data -out smime.p7s
After that step read the PKCS7 to show the certificate chain:
openssl pkcs7 -print_certs -inform DER -in /tmp/smime.p7s
Making a p12 file from pem
You can do this in many way and for different reasons. At this moment I want to have all the CA certificates in a p12 files to be used on my iPhone. I've downloaded the current CA distribution (in a classic, mics and slcs tarball) from the EUGridPMA website and used the following command to create a p12 file only for the CA certificates:
LINE="openssl pkcs12" ; for i in `ls *.0`; do LINE="$LINE -in $i"; done ; COMMAND="$LINE -cacerts -nokeys -export -out IGTF-classic-mics-slcs.p12"; $COMMAND
Making a PEM file from a p12 file
input: cert.p12 – certificate in .p12 format output: cert.pem – certificate in .pem format (use it as both X509_USER_KEY and X509_USER_CERT)
Only the certificate(s):
openssl pkcs12 -in cert.p12 -clcerts -out cert.pem
Certificate plus private key:
openssl pkcs12 -in cert.p12 -out cert.pem
Note: This option will also ask for a new Passphrase for the to be written private key.
Converting CRL from PEM to DER
Take the CRL file in PEM format as input, and the output will be in DER:
IN_CRL="input.r0"
OUT_CRL="output_in_der.r0"
openssl crl -in ${IN_CRL} -outform DER -out ${OUT_CRL}
Converting OpenSSH public keys to PEM
Goto: http://blog.oddbit.com/2011/05/converting-openssh-public-keys.html
Verification examples
How to verify your certificate, private key and other details to be valid, matching or properly formatted.
Show effective certificate content in a file
Show the public certificate
CERT=certificate.pem
openssl x509 -noout -in ${CERT} -text
Show certificate file by its split content
Goto https://github.com/okoeroo/certificate-handling and use the show-split-certs.py script. Example:
./show-split-certs.py /etc/grid-security/hostcert.pem ./show-split-certs.py <other file>
[okoeroo@localhost ~]$ ./show-split-certs.py mkproxy-x509-voms Checking file : mkproxy-x509-voms subject= /DC=org/DC=terena/DC=tcs/C=NL/O=Nikhef/CN=Oscar Koeroo okoeroo@nikhef.nl/CN=proxy issuer= /DC=org/DC=terena/DC=tcs/C=NL/O=Nikhef/CN=Oscar Koeroo okoeroo@nikhef.nl notBefore=Jan 26 09:19:43 2012 GMT notAfter=Jan 27 09:24:43 2012 GMT subject= /DC=org/DC=terena/DC=tcs/C=NL/O=Nikhef/CN=Oscar Koeroo okoeroo@nikhef.nl issuer= /C=NL/O=TERENA/CN=TERENA eScience Personal CA notBefore=Jul 4 00:00:00 2011 GMT notAfter=Aug 2 23:59:59 2012 GMT
Verifying your certificate against the installed CA certificates and CRLs
This is a basic check to verify your certificate:
# The usercert.pem is your user certificate file # The /etc/grid-security/certificates/ is the directory that holds the CA certificates and CRL files. Adjust this to another directory if needed. openssl verify -CApath /etc/grid-security/certificates/ usercert.pem
This can also be used for host certificates.
Matching the private key file with the public certificate file
The modulus and the public exponent portions in the key and the Certificate must match. But since the public exponent is usually 65537 and it's bothering comparing long modulus you can use the following approach:
openssl x509 -noout -modulus -in usercert.pem | openssl md5 openssl rsa -noout -modulus -in userkey.pem | openssl md5
And then compare these really shorter numbers. With overwhelming probability they will differ if the keys are different. As a "one-liner":
openssl x509 -noout -modulus -in usercert.pem | openssl md5 ;\ openssl rsa -noout -modulus -in userkey.pem | openssl md5
The usercert.pem is your user certificate file. This can also be used for host certificates.
grid-proxy-verify
A simple but powerful tool to diagnose a certificate chain with old style/classic or RFC3820 compliant proxy certificates.
http://www.nikhef.nl/~janjust/proxy-verify/
Some grid computing specific tips
Like the subsection tells you.
Proxy certificate generation with pure OpenSSL and Bash
Pure bash implementation of grid-proxy-init using only OpenSSL CLI tools and bash. It can construct old style/classic, gt3 and RFC3820 style proxies:
http://www.nikhef.nl/~janjust/proxy-verify/
Create an .lsc file for VOMS
Take the host certificate of the VOMS server and run it through the following code snippet:
HOSTCERT=/etc/grid-security/hostcert.pem
VONAME="MyVO"
VOMSDIR=/etc/grid-security/vomsdir
openssl x509 -noout -subject -issuer -in ${HOSTCERT} | cut -d " " -f 2- -s > ${VOMSDIR}/${VONAME}/${VONAME}.lsc
It will set the PEM file as input, set the VO name for which it works and sets the default VOMSDIR and write the openssl wrapped output into a file with the .lsc extension in the right location.
Walk your CA certificates
I've pushed a piece of code scribbles to GitHub that might be useful to view meta-data on the CAs installed:https://github.com/okoeroo/certificate-handling
The read_cert_chains.py will provide a nice humanly understandable ASCII-artish view on the installed CAs. By default it will look for .pem files in the /etc/grid-security/certificates/
"/C=GB/ST=Greater Manchester/L=Salford/O=Comodo CA Limited/CN=AAA Certificate Services"
| |-> File : AAACertificateServices.pem
| |-> Depth : 0
| |-> Not Before : 20040101000000Z
| |-> Not After : 20281231235959Z
| |-> Serial number : 1
| |-> Is valid : no
| #
\___| -> "/C=US/ST=UT/L=Salt Lake City/O=The USERTRUST Network/OU=http://www.usertrust.com/CN=UTN-USERFirst-Client Authentication and Email"
| |-> File : UTNAAAClient.pem
| |-> Depth : 1
| |-> Not Before : 20040101000000Z
| |-> Not After : 20281231235959Z
| |-> Serial number : 165470017604460690276855178217067231472
| |-> Is valid : no
| #
\___| -> "/C=NL/O=TERENA/CN=TERENA eScience Personal CA"
| |-> File : TERENAeSciencePersonalCA.pem
| |-> Depth : 2
| |-> Not Before : 20090518000000Z
| |-> Not After : 20281231235959Z
| |-> Serial number : 124943880517709597336269004402877481672
| |-> Is valid : no
| #
"/C=NL/O=NIKHEF/CN=NIKHEF medium-security certification auth"
| |-> File : NIKHEF.pem
| |-> Depth : 0
| |-> Not Before : 20010921000000Z
| |-> Not After : 20210209000000Z
| |-> Serial number : 0
| |-> Is valid : no
| #
DrSSL
DrSSL - "Diagnose your SSL" - is intended to setup an SSL connection, check all kinds of SSL and certificate related things, report about that and diagnose problems: https://github.com/okoeroo/drssl
Usage:
./drssl --help --help --host <host or IP> --port <port> - default is: 443 --4 (force IPv4 - default is system specific) --6 (force IPv6 - default is system specific)
--2 (use SSLv2) --3 (use SSLv3) --10 (use TLSv1.0) - the default --11 (use TLSv1.1) --12 (use TLSv1.2) --cafile <path to CA (bundle) file> --capath <path to CA directory> --cert <path to client certificate> --key <path to client private key file> --passphrase <passphrase to unlock the client private key file> --sni <TLS SNI (Server Name Indication) hostname>
Supports:
- TLS SNI (Server Name Indication)
- OCSP Stapling
- Client certificate
