CILogon Pre-Pilot Work - Jenkins

Introduction

We used Jenkins together with OpenStack to build an automatically deploying CILogon instance. Jenkins VM Image templates are used to boot VMs with relevant software installed on it, while separate Jenkins jobs are used to carry out different stages of the service deployment. The following sections contain a brief overview of the structure of these Jenkins jobs together with a short description of their function. For more information consult the local jenkins instance.

Delegation Service

The Delegation Service is made up of a couple of major components: a myproxy-server, oa4mp server offering an oauth2 frontend to MyProxy, and a Shibboleth SP. Together these components can issue certificates to authenticated users.

The following figure describes the Jenkins jobs used to deploy the Delegation Service:

                       .------
delegation server -----| certificate-master.inject-hostcert
                       |
                       |    Insert a host certificate/key pair into /etc/gridsecurity
                       |
                       |                      .------
                       | myproxy-server  -----| simple-ca
                       |                      |
                       |                      |    This task sets up the myproxy-server as a Certificate Authority using 
                       |                      |    SimpleCA. The new CA is created using 'grid-ca-create'. MyProxy uses  
                       |                      |    the myproxy-certificate-mapapp script to create user DNs by inserting 
                       |                      |    their username into the certificate subject                           
                       |                      |
                       |                      |
                       |                      | restart
                       |                      '------
                       |
                       |                      .------                                                                      .------
                       | oa4mp.server    -----| prerequisites     ---------------------------------------------------------| tomcat-ssl    
                       |                      |                                                                            |
                       |                      |    Set up environment for OA4MP server. This includes setting up the       |    Sets up tomcat as a standalone container
                       |                      |    tomcat container (either with httpd in front of it or without), and     |    serving SSL requests through port 8443  
                       |                      |    installing dependencies (java mail, jglobus)                            |
                       |                      |                                                                            |
                       |                      |                                                                            | tomcat-httpd  
                       |                      | deploy                                                                     |
                       |                      |                                                                            |    Sets up tomcat behind a httpd reverse proxy. 
                       |                      |    Dowloads the latest tagged version of oa4mp-server and copies the       |    In this case the httpd will take care of SSL.
                       |                      |    oauth2.war into the tomcat webapps container                            |    This setup is needed in case Shibboleth SP   
                       |                      |                                                                            |    is to be used.                               
                       |                      |                                                                            '------
                       |                      | configure     
                       |                      |
                       |                      |    Configure oa4mp server with a simple setup using file storage as backend.
                       |                      |    Every relevant server information is kept under /usr/www/server          
                       |                      |
                       |                      |
                       |                      | restart
                       |                      '------
                       |
                       |                      .------
                       | shibboleth.sp   -----| setup         
                       |                      |
                       |                      |    Sets up the Shibboleth SP to forward unauthenticated requests to the      
                       |                      |    /oauth2/authorize endpoint to the Shibboleth IdP. Downloads and configures
                       |                      |    the Shibboleth IdP metadata, and adds a set of expected attributes        
                       |                      |    (uid,cn,mail)                                                             
                       |                      |
                       |                      |
                       |                      | restart
                       |                      '------
                       |
                       |
                       | shibboleth.idp.register-metadata
                       |
                       |    Dowloads the SP Metadata into the Shibboleth IdP and registeres it. 
                       '------

                         oa4mp.server.approve-clients

                            Automatically approves every pending client registration request.   

Shibboleth IdP

We used a machine template configured with Shibboleth IdP v3 and Tomcat 7. The machine template installs both shibboleth IdP and tomcat in their default location and takes care of having the right environmental variables set. The jobs used below are meant to be used with this template to further configure the IdP.

		        ---
shibboleth.idp  --------|  shibboleth.idp.setup
			|
			|	This task is meant to configure an already installed shibboleth + tomcat node by:
			|	
			|	It creates a hostcert/hostkey pair and configures tomcat to use this for enabling
			|	SSL on port 443.
			|
			|	Configure shibboleth attribute release (attribute-release.xml) with a set of attributes
			|       definitions (eduPersonPrincipalName,uid,cn,mail)
			|
			|	Configure shibboleth attribute filter (attribute-filter.xml) to release the attributes
			|	defined above to any SP.
			|	
			|
			|  shibboleth.idp.ldap.setup
			|
			|	Installs and configures OpenLDAP 
			|	
			|	Configures LDAP to be used as backend for the IdP by filling in the shibboleth LDAP 
			|	configuration file (ldap.properties)
			|
			|	Loads relevant LDAP schemas (including eduPerson 201310)
			|
			|
			|   shibboleth.idp.ldap.add-user
			|
			|	Adds a dummy user account for testing into LDAP. Makes sure that the dummy account has every
			|	relevant field which is required later on by the shibboleth attribute-releas.xml (uid,cn,mail)
			|
			|
			|   shibboleth.idp.restart
			|
			|	Restarts the shibboleth IdP service container (tomcat)
			|
			---

			    shibboleth.idp.register-metadata

				Automatically discover the Shibboleth SP running insinde Jenkins and registers its metadata