Difference between revisions of "CILogon Pre-Pilot Work"

From PDP/Grid Wiki
Jump to navigationJump to search
(myproxy-oauth2 server)
 
(34 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
<span style="color:#DD0000">'''NOTE! This page contains information about the CIlogon-based pilot of the AARC-1 project that resulted in the RCauth.eu online CA and its MasterPortals. Please look at the [[RCauth.eu and MasterPortal documentation | RCauth.eu and MasterPortal documentation]] to find updated information about both the production setup and the AARC Piloting background'''</span>
 +
 
= Introduction =
 
= Introduction =
  
[http://www.cilogon.org/ CILogon] is an open source project used to provide secure access to different services from the grid infrastructure. It uses the [http://www.cilogon.org/portal-delegation Myproxy OAuth2] flow in order to retrieve client certificates/proxies from a [http://grid.ncsa.illinois.edu/myproxy/ca/ Myproxy Certificate Authority]. These retrieved credentials are then meant to be used in the authentication procedure for further interaction with grid services.  
+
[https://aarc-project.eu/ AARC] is a European research and collaboration project which explores the possible future generation of authentication and authorization methods used in the grid world. [http://www.cilogon.org/ CILogon] is a candidate AAI component within this project. This document is the starting point of a piloting work on CILogon which will eventually evolve into a larger scale demonstration of the capabilities of this software product. 
 +
 
 +
[http://www.cilogon.org/ CILogon] is an open source project used to provide x509 certificates based on the authenticated user's federated identity. These x509 certificates are then meant to be used as authentication in further interactions with grid services.
 +
 
 +
For high-level information, we refer you to:
 +
* [https://aarc-project.eu/digital-certificates-behind-the-scenes-the-aarc-cilogon-pilot/ Digital certificates behind the scenes: the AARC CILogon pilot]
 +
* [https://wiki.geant.org/display/AARC/RCauth.eu-CILogon-like-TTS-pilot RCauth.eu and the CILogon-like TTS Pilot in Europe]
 +
* [https://wiki.geant.org/display/AARC/Models+for+the+CILogon-like+TTS+Pilot Sustainability models for the CILogon-like TTS pilot] and the status of the RCauth.eu white-label service by Nikhef
 +
* [https://wiki.geant.org/display/AARC/Ongoing+Pilots Pilot ongoing status]
 +
 
 +
= Technical pilot development =
 +
 
 +
== Phase I ==
 +
 
 +
We used [[Jenkins Setup | Jenkins]] together with [[OpenStack Cluster | OpenStack]] to build an automatically deploying demo instance. Jenkins VM Image templates are used to boot VMs with relevant software installed on it, while separate Jenkins jobs are used to carry out different stages of the service deployment. You can find more details at [[CILogon Pre-Pilot Work - Jenkins]] and [https://github.com/ttomttom/aarc-cilogon-pilot github].
 +
 
 +
== Phase II ==
 +
 
 +
After our experience with Jenkins from [[#Phase_I | Phase I]] we decided to use try a more suitable system for deployment and configuration management. We wrote a set of [http://docs.ansible.com/ Ansible] scripts for both  [[AARC_Pilot_-_Ansible#Master_Portal | Master Portal]] and [[AARC_Pilot_-_Ansible#Delegation_Server | Delegation Server]] used in the [[#RCauth.eu | RCAuth.eu ]] setup. Execute these deployment scripts following the [[AARC_Pilot_-_Ansible | guide]].
 +
 
 +
In case you are looking for a guide to build either Master Portal or Delegation Server, check out our [[AARC_Pilot_-_Building_from_Source | building from source]] guide.
 +
 
 +
= Pre - Piloting with OA4MP =
 +
 
 +
== Demonstration Portal ==
 +
 
 +
We aimed our first use case at setting up a Demonstration Portal through which authenticated users can retrieve x509 certificates. This setup mimics the [https://demo.cilogon.org/ CILogon Portal Demonstration].
 +
 
 +
The user goes to the Demonstration Portal and starts the flow by asking for a certificate. The Demonstration Portal creates a new private key and a certificate signing request (CSR). The private key is kept in the Portal, while CSR is eventually sent to the Delegation Service (acting as an online CA). At the Delegation Service the user gets redirected to his home IdP for authentication (for now there is only one single IdP in our setup). After authentication and consent the user attributes are sent to the Delegation Service. The Delegation Service uses MyProxy-Server (CA) to create a user certificate from the CSR and user attributes. This certificate is then sent back to the user. 
 +
 
 +
=== Components ===
 +
 
 +
CILogon is based on [http://grid.ncsa.illinois.edu/myproxy/oauth/ OAuth for MyProxy] (OA4MP). For the purpose of this Demonstration Portal we used OA4MP because it's much better documented then setting up CILogon itself, and because it was the recommended way to go by Jim Basney (CILogon).
 +
 
 +
* [http://grid.ncsa.illinois.edu/myproxy/oauth/ OA4MP] : using both the client and the server components from the latest OAuth 2.0 implementation (3.1.1)
 +
* [https://wiki.shibboleth.net/confluence/dashboard.action Shibboleth] : using the latest Identity Provider (3.0), and Service Provider (2.0)
 +
* [http://grid.ncsa.illinois.edu/myproxy/ca/ MyProxy Server] : using official releases from epel (6.1.13)
 +
* [http://toolkit.globus.org/toolkit/docs/latest-stable/simpleca/ SimpleCA] : using official release from epel (4.20)
 +
 
 +
=== Architecture ===
 +
 
 +
[[File:aarc-cilogon-demoportal.jpg]]
 +
 
 +
; Demonstration Portal
 +
: - The Portal runs as the oa4mp-client (oauth2 client) in this setup. The protected resource in this case is the user certificate that will end up in the Portal at the end of the oauth2 flow. This Portal is then used to access a protected resource with the user certificate as suggested by [http://www.cilogon.org/portal-delegation] (this is not part of our demo setup just yet).
 +
: - The oa4mp-client has to be registered with the corresponding oa4mp-server before it can be used. This is usually done through the /register endpoint of oa4mp-server.
 +
 
 +
; Delegation Service / SP
 +
: - The Delegation Service runs as the oa4mp-server (oauth2 server) and provides a frontend to the MyProxy CA. The oauth2 protocol implementation follows the specifications outlined at [https://docs.google.com/document/d/1cs3peO9FxA81KN-1RC6Z-auEFIwRbJpZ-SFuKbQzS50/pub OpenID Connect for MyProxy].
 +
: - The oa4mp-server runs behind an httpd reverse proxy configured with [https://wiki.shibboleth.net/confluence/display/SHIB2/NativeSPApacheConfig mod_shib], which redirects requests to Shibboleth IdP for user authentication.
 +
 
 +
; Shibboleth IdP
 +
: - This is a simple Shibboleth  IdP setup backed up by an LDAP server. It releases user attributed to the Shibboleth SP running in front of the Delegation Service.
 +
 
 +
; MyProxy Server / SimpleCA
 +
: - MyProxy Server  is configured to run as Certificate Authority (CA). It creates user certificates from the CSR and user attributes. The user attributes are used to fill in the certificate subject with the aid of [http://grid.ncsa.illinois.edu/myproxy/oauth/server/manuals/remote-user.xhtml myproxy-certificate-mapapp].
 +
: - SimpleCA is used to create the initial root CA certificate.
 +
 
 +
=== Sequence Diagram ===
 +
 
 +
The following sequence diagram details the flow of the delegation scenario described above. The user in this diagram stands for the users' browser which is redirecting calls. It contains the OAuth2 calls also outlined in the [https://docs.google.com/document/d/1cs3peO9FxA81KN-1RC6Z-auEFIwRbJpZ-SFuKbQzS50/pub OpenID Connect for MyProxy] specifications, and the SAML calls describing the interaction with the IdP.
 +
 
 +
[[File:aarc-cilogon-sequence.png]]
 +
 
 +
'''Note:''' There are two separate consent pages displayed during the above interaction: one from the Shibboleth IdP and one from the OA4MP Server. The reason for this is mainly because this is only a test setup. In a production environment this might be replaced by one single consent page.
 +
 
 +
== Other Resources ==
 +
 
 +
=== Demo Portal ===
 +
 
 +
There is a variant of the Demonstration Portal run by CILogon at [https://demo.cilogon.org/]. [http://svn.code.sf.net/p/cilogon/code/trunk/org.cilogon/cilogon2/cilogon2-client-oauth2/ This] is an alternative OA4MP Client implementation which can replace the Demonstration Portal described above. It is, however, functionally equivalent to the OA4MP Client with a CSS facelift.  
 +
 
 +
=== CA security ===
  
[https://aarc-project.eu/ AARC] is a European research and collaboration project which explores the possible future generation of authentication and authorization methods used in the grid world. CILogon is a candidate AAI component within this project. This document is the starting point of a piloting work on CILogon which will eventually evolve into a larger scale demonstration of the capabilities of this software product.
+
CILogon uses SafeNet Luna hardware security module to protect their CAs. Details and sources of this setup can be found at [http://ca.cilogon.org/] and [http://cilogon.cvs.sourceforge.net/viewvc/cilogon/service/myproxy/silver/]
  
We used [[Jenkins Setup | Jenkins]] together with [[OpenStack Cluster | OpenStack]] to build an automatically deploying CILogon instance. Jenkins VM Image templates are used to boot VMs with relevant software installed on it, while separate Jenkins jobs are used to carry out different stages of the service deployment. The following sections contain a brief overview of the structure of these Jenkins jobs together with a short description of their function. For more information consult the local [https://jenkins.nikhef.nl:8008/ jenkins instance].
+
=== MyProxy Server Frontend ===
  
= Myproxy-OAuth2 server =
+
It is also possible for the CILogon users to retrieve a p12 certificate through their browser from a frontend service [https://cilogon.org/]. This is simply a php frontend for MyProxy Server which does not utilise the oauth2 flow. Sources for this component can be found at [http://cilogon.cvs.sourceforge.net/viewvc/cilogon/service/html/].
  
The [http://grid.ncsa.illinois.edu/myproxy/oauth/ Myproxy-OAuth2 server], together with the [http://grid.ncsa.illinois.edu/myproxy/ca/ myproxy-server] set up as a Certificate Authority is used to provide user certificates for different OAuth2 clients (portals).
+
=== Using a MyProxy CA setup for generating 'Per-User Sub-Proxies' ===
  
        ---
+
It is possible to reconfigure a MyProxy server (running in CA mode) to produce proxy certificates from an End-Entity certificate. This can be used to create so-called 'Per-User Sub-Proxies' (PUSPs). This is described on a [[PUSP_from_MyProxy|dedicated wiki page]]. Such an 'online CA' can be used in a transitional phase instead of a proper online CA.
myproxy-server  --------|  myproxy-server.simpleCA
 
|
 
| ~ This task sets up the myproxy-server as a Certificate Authority using SimpleCA.
 
| It creates a new self signed CA, a host certificate and a service certificate
 
| (for myrpoxy-server) using the: 'grid-ca-create' 'grid-cert-request' and
 
| 'grid-ca-sign' commands.
 
|
 
|
 
|  myproxy-server.pam.useradd
 
|
 
| ~ Adds a local UNIX account that will be used to PAM authentication through the |
 
| myproxy-server (in this sense myproxy-server becomes an IdP). This will later
 
| be replaced by a more suitable authentication method with an IdP like shib IdP
 
|
 
| ~ myproxy-server creates user certificates for users by matching authenticated
 
| users' name against an entry from the grid-mapfile. Therefore, an entry is added
 
| into the grid-mapfile with the name of the newly created user.
 
|
 
|
 
|  myproxy-server.restart
 
|
 
| ~ Restarts the myproxy-server
 
|
 
---
 
 
 
        ---
 
myproxy-oauth2  --------|  myproxy-oauth2.prerequisites
 
|
 
| ~ Sets up the local environment in which the cilogon server will be run:
 
|   - set up SSL in tomcat
 
|   - install java 1.8
 
|   - install java mail
 
|   - install jsse and globus-ssl-proxies (jglobus-core)
 
|
 
|
 
|  myproxy-oauth2.prerequisites.slave-reboot
 
|
 
| ~ Because of a Jenkins limitation whenever the default jvm changes the slave agent
 
| running on the node has to be restarted.
 
|
 
|
 
|  myproxy-oauth2.deploy
 
|
 
| ~ Downloads the latest tagged version of the CILogon oauth2 server and copies the
 
| oauth2.war into the tomcat container
 
|
 
|
 
|  myproxy-oauth2.configure
 
|
 
| ~ Configures the oauth2 server with a simple setup using file storage as backend.
 
| Every relevant server information is kept under /usr/www/server
 
 
|
 
|  myproxy-oauth2.restart
 
|
 
|  ~ Restarts tomcat
 
|
 
---
 
 
 
  myproxy-oauth2.approve-clients
 
 
~ Automatically approves every pending client registration request.
 

Latest revision as of 08:41, 5 September 2019

NOTE! This page contains information about the CIlogon-based pilot of the AARC-1 project that resulted in the RCauth.eu online CA and its MasterPortals. Please look at the RCauth.eu and MasterPortal documentation to find updated information about both the production setup and the AARC Piloting background

Introduction

AARC is a European research and collaboration project which explores the possible future generation of authentication and authorization methods used in the grid world. CILogon is a candidate AAI component within this project. This document is the starting point of a piloting work on CILogon which will eventually evolve into a larger scale demonstration of the capabilities of this software product.

CILogon is an open source project used to provide x509 certificates based on the authenticated user's federated identity. These x509 certificates are then meant to be used as authentication in further interactions with grid services.

For high-level information, we refer you to:

Technical pilot development

Phase I

We used Jenkins together with OpenStack to build an automatically deploying demo instance. Jenkins VM Image templates are used to boot VMs with relevant software installed on it, while separate Jenkins jobs are used to carry out different stages of the service deployment. You can find more details at CILogon Pre-Pilot Work - Jenkins and github.

Phase II

After our experience with Jenkins from Phase I we decided to use try a more suitable system for deployment and configuration management. We wrote a set of Ansible scripts for both Master Portal and Delegation Server used in the RCAuth.eu setup. Execute these deployment scripts following the guide.

In case you are looking for a guide to build either Master Portal or Delegation Server, check out our building from source guide.

Pre - Piloting with OA4MP

Demonstration Portal

We aimed our first use case at setting up a Demonstration Portal through which authenticated users can retrieve x509 certificates. This setup mimics the CILogon Portal Demonstration.

The user goes to the Demonstration Portal and starts the flow by asking for a certificate. The Demonstration Portal creates a new private key and a certificate signing request (CSR). The private key is kept in the Portal, while CSR is eventually sent to the Delegation Service (acting as an online CA). At the Delegation Service the user gets redirected to his home IdP for authentication (for now there is only one single IdP in our setup). After authentication and consent the user attributes are sent to the Delegation Service. The Delegation Service uses MyProxy-Server (CA) to create a user certificate from the CSR and user attributes. This certificate is then sent back to the user.

Components

CILogon is based on OAuth for MyProxy (OA4MP). For the purpose of this Demonstration Portal we used OA4MP because it's much better documented then setting up CILogon itself, and because it was the recommended way to go by Jim Basney (CILogon).

  • OA4MP : using both the client and the server components from the latest OAuth 2.0 implementation (3.1.1)
  • Shibboleth : using the latest Identity Provider (3.0), and Service Provider (2.0)
  • MyProxy Server : using official releases from epel (6.1.13)
  • SimpleCA : using official release from epel (4.20)

Architecture

Aarc-cilogon-demoportal.jpg

Demonstration Portal
- The Portal runs as the oa4mp-client (oauth2 client) in this setup. The protected resource in this case is the user certificate that will end up in the Portal at the end of the oauth2 flow. This Portal is then used to access a protected resource with the user certificate as suggested by [1] (this is not part of our demo setup just yet).
- The oa4mp-client has to be registered with the corresponding oa4mp-server before it can be used. This is usually done through the /register endpoint of oa4mp-server.
Delegation Service / SP
- The Delegation Service runs as the oa4mp-server (oauth2 server) and provides a frontend to the MyProxy CA. The oauth2 protocol implementation follows the specifications outlined at OpenID Connect for MyProxy.
- The oa4mp-server runs behind an httpd reverse proxy configured with mod_shib, which redirects requests to Shibboleth IdP for user authentication.
Shibboleth IdP
- This is a simple Shibboleth IdP setup backed up by an LDAP server. It releases user attributed to the Shibboleth SP running in front of the Delegation Service.
MyProxy Server / SimpleCA
- MyProxy Server is configured to run as Certificate Authority (CA). It creates user certificates from the CSR and user attributes. The user attributes are used to fill in the certificate subject with the aid of myproxy-certificate-mapapp.
- SimpleCA is used to create the initial root CA certificate.

Sequence Diagram

The following sequence diagram details the flow of the delegation scenario described above. The user in this diagram stands for the users' browser which is redirecting calls. It contains the OAuth2 calls also outlined in the OpenID Connect for MyProxy specifications, and the SAML calls describing the interaction with the IdP.

Aarc-cilogon-sequence.png

Note: There are two separate consent pages displayed during the above interaction: one from the Shibboleth IdP and one from the OA4MP Server. The reason for this is mainly because this is only a test setup. In a production environment this might be replaced by one single consent page.

Other Resources

Demo Portal

There is a variant of the Demonstration Portal run by CILogon at [2]. This is an alternative OA4MP Client implementation which can replace the Demonstration Portal described above. It is, however, functionally equivalent to the OA4MP Client with a CSS facelift.

CA security

CILogon uses SafeNet Luna hardware security module to protect their CAs. Details and sources of this setup can be found at [3] and [4]

MyProxy Server Frontend

It is also possible for the CILogon users to retrieve a p12 certificate through their browser from a frontend service [5]. This is simply a php frontend for MyProxy Server which does not utilise the oauth2 flow. Sources for this component can be found at [6].

Using a MyProxy CA setup for generating 'Per-User Sub-Proxies'

It is possible to reconfigure a MyProxy server (running in CA mode) to produce proxy certificates from an End-Entity certificate. This can be used to create so-called 'Per-User Sub-Proxies' (PUSPs). This is described on a dedicated wiki page. Such an 'online CA' can be used in a transitional phase instead of a proper online CA.