Difference between revisions of "CILogon Pre-Pilot Work"

From PDP/Grid Wiki
Jump to navigationJump to search
(→‎RCAuth.eu: detailed component descriptions)
 
(4 intermediate revisions by one other user not shown)
Line 1: Line 1:
 +
<span style="color:#DD0000">'''NOTE! This page contains information about the CIlogon-based pilot of the AARC-1 project that resulted in the RCauth.eu online CA and its MasterPortals. Please look at the [[RCauth.eu and MasterPortal documentation | RCauth.eu and MasterPortal documentation]] to find updated information about both the production setup and the AARC Piloting background'''</span>
 +
 
= Introduction =
 
= Introduction =
  
Line 6: Line 8:
  
 
For high-level information, we refer you to:
 
For high-level information, we refer you to:
 +
* [https://aarc-project.eu/digital-certificates-behind-the-scenes-the-aarc-cilogon-pilot/ Digital certificates behind the scenes: the AARC CILogon pilot]
 
* [https://wiki.geant.org/display/AARC/RCauth.eu-CILogon-like-TTS-pilot RCauth.eu and the CILogon-like TTS Pilot in Europe]
 
* [https://wiki.geant.org/display/AARC/RCauth.eu-CILogon-like-TTS-pilot RCauth.eu and the CILogon-like TTS Pilot in Europe]
 
* [https://wiki.geant.org/display/AARC/Models+for+the+CILogon-like+TTS+Pilot Sustainability models for the CILogon-like TTS pilot] and the status of the RCauth.eu white-label service by Nikhef
 
* [https://wiki.geant.org/display/AARC/Models+for+the+CILogon-like+TTS+Pilot Sustainability models for the CILogon-like TTS pilot] and the status of the RCauth.eu white-label service by Nikhef
Line 18: Line 21:
 
== Phase II ==
 
== Phase II ==
  
−
After our experience with Jenkins from [[#Phase_I | Phase I]] we decided to use try a more suitable system for deployment and configuration management. We wrote a set of [http://docs.ansible.com/ Ansible] scripts for both  [[CILogon_Pre-Pilot_Work_-_Ansible#Master_Portal | Master Portal]] and [[CILogon_Pre-Pilot_Work_-_Ansible#Delegation_Server | Delegation Server]] used in the [[#RCauth.eu | RCAuth.eu ]] setup. Execute these deployment scripts following the [[CILogon_Pre-Pilot_Work_-_Ansible | guide]].  
+
After our experience with Jenkins from [[#Phase_I | Phase I]] we decided to use try a more suitable system for deployment and configuration management. We wrote a set of [http://docs.ansible.com/ Ansible] scripts for both  [[AARC_Pilot_-_Ansible#Master_Portal | Master Portal]] and [[AARC_Pilot_-_Ansible#Delegation_Server | Delegation Server]] used in the [[#RCauth.eu | RCAuth.eu ]] setup. Execute these deployment scripts following the [[AARC_Pilot_-_Ansible | guide]].  
  
−
In case you are looking for a guide to build either Master Portal or Delegation Server, check out our [[CILogon_Pre-Pilot_Work_-_Building_from_Source | building from source]] guide.
+
In case you are looking for a guide to build either Master Portal or Delegation Server, check out our [[AARC_Pilot_-_Building_from_Source | building from source]] guide.
  
−
= Use Cases =  
+
= Pre - Piloting with OA4MP =  
  
 
== Demonstration Portal ==
 
== Demonstration Portal ==
Line 66: Line 69:
 
'''Note:''' There are two separate consent pages displayed during the above interaction: one from the Shibboleth IdP and one from the OA4MP Server. The reason for this is mainly because this is only a test setup. In a production environment this might be replaced by one single consent page.
 
'''Note:''' There are two separate consent pages displayed during the above interaction: one from the Shibboleth IdP and one from the OA4MP Server. The reason for this is mainly because this is only a test setup. In a production environment this might be replaced by one single consent page.
  
−
=== Other Resources ===
+
== Other Resources ==
  
−
==== Demo Portal ====
+
=== Demo Portal ===
  
 
There is a variant of the Demonstration Portal run by CILogon at [https://demo.cilogon.org/]. [http://svn.code.sf.net/p/cilogon/code/trunk/org.cilogon/cilogon2/cilogon2-client-oauth2/ This] is an alternative OA4MP Client implementation which can replace the Demonstration Portal described above. It is, however, functionally equivalent to the OA4MP Client with a CSS facelift.  
 
There is a variant of the Demonstration Portal run by CILogon at [https://demo.cilogon.org/]. [http://svn.code.sf.net/p/cilogon/code/trunk/org.cilogon/cilogon2/cilogon2-client-oauth2/ This] is an alternative OA4MP Client implementation which can replace the Demonstration Portal described above. It is, however, functionally equivalent to the OA4MP Client with a CSS facelift.  
  
−
==== CA security ====
+
=== CA security ===
  
 
CILogon uses SafeNet Luna hardware security module to protect their CAs. Details and sources of this setup can be found at [http://ca.cilogon.org/] and [http://cilogon.cvs.sourceforge.net/viewvc/cilogon/service/myproxy/silver/]
 
CILogon uses SafeNet Luna hardware security module to protect their CAs. Details and sources of this setup can be found at [http://ca.cilogon.org/] and [http://cilogon.cvs.sourceforge.net/viewvc/cilogon/service/myproxy/silver/]
  
−
==== MyProxy Server Frontend ====
+
=== MyProxy Server Frontend ===
  
 
It is also possible for the CILogon users to retrieve a p12 certificate through their browser from a frontend service [https://cilogon.org/]. This is simply a php frontend for MyProxy Server which does not utilise the oauth2 flow. Sources for this component can be found at [http://cilogon.cvs.sourceforge.net/viewvc/cilogon/service/html/].
 
It is also possible for the CILogon users to retrieve a p12 certificate through their browser from a frontend service [https://cilogon.org/]. This is simply a php frontend for MyProxy Server which does not utilise the oauth2 flow. Sources for this component can be found at [http://cilogon.cvs.sourceforge.net/viewvc/cilogon/service/html/].
  
−
==== Using a MyProxy CA setup for generating 'Per-User Sub-Proxies' ====
+
=== Using a MyProxy CA setup for generating 'Per-User Sub-Proxies' ===
  
 
It is possible to reconfigure a MyProxy server (running in CA mode) to produce proxy certificates from an End-Entity certificate. This can be used to create so-called 'Per-User Sub-Proxies' (PUSPs). This is described on a [[PUSP_from_MyProxy|dedicated wiki page]]. Such an 'online CA' can be used in a transitional phase instead of a proper online CA.
 
It is possible to reconfigure a MyProxy server (running in CA mode) to produce proxy certificates from an End-Entity certificate. This can be used to create so-called 'Per-User Sub-Proxies' (PUSPs). This is described on a [[PUSP_from_MyProxy|dedicated wiki page]]. Such an 'online CA' can be used in a transitional phase instead of a proper online CA.
−
 
−
== Master Portal ==
 
−
 
−
As a second step for the pre-piloting work we propose extending the Demonstration Portal scenario described above. The setup described above serves as the basis of what's to follow. Our aim with this extension is to interface with VOMS as an attribute provider and to propose different ways of retrieving the vomsified proxy.
 
−
 
−
=== High Level Architecture ===
 
−
 
−
[[File:Master-portal-v1.1.png]]
 
−
 
−
* the yellow services (bottom right box) are the central (European-wide?) CILogon/Online CA services.
 
−
* the red services (center box) are the myproxy credential store plus its frontend services: master portal, ssh server plus the needed extra services. There could be multiple, e.g. one per federation, effectively different OpenID Connect clients.
 
−
* the blue services (left box) are run by the VO
 
−
* the green services are e.g. Grid UIs but could also be CEs or laptops and are using non-web 'SSO'.
 
−
* the purple service is the IdP.
 
−
 
−
==== Flow ====
 
−
 
−
Here we describe a brief overview of the flow through which users create and retrieve a voms proxy. A detailed explanation will follow later.
 
−
 
−
1) user enters via very-light-weight '''VO Portal''' on the left, specifying something such as requested VOMS FQANs.
 
−
 
−
2) '''VO Portal''' redirects to '''Master Portal''' which is a customized CILogon portal doing the CILogin portal delegation. The next few steps are the standard portal delegation also described [[#Sequence Diagram|above]]
 
−
 
−
3a) redirect to '''Delegation Service'''.
 
−
 
−
3b) '''Delegation Service''' sends to WAYF and IdP
 
−
 
−
3c) return (few steps) to '''Delegation Service''' which returns OAuth token to '''Master Portal'''
 
−
 
−
3d) '''Master Portal''' uses token to send a request to /userinfo to get user's identity.
 
−
 
−
 
−
When there is no valid certificate in the '''MyProxy Credential Store''' step 4) will be done
 
−
 
−
4a) '''Master Portal''' sends CSR and obtains certificate+key. The remaining steps are not part of standard CILogon/OA4MP.
 
−
 
−
4b) '''Master Portal''' will store the cert+key into a seconds myproxy server ('''MyProxy Credential Store''') with the obtained identity. The myproxy server uses server-side VOMS integration.
 
−
 
−
 
−
5) '''Master Portal''' can now use jGlobus' myproxy API to obtain vomsified proxies from this '''MyProxy Credential Store''', where it will be a trusted retriever. It will send those back (on demand) to the VO portal. When the user asked for invalid VOMS credentials, this will also be fed back to the user.
 
−
 
−
 
−
 
−
The next two steps are an extension to enable access on e.g. workernodes or UIs. i.e. non-web 'sso'. It's quite similar to what github is doing.
 
−
 
−
7) '''Master Portal''' allows uploading of ssh public keys which are put in the authorized keys file on '''SSH Server'''.
 
−
 
−
8) When the user logs in onto the '''SSH Server''' it will automatically (using the command option in the authorized keys file) run a script which will run something like a myproxy-get-delegation and return a short-lived proxy to the user via e.g. stdout over the ssh. We can probably use the myproxy-server-side voms for that, by passing the right information to the SSH server, alternatively the user or the script can do that on client side.
 
−
 
−
 
−
 
−
As an alternative way of accessing a user proxy from the command line with user credentials (username and password) the Master portal can do the following:
 
−
 
−
7) provision a custom account in for example an '''LDAP''' server, belonging to the user, where the user picks a password. Other methods are possible too, as long as they are compatible with PAM.
 
−
 
−
8) This can then be used by the user to authenticate using PAM against the '''MyProxy Credential Store''' to retrieve a short-lived proxy. In this case, it is probably necessary to do the vomsification on client side.
 
−
 
−
==== Public Demo ====
 
−
 
−
There is a public instance of the setup described above living in [https://okeanos.grnet.gr/home/ Okeanos]. You can start the flow of receiving a voms proxy by navigating to the [https://snf-676786.vm.okeanos.grnet.gr/vo-portal/ VO Portal]. The public demo works with the [https://wiki.surfnet.nl/display/surfconextdev/Connecting+to+the+SURFconext+Connect+environment SURFconext Test Instance] to showcase the integration with an identity federation. You will have to log in with one of the test professor accounts. Please note, that this setup is still under development, therefore it is subject to frequent changes.
 
−
 
−
=== Detailed Architecture ===
 
−
 
−
[[File:Masterportal singleWAYF.png]]
 
−
 
−
==== Detailed Flow ====
 
−
 
−
<ol>
 
−
<li>user browses to VO portal
 
−
 
−
<li>redirect to /authorize on Master Portal (OIDC<sub>1</sub>)
 
−
 
−
<li>redirect to /authorize on DS (OIDC<sub>2</sub>)
 
−
 
−
<li>redirect to WAYF (SAML)
 
−
 
−
<li>redirect to IdP (SAML) -> user logs in
 
−
 
−
<li>redirect back to /authorize on DS (SAML)
 
−
 
−
<li>redirect to redirect_uri on Master Portal (/ready endpoint) (OIDC<sub>2</sub>)<br>
 
−
(Master portal retrieves access_token<sub>2</sub> and userinfo)
 
−
 
−
<li>redirect back to redirect_uri on VO portal (OIDC<sub>1</sub>)
 
−
</ol>
 
−
 
−
 
−
Next steps are all hidden for the end-user.
 
−
 
−
<ol style="list-style-type:lower-roman">
 
−
<li>VO portal retrieves access_token<sub>1</sub>
 
−
<li>VO portal retrieves userinfo from Master Portal using access_token<sub>1</sub>
 
−
<li>VO portal calls /getproxy endpoint on Master Portal using access_token<sub>1</sub>, optionally with VOMSES and VOMS string
 
−
<li>Master Portal does INFO on MyProxy credstore to check for long-lived proxy:<br>
 
−
no proxy yet:
 
−
<ol style="list-style-type:lower-alpha">
 
−
<li>Master Portal creates keypair + CSR
 
−
<li>Master Portal calls /getcert on DS using access_token<sub>2</sub> and CSR
 
−
<li>DS does a MyProxy GET request at online CA, using the CSR
 
−
<li>online CA signs the CSR and returns the end-entity certificate to the DS<br>
 
−
DS returns end-entity certificate to Master Portal<br>
 
−
Master Portal uses end-entity certificate and key to delegate (MyProxy PUT) a long-lived proxy to MyProxy credstore
 
−
</ol>
 
−
<li>Master Portal retrieves short lived (VOMS) proxy from MyProxy credstore
 
−
<li>Master Portal returns proxy to VO portal
 
−
</ol>
 
−
 
−
 
−
[[File:Aarc-masterportal-sequence-numbered-v2.png]]
 
−
 
−
=== Other Resources ===
 
−
 
−
==== Sources ====
 
−
 
−
The sources for the Master Portal and the VO Portal can be found on [https://github.com/ttomttom/aarc-portal github]. In order to build them you first have to make sure that the two dependency projects [https://github.com/ttomttom/myproxy-fork myproxy-fork] and [https://github.com/ttomttom/ncsa-security-all-fork ncsa-security-all-fork] are build and installed in the local maven repository. Check out the 'devel' branch of each dependency and build them in the following order: first '''ncsa-security-all-fork''', then '''myproxy-fork'''.
 
−
 
−
==== GetProxy Endpoint ====
 
−
 
−
In order to retrieve proxies instead of end entity certificates from the Master Portal we proposed an extension to the [https://docs.google.com/document/d/1cs3peO9FxA81KN-1RC6Z-auEFIwRbJpZ-SFuKbQzS50/pub OIDC/OA4MP Protocol]. This is called the [[OAuth for MyProxy GetProxy Endpoint]] extension.
 
−
 
−
== RCAuth.eu  ==
 
−
 
−
RCauth.eu is a white-label Research and Collaboration Authentication CA Service for Europe. For more detailed information about it including policies and sustainability models please refer to [https://www.rcauth.eu/ RCauth.eu] website.
 
−
 
−
=== Architecture ===
 
−
 
−
[[File:RCAuth Prototype.svg]]
 
−
 
−
==== Components ====
 
−
 
−
* <span style="color:#7EA6E0">'''Blue component'''</span> [many] represent the Service Provider Portal which the user wants to use. These are usually the Science Gateways.
 
−
** ''Example Portal'' : As an example, we provide a [https://www.nikhef.nl/~msalle/oidc_getproxy_demo_source.php demo portal] (written in php). This is a good starting point to learn how to interface with the Master Portal.
 
−
* <span style="color:#EA6B66">'''Red components'''</span> [few] correspond to the Master Portal and they are functionally equivalent to the component described [[#Master Portal | above]].
 
−
** ''Master Portal'' : acts as a caching service for user credential (proxy certificates), taking some load of the RCauth.eu backend. Moreover, it also intermediates between two separate trust domains: the domain (single) of the Delegation Server and the domains (many) of connecting Service Provider Portals (Science Gateways). This improves the scalability of the model, since instead of registering ALL Portals to the single Delegation Server directly, now registered Portals can be split between a few Master Portals running in front of the Delegation Server.
 
−
** ''Credential Store'' : is a MyProxy server used by the Master Portal to actually store the user proxies.
 
−
* <span style="color:#FFD966">'''Yellow components'''</span> [one] represent an Online CA with a web frontend, and its what we call [https://www.rcauth.eu/ RCauth.eu]. This includes the following subcomponents:
 
−
** ''Delegation Server'' : is the web frontend service which talks to the Online CA to generate certificates for authenticated users.
 
−
** ''Online CA'' : is a Certificate Authority running on an HSM. This service (although called online) is only directly accessible to the Delegation Server in front of it.
 
−
** ''WAYF'' : an IdP / SP Proxy with an internal filter for accepting authentication sources directly. This gives full control for RCauth.eu over the eligibility of IdPs.
 
−
* <span style="color:#8C6C9C">'''Purple components'''</span> [many] represent the different authentication sources that RCauth.eu is accepting (or planning to accept in the near future).
 
−
 
−
==== Flow ====
 
−
 
−
The flow of this setup is equivalent to the flow described [[#Detailed Flow | before]], with a small difference between steps 4 and 5. Since RCauth.eu is using its own WAYF (IdP/SP Proxy) there will be an additional redirect between steps 4 and 5 that will take care of redirecting the user to his selected RI (Research Infrastructure) or IdP for authentication. From the point of view of the flow this additional redirect can be automated by using an '''IdP hint'''. An '''IdP hint''', containing a entity ID of the home IdP of the user, can be sent by the Service Provider Portal (Science Gateway) upon first contacting the Master Portal for a proxy. The '''IdP hint''' is sent along to the WAYF (IdP/SP Proxy) which will verify it to be a valid authentication source, and automatically redirect the user to his preferred IdP. This hides some of the complexities of the setup from the user, since he is no longer required to choose his home IdP in case he is already authenticated at the Service Provider Portal (Science Gateway). This functionality completely relies on the cooperation of the Service Provider Portal (Science Gateway), who is responsible of constructing and sending a valid '''IdP hint''' for its users in the first place.
 
−
 
−
==== Credential Lifetimes ====
 
−
 
−
'''Note!''' Although this setup is fully compatible with [[PUSP from MyProxy | PUSPs]], RCauth.eu is only producing end entity certificates (EEC).
 
−
 
−
We distinguish three different type of credentials in our setup:
 
−
 
−
* End Entity Certificates (EEC) are produced by the ''Delegation Server'' for authenticated users. The maximum lifetime of these credentials is '''11 days'''. 
 
−
* Long Lived Proxy Certificates are produced by the ''Master Portal'' from the EEC retrieved from the ''Delegation Server'' and they are stored in the Credential Store. The lifetime of these proxies will match the lifetime of the EEC they are based on, which in this case will be '''11 days'''.
 
−
* Short Lived Proxy Certificates are produced by the ''Master Portal'' from the stored Long Lived Proxy Certificate and they are released to the Portals (Science Gateways). The lifetime of these proxies is decided based on the lifetime value sent by the Portals. In case no lifetime value is requested, it will default to '''12 hours'''.
 

Latest revision as of 08:41, 5 September 2019

NOTE! This page contains information about the CIlogon-based pilot of the AARC-1 project that resulted in the RCauth.eu online CA and its MasterPortals. Please look at the RCauth.eu and MasterPortal documentation to find updated information about both the production setup and the AARC Piloting background

Introduction

AARC is a European research and collaboration project which explores the possible future generation of authentication and authorization methods used in the grid world. CILogon is a candidate AAI component within this project. This document is the starting point of a piloting work on CILogon which will eventually evolve into a larger scale demonstration of the capabilities of this software product.

CILogon is an open source project used to provide x509 certificates based on the authenticated user's federated identity. These x509 certificates are then meant to be used as authentication in further interactions with grid services.

For high-level information, we refer you to:

Technical pilot development

Phase I

We used Jenkins together with OpenStack to build an automatically deploying demo instance. Jenkins VM Image templates are used to boot VMs with relevant software installed on it, while separate Jenkins jobs are used to carry out different stages of the service deployment. You can find more details at CILogon Pre-Pilot Work - Jenkins and github.

Phase II

After our experience with Jenkins from Phase I we decided to use try a more suitable system for deployment and configuration management. We wrote a set of Ansible scripts for both Master Portal and Delegation Server used in the RCAuth.eu setup. Execute these deployment scripts following the guide.

In case you are looking for a guide to build either Master Portal or Delegation Server, check out our building from source guide.

Pre - Piloting with OA4MP

Demonstration Portal

We aimed our first use case at setting up a Demonstration Portal through which authenticated users can retrieve x509 certificates. This setup mimics the CILogon Portal Demonstration.

The user goes to the Demonstration Portal and starts the flow by asking for a certificate. The Demonstration Portal creates a new private key and a certificate signing request (CSR). The private key is kept in the Portal, while CSR is eventually sent to the Delegation Service (acting as an online CA). At the Delegation Service the user gets redirected to his home IdP for authentication (for now there is only one single IdP in our setup). After authentication and consent the user attributes are sent to the Delegation Service. The Delegation Service uses MyProxy-Server (CA) to create a user certificate from the CSR and user attributes. This certificate is then sent back to the user.

Components

CILogon is based on OAuth for MyProxy (OA4MP). For the purpose of this Demonstration Portal we used OA4MP because it's much better documented then setting up CILogon itself, and because it was the recommended way to go by Jim Basney (CILogon).

  • OA4MP : using both the client and the server components from the latest OAuth 2.0 implementation (3.1.1)
  • Shibboleth : using the latest Identity Provider (3.0), and Service Provider (2.0)
  • MyProxy Server : using official releases from epel (6.1.13)
  • SimpleCA : using official release from epel (4.20)

Architecture

Aarc-cilogon-demoportal.jpg

Demonstration Portal
- The Portal runs as the oa4mp-client (oauth2 client) in this setup. The protected resource in this case is the user certificate that will end up in the Portal at the end of the oauth2 flow. This Portal is then used to access a protected resource with the user certificate as suggested by [1] (this is not part of our demo setup just yet).
- The oa4mp-client has to be registered with the corresponding oa4mp-server before it can be used. This is usually done through the /register endpoint of oa4mp-server.
Delegation Service / SP
- The Delegation Service runs as the oa4mp-server (oauth2 server) and provides a frontend to the MyProxy CA. The oauth2 protocol implementation follows the specifications outlined at OpenID Connect for MyProxy.
- The oa4mp-server runs behind an httpd reverse proxy configured with mod_shib, which redirects requests to Shibboleth IdP for user authentication.
Shibboleth IdP
- This is a simple Shibboleth IdP setup backed up by an LDAP server. It releases user attributed to the Shibboleth SP running in front of the Delegation Service.
MyProxy Server / SimpleCA
- MyProxy Server is configured to run as Certificate Authority (CA). It creates user certificates from the CSR and user attributes. The user attributes are used to fill in the certificate subject with the aid of myproxy-certificate-mapapp.
- SimpleCA is used to create the initial root CA certificate.

Sequence Diagram

The following sequence diagram details the flow of the delegation scenario described above. The user in this diagram stands for the users' browser which is redirecting calls. It contains the OAuth2 calls also outlined in the OpenID Connect for MyProxy specifications, and the SAML calls describing the interaction with the IdP.

Aarc-cilogon-sequence.png

Note: There are two separate consent pages displayed during the above interaction: one from the Shibboleth IdP and one from the OA4MP Server. The reason for this is mainly because this is only a test setup. In a production environment this might be replaced by one single consent page.

Other Resources

Demo Portal

There is a variant of the Demonstration Portal run by CILogon at [2]. This is an alternative OA4MP Client implementation which can replace the Demonstration Portal described above. It is, however, functionally equivalent to the OA4MP Client with a CSS facelift.

CA security

CILogon uses SafeNet Luna hardware security module to protect their CAs. Details and sources of this setup can be found at [3] and [4]

MyProxy Server Frontend

It is also possible for the CILogon users to retrieve a p12 certificate through their browser from a frontend service [5]. This is simply a php frontend for MyProxy Server which does not utilise the oauth2 flow. Sources for this component can be found at [6].

Using a MyProxy CA setup for generating 'Per-User Sub-Proxies'

It is possible to reconfigure a MyProxy server (running in CA mode) to produce proxy certificates from an End-Entity certificate. This can be used to create so-called 'Per-User Sub-Proxies' (PUSPs). This is described on a dedicated wiki page. Such an 'online CA' can be used in a transitional phase instead of a proper online CA.

Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=CILogon_Pre-Pilot_Work&oldid=10837"