Master Portal Administrator Guide

Introduction

The Master Portal is a central component in the AARC Piloting work. The Master Portal caches long lived user proxies into its Credential Store, and returns short lived proxies on demand to a trusted portal (Science Gateway) for authenticated users. This page is intended for Master Portal operators and administrators who wish to configure their Master Portals.

In case you want to deploy a Master Portal, see this guide for using our Ansible deployment scripts. Most of the presented configurations below are set by the ansible scripts.

Credential Store Configuration

The Master Portal cannot be operated without its Credential Store. Missing the Credential Store from your setup will render your Master Portal unable to cache and retrieve any proxy certificates. Without the Credential Store a Master Portal could still be used for user authentication (via OIDC/OA4MP), but its GetProxy Endpoint will be unable to retrieve proxy certificates.

MyProxy Server

The Credential Store is just a MyProxy Server installation running in Credential Store mode. It being an unmodified MyProxy Server, you can tailor its configuration to your needs following the official MyProxy Server admin guide. In order to run it as a Credential Store backend for the Master Portal, you will need the following set of configurations:

accepted_credentials            <Master Portal Host DN>
authorized_retrievers           <Master Portal Host DN>
 
cert_dir                        /etc/grid-security/certificates
max_proxy_lifetime              264

You should not need to also set authorized_renewers, default_authorized_renewers or trusted_retrievers.

The Master Portal authenticates to the MyProxy Server via its host certificates. Make sure to configure the DN of the host certificate of the Master Portal (<Master Portal Host DN>) as the authorized party in the MyProxy Server configuration snippet above. The meaning of each of the individual authorizations can be found in the man page of myproxy-server.config.
The cert_dir has to point to the set of trust roots trusted by the MyProxy Server.
The max_proxy_lifetime determines the maximum lifetime in hours that a returned proxy credential can have. I.e. one can request shorter-lived proxies, not longer-lived. Since the Master Portal is only allowed to release short lived proxy certificates, a lifetime of 264 hours (11 days) is imposed by default (additionally, the current RCauth online CA produces EECs with a lifetime of 11 days making it meaningless to create proxies with longer lifetime).

VOMS Support

The MyProxy Server returns returns plain (i.e. non-VOMS) proxies by default, but it has support for releasing VOMS-ified proxies as well. Asking for VOMS-ified proxies is also supported in the Master Portal via the GetProxy Endpoint. In order to enable VOMS support in the MyProxy Server, set allow_voms_attribute_requests

allow_voms_attribute_requests   True

Additionally, one can add local configuration for supported VOs, by adding something like

voms_userconf                   /etc/vomses/

this prevents the need for sending 'vomses' information to the myproxy server (see GetProxy Endpoint).

Self-Authorized Retrieval

By default, the MyProxy Server disallows self renewal of certificates, but this function could be enabled. By enabling this, users possessing a valid proxy certificate could in principle renew it by directly talking to the MyProxy Server, and without the intervention of any of the web components (such as the Master Portal). Of course they can only renew up to the lifetime of the long-lived proxy certificate cached in the myproxy-server.
Note that even if self-authorization is configured, configuration will NOT allow users to renew other user's credentials, because there are per-credential policies in place that restrict the access of a single proxy certificate to its own full "DN". These policies are added to each credential during their upload by the Master Portal.
In practice such a setup is hard to make secure, as it would require storing the long-lived proxy certificates without a passphrase. Additionally it is not required for the delegation scenario involving the Master Portal, therefore it is better leave this unset. A much safer alternative is the use of ssh key authentication.

Trust Root

Deploying the trust root of the Online CA (typically RCauth.eu, see also Online CA of detailed achitecture) is an important step. Make sure to install it in the directory pointed by the cert_dir above, otherwise the MyProxy Server will not allow the MasterPortal to upload the generated user proxies. Make sure that the trust root has its signing policy, crl, and subject hash links in order.

Extras

Purge Script

The MyProxy Server does not automatically remove expired or revoked credentials. Furthermore, although the Master Portal can detect if a proxy certificate stored in the MyProxy Server is expired (it can then replace it with a new one) it can not detect whether a certificate in the chain is revoked (the myproxy server does not provide that information). We therefore developed a simple myproxy-purge.sh script which is called periodically from a cron job. It will check every proxy certificate stored in the MyProxy Server and remove the ones that have expired or have been revoked. By default this script is called every 6 hours, matching the period of fetch-crl. You can change the frequency by editing /etc/cron.d/myproxy_purge.

Master Portal Configuration

The Master Portal is made up internally of a OA4MP Server and Client, as described here. Both OA4MP Server and Client are web applications packaged in .war files. In order to deploy these you will need a web container. The setup used for this includes Tomcat as the web container (following the suggestions given by OA4MP Server), with an Apache httpd server running in front of the web container, acting as a reverse http(s) proxy. Details on the reverse proxy setup can be found here and here.

Apache

The Apache server has configuration relating to the SSL setup of the Master Portal and to the (reverse) proxy setup for Tomcat. Both are configured in /etc/httpd/conf.d/ssl.conf.
Make sure mod_proxy is configured to allow all the relevant MP Client and Server endpoints to pass through, consult this page for details.
Although it is better to explicitly proxy all the required endpoints, as a bare minimum, you can use something like the following:

ProxyPass /mp-oa2-server/authorized !
ProxyPass /mp-oa2-server ajp://127.0.0.1:8009/mp-oa2-server

ProxyPass /mp-oa2-client/forwardgetcert !
ProxyPass /mp-oa2-client/startRequest !
ProxyPass /mp-oa2-client ajp://127.0.0.1:8009/mp-oa2-client

Note! In particular the /authorized endpoint should NOT be forwarded: it is a private endpoint meant only to be called from within the Master Portal.

Also make sure to define proxy rules in order of decreasing restrictiveness, i.e. with the most restrictive rule first.

Tomcat

In the Tomcat server.xml configuration make sure to have the AJP connector opened on port 8009. There are a couple of restrictions you should enable on this connector:

• Setting address="127.0.0.1" will enforce local access only on the container. Without this a user could bypass the Apache server (and authentication) by directly accessing port 8009 from the browser. This should not be possible due to firewall restrictions in the first place, but even if the firewall fails, this configuration will still keep out any access not coming from localhost.
• Setting URIEncoding="UTF-8" ensures incoming GET requests will be processed as UTF-8, instead of ISO-8859-1.
• Setting tomcatAuthentication="false" will tell Tomcat to trust the external authenticator (Apache).

<Connector port="8009" URIEncoding="UTF-8" address="127.0.0.1" tomcatAuthentication="false" protocol="AJP/1.3" redirectPort="8443"/>

Since the Master Portal is made up of two separate components, a MP Server and Client , they run as two independent web applications inside the web container. In order to be able to forward requests internally between the two (without going through the browser) the crossContext flag has to be enabled in context.xml.

<Context crossContext="true">     
...
</Context>

In order to allow the MasterPortal to send notifications upon new client registration requests, you have to also add the following resource to the context.xml file:

<Resource name="mail/Session" type="javax.mail.Session" auth="Container"></Resource>

this requires additional configuration in the config file, see email notifications for details.

MP Server Config

The MP Server is the web application called mp-oa2-server.war living in the web container. Next to the web application there are a set of additional resources used by the servers under /var/www/server structured in the following way:

• /var/www/server/conf holds server configuration files
• /var/www/server/log holds the server logs produced by the web application
• /var/www/server/storage is only used if using file system for storage backend; when using a database this directory is unused
• /var/www/server/tools holds external tools, such as the client approval cli.

Configuration

The MP Server configuration file can be found under /var/www/server/conf/cfg.xml. The configuration file is XML formatted and follows the structure give by the OA4MP Server. For more information about the standard OA4MP Server configuration, consult this page. Here we will describe the additional configuration options that are specific to the MP Server implementation.

      <sshkeys max="5"
               scope="eu.rcauth.sshkeys"
      />

      ...

      <scopes>
          <scope local="true">eu.rcauth.sshkeys</scope>
      </scopes>

      ...

      <myproxy host="credstore_host" port="credstore_port" password="credstore_pw">

                ...

                <defaultLifetime>43200</defaultLifetime>

                <validators>
                       <validator handler="eu.rcauth.masterportal.server.validators.DNValidator">
                               <input name="input_claim">cert_subject_dn</input>
                       </validator>
                       <validator handler="eu.rcauth.masterportal.server.validators.LifetimeValidator">
                               <input name="max_proxy_lifetime">950400</input>
                               <input name="tolerance">86400</input>
                       </validator>
                </validators>

      </myproxy>

The sshkeys section configures the ssh key upload API, see here for configuration details.

In order to allow configuring scopes locally for the MasterPortal (such as the scope for allowing the ssh key upload API), without forwarding them to the Delegation Server, we extend the standard scopes configuration with a boolean local attribute. We need this functionality since unknown scopes are not ignored but treated as error.

The standard myproxy configuration only allows for setting the host and port, but not the password to use when talking to a MyProxy Server. The MP Server adds the password attribute to the myproxy configuration tag. The configured password will be used when GET-ing a proxy of the stored credential. Note that this password should be the same as the password configured for myproxy in the MP Client configuration, otherwise the MP Server will be unable to retrieve proxies inserted by the MP Client.

The Master Portal expects a proxy lifetime value to be provided with a /getproxy request. This parameter, however, is optional. In case no proxy lifetime value is provided in the request, the Master Portal will default to the configured value in defaultLifetime. The defaultLifetime is set in seconds and its recommended value is 12 hours.

The MP Server introduces validators. Validators are executed during a /getproxy call in order to decide whether a stored credential is still valid, or not. The validators defined in the configuration file are executed in the order in which the appear in the file. Every validator gets as input the current transaction object, request object and the response of the MyProxy INFO command on the requested credentials. A validator can either pass or fail. In case ALL validators pass, the stored credential is deemed valid, the /getproxy call resumes and a proxy is returned. In case ANY validator fails, the stored credential is treated as invalid in which case the Master Portal will renew it by asking for a new certificate from the Delegation Server. Once the certificate is renewed, the /getproxy call resumes and a proxy is returned. The MP Server comes with two validator implementations:

• DNValidator : A user's certificate DN is constructed from the attributes released by his IdP. The way in which DNs are constructed is outlined in the RCauth.eu CP/CPS. In case the attributes released by a user's IdP change, the DN constructed for the user also changes with it. Since the Master Portal is a caching service, it caches a user's proxy certificate for its full lifetime, but it will not account for attribute/DN changes by default. Using the DNValidator will make sure that the DN of the cached certificate matches that of the DN generated from the current user attributes (from the current session). This validator takes one input:
  1. input_claim : The Master Portal does not know how to construct a DN from user attributes (that is determined by the Delegation Server), therefore the DN of the cached certificate will be compared with the DN released in the current set of claims (from the current session) by the Delegation Server. Normally this should be the cert_subject_dn claim, but it can be configured here to the name of the actual claim used for comparison.
• LifetimeValidator : Validates the requested proxy lifetime (from the /getproxy call) against the actual proxy lifetime remaining in the cached credential. This validator will check against invalid proxy lifetime requests that exceed server maximum, and against proxy lifetime requests that are larger than the time left in the cached proxy. The maximum proxy lifetime used by this validator is constructed by max_proxy_lifetime - tolerance from the following two inputs:
  1. max_proxy_lifetime : This is a Master Portal server side maximum value in seconds imposed on the proxies being retrieved via /getproxy. Recommended value is 11 days.
  2. tolerance : This is a tolerance applied to the max_proxy_lifetime. It is a small timeframe in seconds (usually a day) which prevents the Delegation Server from being flooded with requests. Without the tolerance value, consecutive /getproxy requests that keep asking for same proxy with the maximum allowed lifetime (max_proxy_lifetime) would trigger consecutive credential renewals via the Delegation Server.

In case you are interested in writing your own validators, you will need to implement the GetProxyRequestValidator interface, and rebuild the Master Portal from source.

Apart from these additions, it is also possible to add a boolean attribute enableAutoRegisterEndpoint to the 'service' node itself. This endpoint (/autoregister, disabled by default) allows clients to register without the need for manual approval. Such functionality must NOT be exposed directly to end-users, but can be used as building block, via localhost, in the construction of an OIDCfed enabled OP. Alternatively, the v0.2.0 and up client management API is a more complex but probably more secure way of doing that.

Database

Using the database backend will require you to prepare the database and its tables before using the web application. The OA4MP Server comes with a set of defined schemas used by the application. The Master Portal implementation uses an updated set of schemas which introduces two additional columns in the transactions table:

Column Name	Column Type
claims	TEXT CHARACTER SET 'utf8'
mp_client_session_identifier	TEXT

The claims column is used to store the set of released claims of the current transaction in JSON format. The mp_client_session_identifier column is used to store the session identifier used by the MP Client for the current transaction. See more about session keeping inside the Master Portal here.

In addition, it add a new table ssh_keys used for storing ssh public keys via the ssh key upload API:

Column Name	Column Type	Primary key
username	VARCHAR(255) NOT NULL	y
label	VARCHAR(255) NOT NULL	y
pub_key	TEXT NOT NULL	n
description	TEXT	n
import_time	TIMESTAMP	n

CLI

The client approval cli, oa2-cli, can be found deployed under the /var/www/server/tools directory. OA4MP is built with self-registration support. Clients can access a special /register endpoint on the server for self-registration. However, before a client is accepted by the Master Portal, it has to be approved. The oa2-cli tool is used to approve pending self-registrations from clients. The clients in this context are the portals (Science Gateways). For more information about this cli, consult this page.
The cli can also be used to create the JSON web keys, see the README for the Ansible scripts for details.

MP Client Config

The MP Client is the web application called mp-oa2-client.war living in the web container. Next to the web application there are a set of additional resources used by the client under /var/www/client structured in the following way:

/var/www/client/conf holds client configuration files
/var/www/client/log holds the client logs produced by the web application
/var/www/client/storage is only used if using file system for storage backend; when using a database this directory is unused

Configuration

The MP Client configuration file can be found under /var/www/client/conf/cfg.xml. The configuration file is XML formatted and follows the structure give by the OA4MP Client. For more information about the standard OA4MP Client configuration, consult this page. Here we will follow with describing additional configuration options that were made available in the MP Client implementation only. These do not belong to the standard OA4MP configurations.

          <myproxy host="credstore_host" port="credstore_port" password="credstore_pw">
  
             ...

          </myproxy>

Just like in case of the MP Server configuration, the MP Client also needs a password attribute in the myproxy configuration tab. The configured password will be used to STORE/PUT proxy certificates in the credential store. Note that this password should be the same as the password configured for myproxy in the MP Server configuration, otherwise the MP Server will be unable to retrieve proxies inserted by the MP Client.

Database

Using the database backend will require you to prepare the database and its tables before using the web application. The original OA4MP Client comes with a set of defined schemas used by the application. The Master Portal implementation uses an updated set of schemas which introduces two additional columns in the assets table:

Column Name	Column Type
mp_server_request_state	TEXT
mp_server_request_code	TEXT

The mp_server_request_code & mp_server_request_state are the same as the code & state parameters received by the MP Server during the current session. These two variables are there to keep session between the MP Server and Client. See more about session keeping inside the Master Portal here.

Logs

For general log messages produced by the Master Portal you should take a look at the following two log files:

/var/www/server/log/mp-oa2-server.log.0
/var/www/client/log/mp-oa2-client.log.0

If an error occurs, it is generally a good idea to check the logs of the web container (Tomcat). Exceptions and stack traces thrown by the Master Portal application will end up in this log. Here you should see (hopefully) some more details on why things have failed.

# on CentOS6
less /var/log/tomcat/catalina.out

# on CentOS7
journalctl -l -u tomcat.service 

For problems involving the MyProxy Server, you can take a look at the MyProxy logs on the Credential Store.

# on CentOS6
less /var/log/messages

# on CentOS7
journalctl -l -u myproxy-server.service