AARC Pilot - Delegation Server Administrator Guide

From GridWiki
Jump to: navigation, search



The Delegation Server is a key component in the AARC Piloting work. The Delegation Server issues certificates for authenticated users based on their released attributes. Our current Delegation Server implementation is called RCAuth.eu. This page is dedicated for Delegation Server operators and administrators who wish to fine tune their server.

In case you want to deploy the Delegation Server, make sure to check out our guide on using Ansible deployment scripts. Most of the presented configuration from below are set by the Ansible scripts.

Delegation Server Configuration

The Delegation Server is a web application packaged in a .war file, based on the OA4MP Server. In order to deploy it you will need a web container. The setup used for this includes Tomcat as the web container (following the suggestions given by the OA4MP Server), with an Apache httpd server running in front of the web container, acting as a reverse https proxy, and authenticating users through mod_shib. Details on the reverse proxy setup can be found here and here.


The Apache server holds configurations relating to the SSL setup of the Delegation Server and to the proxy setup talking to Tomcat. In case you want to tweak either of these options, you should look into /etc/httpd/conf.d/ssl.conf. Make sure to configure the mod_proxy setting to allow the Delegation Server to pass through. Your configuration should contain something like the following:

ProxyPass /oauth2/authorized !
ProxyPass /oauth2 ajp://

Note! Take special care to disallow the /authorized endpoint to be forwarded, since it's a private endpoint meant only to be called from within the server. Make sure to define proxy rules in order of precedence from most restrictive to least restrictive!

Inside the apache configuration /etc/httpd/conf.d/shib.conf you should make sure to protect the authorization endpoint of the Delegation Server to be only reachable to authenticated users. It is also a good idea to protect the /register (client self-registration) endpoint, in order to avoid the Delegation Server getting flooded with self-registration requests.

<Location /oauth2/authorize>
  AuthType shibboleth
  ShibRequestSetting requireSession 1
  ShibRequestSetting exportAssertion true
  ShibUseHeaders On
  Require valid-user

<Location /oauth2/register>
  AuthType shibboleth
  ShibRequestSetting requireSession 1
  ShibUseHeaders On
  Require valid-user

Take a look at the mod_shib documentation to see what the above options all mean.

Note! Beware of the spoofing voulnerability which occurs when using ShibUseHeaders as a means of releasing attributes to the web application. Currently OA4MP Server expects attributes through the use of headers, but as soon as it moves away from it, ShibUseHeaders should be exchanged it's alternative: ShibUseEnvironment.


In the Tomcat server.xml configuration make sure to have the AJP connector opened on port 8009. There are a couple of restrictions you should enable on this connector:

  • Setting address="" will enforce local access only on the container. Without this a user could bypass the Apache server (and authentication) by directly accessing port 8009 from the browser. This should not be possible due to firewall restrictions in the first place, but even if the firewall fails, this configuration will still keep out any access not coming from localhost.
  • Setting tomcatAuthentication="false" will tell Tomcat to trust the external authenticator (Apache).
<Connector port="8009" address="" tomcatAuthentication="false" protocol="AJP/1.3"/>

The Delegation Server is communicating with the Online CA through a dedicated private channel, using a private ssl setup (with a private trust root). The communication is initiated from the Delegation Server web application, but using the SSL context setting of its Tomcat web container. In order to initiate the ssl connection, Tomcat has to trust the private trust root created for the communication. For this you will have to set the X509_CERT_DIR environmental variable in the tomcat.conf configuration file to the location of the private trust anchor.



You will need to install and configure Shibboleth as a Service Provider which will take care of redirecting unauthenticated users to the right authentication endpoint. In case of the RCAuth.eu scenario the Shibboleth SP will have to redirect users to the WAYF. Some general information on how to configure Shibboleth SP can be found here. After successful authentication, the Shibboleth SP is expected to map and release a set of user attributes received from the IdP to the Delegation Server web application. The list of attributes expected by the Delegation Server (also outlined in the RCAuth.eu policy) is:

Name Friendly Name Mapped Name (id)
urn:oid: eduPersonPrincipalName eppn
urn:oid: eduPersonUniqueId epuid
urn:oid: eduPersonTargetedID eptid
urn:oid:2.16.840.1.113730.3.1.241 displayName displayName
urn:oid: givenName givenName
urn:oid: sn sn
urn:oid: cn cn
urn:oid: o o
urn:oid:0.9.2342.19200300.100.1.3 mail mail
urn:oid: schacHomeOrganization schacHomeOrganization

Next to the attributes mapped above, there is an additional attribute called Shib-Authenticating-Authority. Since the Shibboleth SP is always talking to the same WAYF, the entity ID of the IdP that the Delegation Server is talking to is always going to be the WAYF, and not the users' real authenticating IdP. There is, however, the Authenticating Authority attribute which can also be mapped from the incoming SAML response. To do so, you will have to configure an attribute extractor in the shibboleth2.xml:

<AttributeExtractor type="Chaining">
      <AttributeExtractor type="Assertion" AuthenticatingAuthority="Shib-Authenticating-Authority"/>

Delegation Server

The Delegation Server is the web application called oauth2.war living in the web container. Next to the web application there are a set of resources used by the server under /var/www/server structured in the following way:

  • /var/www/server/conf holds server configuration files
  • /var/www/server/log holds the server logs produced by the web application
  • /var/www/server/storage is only used if using file system for storage backend; when using a database this directory is unused
  • /var/www/server/tools holds external tools, such as client approval cli.
  • /var/www/server/certificates is an alternative to /etc/grid-security/certificates holding only the private trust root used for connecting to the Online CA. See Tomcat configuration.


The Delegation Server configuration file can be found under /var/www/server/conf/cfg.xml. The configuration file is XML formatted and follows the structure give by the OA4MP Server. For more information about the standard OA4MP Server configuration, consult this page. Here we will follow with describing additional configuration options that were made available in the Delegation Server implementation only. These do not belong to the standard OA4MP configurations.

Trace Logging

For traceability reasons outlined in the RCAuth.eu policy document, special log entries had to be introduced in order to keep record of mappings between SAML user attributes and their generated certificate. In order not to pollute the already existing server log file, we decided to add a separate logging capability, trace logging, which would log to a separate file. You can configure the trace logging capabilities by adding the following section to the server configuration file:

      <traceLogging  logFileName="/var/www/server/log/trace.log"

The traceLogging tag supports the same configuration options as the OA4MP server's logging tag. When setting the debug flag false you get a limited trace after a users' visit including the information: raw SAML response including user attributes, source and original attribute form of subject DN components (see DN Generator) and the final subject DN. When setting the debug flag true you will get a detailed trace of how attribute were used and transformed.

Claim Mapping

The OA4MP implementation has a scopes configuration element by which it offers support for additional scopes (other than its predefined set in OA2Scopes.basicScopes). The Delegation Server implementation extends this configuration element, allowing the definition of claim mapping for a within a specific scope:

      <scopes handler="org.delegserver.oauth2.DSDynamicScopeHandler">

          <scope name="edu.uiuc.ncsa.myproxy.getcert">
               <claim name="cert_subject_dn">X509_CERT_SUBJECT</claim>

          <scope name="email">
               <claim name="email">mail</claim>

          <scope name="openid">

          <scope name="profile">
               <claim name="given_name">givenName</claim>
               <claim name="family_name">sn</claim>

          <scope name="org.cilogon.userinfo">
               <claim name="idp">Shib-Authenticating-Authority</claim>
               <claim name="idp_display_name">o</claim>
               <claim name="eduPersonTargetedID">eptid</claim>
               <claim name="eduPersonPrincipalName">eppn</claim>
               <claim name="oidc">oidc</claim>
               <claim name="affiliation">affiliation</claim>
               <claim name="ou">ou</claim>
               <claim name="name">displayName</claim>


The handler attribute of the outer scopes element defines which handler implementation to use. More on custom scope handlers here.

Every inner scope definition adds support for a scope. Within each scope definition, multiple claim definition can appear. This means that for a specific scope requested, the Delegation Server will return a set of defined claims. A scope definition without any claims is also valid. The names of the scopes and claims defined in the Delegation Server are set according to the OIDC Scopes and Claims.

A claim element is defined by its name attribute, and its inner value defines the source of that claim. Claim are built from user attributes (usually released by the users' home IdP). You will see the correlation between claims defined here and the Mapped Name (id) column of the shibboleth attributes table. Note that the X509_CERT_SUBJECT is not defined in the shibboleth attribute mapping, since it's not a user attribute released by the IdP, rather a new attribute defined by the Delegation Server. See the DN Generator section for more details on this attribute.

DN Generator

The Delegation Server creates user certificate for authenticated users. In order to provide a unique certificate for every authenticated user, the certificate subject DN is generated from a set of user attributes in a way that ensures uniqueness. See the RCAuth.eu policy document on details about how DNs are generated. In order to make DN generation configurable and flexible the Delegation Server offers a configuration block that can be used for this:

      <dnGenerator attributeName="X509_CERT_SUBJECT" type="rfc2253" baseDN="DC=rcauth-clients,DC=rcauth,DC=eu">



               <source filter="url">o</source>

               <source name="email">mail</source>


The dnGenerator element contains sets of source attributes for various DN components. The source attributes, just like in case of the Claim Mapping, are the user attributes released by the user's IdP (shibboleth attributes). The dnGenerator element contains three attributes, all referring to the new DN attribute being created:

  • attributeName : The name of the DN attribute created by this generator. This is an internal attribute name which can be used to reference the generated DN within the Delegation Server. This attribute can be used internally as source attribute for claim mappings.
  • type : Defines the formatting of the generated DN. Accepted values are rfc2253 and openssl. If the attribute is missing, the generated type defaults to rfc253. Note that the type of the following baseDN attribute has to match the type specified here.
  • baseDN : The base DN to which the generated DN gets appended. This is a static part of the DN that every certificate generated by this Delegation Server will have. Note that the format type of the baseDN should match the type attribute. In case of missing type attribute the baseDN should be provided in rfc2253 format.

The dnGenerator generates partial subject DNs of the following format:

/O=organisation/CN=cnName cnUniqueId [sequence_number]

With the following generated elements:

  • organisation : Contains an ordered list of source attributes for generating the 'O' RDN.
  • cnName : Contains an ordered list of source attributes for generating the display name part of the 'CN' RDN.
  • cnUniqueId : Contains an ordered list of source attributes for generating the unique identifier part of the 'CN' RDN.
  • sequence_number : Given enough suspicion of clashing DNs for different users, an incremental sequence number gets appended to the generating DN. This does not have any configuration option, and will be absent in most cases.

Every list of source attributes (organisation, cnName and cnUniqueId) has to be ordered from the MOST preferred attribute to the LEAST preferred attribute. The first source attribute from a list that will have a non-empty user attribute present (from the shibboleth attribute set) will be taken as the source value for that particular DN element.

The generated partial subject DN will get appended to the baseDN provided as an attribute in the configuration.


The Delegation Server also supports adding extension into the generated certificate. Since the MyProxy GET Command is not designed to transport any additional user extension requests, the USERNAME field is overloaded for this purpose. The MyProxy Server is then responsible to strip any additional extension requests using the certificate_issuer_program (see man myproxy_server.config).

The only supported extension for now is the email address of the authenticated user inserted in the certificate's subjectAltName. It can be configured by setting adding an extension with the appropriate name and source value. See the example above. The extension source values work the same way as other source values of the dnGenerator element.

Attribute Filters

Sometimes there is some special transformation that a source user attribute has to go through before it being used for building the subject DN. To accommodate these special corner cases that only apply to a limited set of attributes (depending on the attribute itself) we developed Attribute Filters. Attribute Filters take a single user attribute as input, do some transformation on them, and then give them back to the processing pipeline. You can develop your own attribute filter by implementing the ShibAttributeFilter interface and configuring your implementation as such:

               <filter name="url">org.delegserver.oauth2.shib.filters.URLDomainNameFilter</filter>

The attributeFilters definition does not belong under dnGenerator element, simply add it to the service element. Every attribute filter defined here will have a name by which it can be referenced in the rest of the configuration file. In order to apply a filter on a specific source user attribute simply add the filter attribute with the appropriate name to the desired source. See the example above.

In the example above the URLDomainNameFilter is used to process a URL by extracting its domain name component.


Using the database backend will require you to prepare the database and its tables before using the web application. The original OA4MP Server comes with a set of defined schemas used by the application. The Delegation Server implementation uses an updated set of schemas which adds an additional table called trace_records and introduces changes in the clients and transactions tables.

The clients table will have one additional column holding a description about the registered client. This description text about the client is presented to the user during the consent giving.

Column Name Column Type
description TEXT CHARACTER SET 'utf8'

The transactions table will have three additional columns:

Column Name Column Type
claims TEXT CHARACTER SET 'utf8'
user_attributes TEXT CHARACTER SET 'utf8'
trace_record TEXT

The new table called trace_records holds a single entry for every authenticated user who ever requested a certificate. This table is kept for two reasons: to avoid issuing the same certificate DN to two separate users, and to keep a traceable record of usage. The table looks like the following:

Column Name Column Type
cn_hash CHAR(44)
attribute_hash CHAR(44)
attribute_salt CHAR(44)
attribute_names TEXT
last_seen TIMESTAMP

Note that no actual user attributes are kept in this table, only one way hashes!


The client approval cli, oa2-cli, can be found deployed under the /var/www/server/tools directory. OA4MP is built with self-registration support. Clients can access a special /register endpoint on the Delegation Server for self-registration. However, before a client is accepted by the Delegation Server, it has to be approved. The oa2-cli tool is used to approve pending self-registrations from clients. The clients in this context are the Master Portals. For more information about this cli, consult this page.

The Delegation Server extends the regular OA4MP cli in order to accommodate the database changes introduced in the clients table. With the extended cli, you can add or modify the description field of registered clients (Master Portals).


The Online CA creates new CRLs, which it syncs to the Delegation Server. In order to make these CRLs public and available the Delegation Server fetches the synced CRL and publishes it into the httpd web root. This way anybody can access the CRL via a web call to the Delegation Server (https://delegation-server/crl.pem). A cron job is responsible for refreshing the CRL on the Delegation Server every minute. In case you want to tweak this interval, edit the /etc/cron.d/crl_publish file.


For general log messages produced by the Delegation Server you should take a look at the following two log file:


For trace logs you should take a look at the file below. Read more about trace logging here.


If an error occurs, it is generally a good idea to check the logs of the web container (Tomcat). Exceptions and stack traces thrown by the Delegation Server application will end up in this log. Here you should see (hopefully) some more details on why things have failed.

# on CentOS6
less /var/log/tomcat/catalina.out

# on CentOS7
journalctl -l -u tomcat.service 

For problems involving the MyProxy Server, you will have to investigate the logs on the Online CA. This is not a trivial task, since you will need physical access to the box in order to inspect the logs in real time. However, the Online CA backs up its log to the Delegation Server, and therefore you can view its outdated (a day old) logs from

Personal tools