Difference between revisions of "Master Portal Administrator Guide"

From PDP/Grid Wiki
Jump to navigationJump to search
(tomcat v1)
 
(33 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
= Introduction =
 
= Introduction =
  
The Master Portal is a central component in the [[AARC_Pilot | AARC Piloting]] work. The Master Portal caches long lived user proxies into its Credential Store, and returns short lived proxies on demand to a trusted portal (Science Gateway) for authenticated users. This page is dedicated for Master Portal operators and administrators who wish to configure their Master Portals.  
+
The Master Portal is a central component in the [[RCauth.eu_and_MasterPortal_overview | RCauth architecture]]. The Master Portal caches long lived user proxies into its Credential Store, and returns short lived proxies on demand to a trusted portal (Science Gateway) for authenticated users. This page is intended for Master Portal operators and administrators who wish to configure their Master Portals.
 +
 
 +
In case you want to '''deploy''' a Master Portal, see this [[RCauth_Delegation_Server_%26_MasterPortal_-_Ansible_scripts | guide]] for using our Ansible deployment scripts.
 +
This page highlights the important parts of the configuration.
  
 
= Credential Store Configuration =
 
= Credential Store Configuration =
  
The Master Portal cannot be operated without its Credential Store. Missing the Credential Store from your setup will render your Master Portal unable to cache and retrieve any proxy certificates. Without the Credential Store a Master Portal can still be used for user authentication (via [https://docs.google.com/document/d/1cs3peO9FxA81KN-1RC6Z-auEFIwRbJpZ-SFuKbQzS50/pub OIDC]), but it's [[OAuth_for_MyProxy_GetProxy_Endpoint | GetProxy Endpoint]] will be unable to retrieve proxy certificates.
+
The Master Portal cannot be operated without its Credential Store. Missing the Credential Store from your setup will render your Master Portal unable to cache and retrieve any proxy certificates. Without the Credential Store a Master Portal could still be used for user authentication (via [http://goo.gl/VnMKXS OIDC/OA4MP]), but its [[OAuth_for_MyProxy_GetProxy_Endpoint | GetProxy Endpoint]] will be unable to retrieve proxy certificates.
  
 
== MyProxy Server ==
 
== MyProxy Server ==
  
The Credential Store is nothing else than a simple [http://grid.ncsa.illinois.edu/myproxy/serverinstall.html MyProxy Server] installation running in Credential Store mode. It being an unmodified MyProxy Server, you can tailor its configuration to your needs following the official MyProxy Server [http://grid.ncsa.illinois.edu/myproxy/adminguide.html admin guide]. In order to run it as a Credential Store backend for the Master Portal, you will need the following set of configurations:
+
The Credential Store is just a [http://grid.ncsa.illinois.edu/myproxy/serverinstall.html MyProxy Server] installation running in Credential Store mode. It being an unmodified MyProxy Server, you can tailor its configuration to your needs following the official MyProxy Server [http://grid.ncsa.illinois.edu/myproxy/adminguide.html admin guide]. In order to run it as a Credential Store backend for the Master Portal, you will need the following set of configurations:
  
 
  accepted_credentials            <Master Portal Host DN>
 
  accepted_credentials            <Master Portal Host DN>
 
  authorized_retrievers          <Master Portal Host DN>
 
  authorized_retrievers          <Master Portal Host DN>
authorized_renewers            <Master Portal Host DN>
+
 
default_authorized_renewers    <Master Portal Host DN>
 
trusted_retrievers              <Master Portal Host DN>
 
 
 
  cert_dir                        /etc/grid-security/certificates
 
  cert_dir                        /etc/grid-security/certificates
 
  max_proxy_lifetime              264
 
  max_proxy_lifetime              264
  
The Master Portal authenticates to the MyProxy Server via its host certificates. Make sure to configure the DN of the host certificate of the Master Portal ('''<Master Portal Host DN>''') as the authorized party in the MyProxy Server configuration snippet above. If you are interested to see what each of those individual authrizations stand for please consult the man page of myproxy-server.config. The '''cert_dir''' has to point to the set of trust roots trusted by the MyProxy Server. The '''max_proxy_lifetime''' determines the maximum lifetime in hours that a returned proxy credential can have. Since the Master Portal is only allowed to release short lived proxy certificates, a lifetime of 264 hours (11 days) is imposed by default. The MyProxy Server will be able to release proxies with shorter lifetime on demand, but not with bigger lifetime than that of '''max_proxy_lifetime'''.
+
You should not need to also set ''authorized_renewers'', ''default_authorized_renewers'' or ''trusted_retrievers''.
 +
 
 +
The Master Portal authenticates to the MyProxy Server via its host certificates. Make sure to configure the DN of the host certificate of the Master Portal ('''<Master Portal Host DN>''') as the authorized party in the MyProxy Server configuration snippet above. The meaning of each of the individual authorizations can be found in the man page of myproxy-server.config.<br>
 +
The '''cert_dir''' has to point to the set of trust roots trusted by the MyProxy Server.<br>
 +
The '''max_proxy_lifetime''' determines the maximum lifetime in hours that a returned proxy credential can have. I.e. one can request shorter-lived proxies, not longer-lived. Since the Master Portal is only allowed to release short lived proxy certificates, a lifetime of 264 hours (11 days) is imposed by default (additionally, the current RCauth online CA produces EECs with a lifetime of 11 days making it meaningless to create proxies with longer lifetime).
  
 
=== VOMS Support ===
 
=== VOMS Support ===
  
The MyProxy Server returns returns non-vomsified proxies by default, but it has support for releasing VOMS-ified proxies as well. Asking for VOMS-ified proxies is also supported in the Master Portal via the [[OAuth_for_MyProxy_GetProxy_Endpoint | GetProxy Endpoint]]. In order to enable VOMS support in the MyProxy Server, set '''allow_voms_attribute_requests'''
+
The MyProxy Server returns returns plain (i.e. non-VOMS) proxies by default, but it has support for releasing VOMS-ified proxies as well. Asking for VOMS-ified proxies is also supported in the Master Portal via the [[OAuth_for_MyProxy_GetProxy_Endpoint | GetProxy Endpoint]]. In order to enable VOMS support in the MyProxy Server, set '''allow_voms_attribute_requests'''
  
 
  allow_voms_attribute_requests  True
 
  allow_voms_attribute_requests  True
  
=== Self-Authorized Rertieval ===
+
Additionally, one can add local configuration for supported VOs, by adding something like
 +
 
 +
voms_userconf                  /etc/vomses/
  
The MyProxy Server disallows self renewal of certificates by default, but this function can be enabled. By enabling this, users possessing a valid proxy certificate will be able to renew it (prolong its lifetime) by directly talking to the MyProxy Server, and without the intervention of any of the web components (such as the Master Portal). This setup is still experimental and it does not affect the [[AARC_Pilot_-_Architecture#Detailed_Flow | delegation scenario]] involving the Master Portal, therefore it is safe to unset when not needed.  
+
this prevents the need for sending 'vomses' information to the myproxy server (see [[OAuth_for_MyProxy_GetProxy_Endpoint | GetProxy Endpoint]]).
  
authorized_renewers            "/DC=NL/DC=Example/O=*"
+
=== Self-Authorized Retrieval ===
allow_self_authorization        True
 
  
In the above configuration snippet the '''authorized_renewers''' should contain a "DN regex". This regex should be the common bit of the DNs of the proxy certificates stored in the MyProxy Server. This configuration will '''NOT''' allow users to renew other user's credentials, because there are per-credential policies in place that restricts the access of a single proxy certificate to it's own full "DN regex". These policies are added to each credential during their upload by the Master Portal.
+
By default, the MyProxy Server disallows self renewal of certificates, but this function could be enabled. By enabling this, users possessing a valid proxy certificate could in principle renew it by directly talking to the MyProxy Server, and without the intervention of any of the web components (such as the Master Portal). Of course they can only renew up to the lifetime of the long-lived proxy certificate cached in the myproxy-server.<br>
 +
Note that even if self-authorization is configured, configuration will '''NOT''' allow users to renew other user's credentials, because there are per-credential policies in place that restrict the access of a single proxy certificate to its own full "DN". These policies are added to each credential during their upload by the Master Portal.<br>
 +
In practice such a setup is hard to make secure, as it would require storing the long-lived proxy certificates without a passphrase. Additionally it is not required for the [[RCauth.eu_and_MasterPortal_architecture#Detailed_Flow | delegation scenario]] involving the Master Portal, therefore it is better leave this unset. A much safer alternative is the use of [[Master_Portal_sshkey_endpoint | ssh key authentication]].<br>
  
 
== Trust Root ==
 
== Trust Root ==
  
Deploying the trust root of the Online CA (see Online CA of [[AARC_Pilot_-_Architecture#Detailed_Architecture | detailed achitecture]]) is an important step. Make sure to deploy this trust root in the directory pointed by the '''cert_dir''' above, otherwise the MyProxy Server will not accept the generated user proxies the Master Portal will try to upload in its store. Make sure that the trust root has its signing policy, crl, and subject hash links in order.
+
Deploying the trust root of the Online CA (typically RCauth.eu, see also Online CA of [[RCauth.eu_and_MasterPortal_architecture#Detailed_Architecture | detailed achitecture]]) is an important step. Make sure to install it in the directory pointed by the '''cert_dir''' above, otherwise the MyProxy Server will not allow the MasterPortal to upload the generated user proxies. Make sure that the trust root has its signing policy, crl, and subject hash links in order.
  
 
== Extras ==
 
== Extras ==
Line 45: Line 52:
 
=== Purge Script ===
 
=== Purge Script ===
  
The MyProxy Server doesn't remove expired or revoked credentials. The Master Portal can detect if a proxy certificate stored in the MyProxy Server is expired and it can replace it with a new one, but it cannot detect revoked certificates. Because of this, it can happen that the MyProxy Server can contain expired or revoked certificates. In order to account for this (mostly because a revoked certificate poses a threat) we developed a script called [http://ndpfsvn.nikhef.nl/viewvc/pdpsoft/trunk/aarc.master-portal/ansible/roles/credstore/files/myproxy_purge.sh?revision=3073&content-type=text%2Fplain&view=co '''myproxy-purge.sh''']. The '''myproxy-purge.sh''' script is periodically called by a cron job to check every proxy certificate stored in the MyProxy Server and remove the ones that have expired or have been revoked. By default this script is called every 6 hours. You can change this frequency by editing '''/etc/cron.d/myproxy_purge'''.
+
The MyProxy Server does not automatically remove expired or revoked credentials. Furthermore, although the Master Portal can detect if a proxy certificate stored in the MyProxy Server is expired (it can then replace it with a new one) it can '''not''' detect whether a certificate in the chain is revoked (the myproxy server does not provide that information). We therefore developed a simple [https://github.com/rcauth-eu/aarc-ansible-master-portal/blob/0.2-devel/roles/credstore/files/myproxy_purge.sh myproxy-purge.sh script] which is called periodically from a [https://github.com/rcauth-eu/aarc-ansible-master-portal/blob/0.2-devel/roles/credstore/files/myproxy_purge cron job]. It will check every proxy certificate stored in the MyProxy Server and remove the ones that have expired or have been revoked. By default this script is called every 6 hours, matching the period of fetch-crl. You can change the frequency by editing '''/etc/cron.d/myproxy_purge'''.
  
 
= Master Portal Configuration =
 
= Master Portal Configuration =
  
The Master Portal is made up internally of a [http://grid.ncsa.illinois.edu/myproxy/oauth/ OA4MP] Server and Client, just as described [[Master_Portal_Internals#Structure | here]]. Both OA4MP Server and Client are web applications packaged in .war files. In order to deploy these you will need a web container. The setup used for this includes [https://tomcat.apache.org/tomcat-7.0-doc/index.html Tomcat] as the web container (following the suggestions given by [http://grid.ncsa.illinois.edu/myproxy/oauth/server/manuals/getting-started.xhtml OA4MP Server]), with an [https://httpd.apache.org/ Apache] httpd server running in front of the web container, acting as a reverse https proxy. Details on the reverse proxy setup can be found [https://tomcat.apache.org/tomcat-7.0-doc/proxy-howto.html here] and [http://grid.ncsa.illinois.edu/myproxy/oauth/server/manuals/getting-started.xhtml here].
+
The Master Portal is made up internally of a [http://grid.ncsa.illinois.edu/myproxy/oauth/ OA4MP] Server and Client, as described [[Master_Portal_Internals#Structure | here]]. Both OA4MP Server and Client are web applications packaged in .war files. In order to deploy these you will need a web container. The setup used for this includes [https://tomcat.apache.org/tomcat-7.0-doc/index.html Tomcat] as the web container (following the suggestions given by [http://grid.ncsa.illinois.edu/myproxy/oauth/server/manuals/getting-started.xhtml OA4MP Server]), with an [https://httpd.apache.org/ Apache] httpd server running in front of the web container, acting as a reverse http(s) proxy. Details on the reverse proxy setup can be found [https://tomcat.apache.org/tomcat-7.0-doc/proxy-howto.html here] and [http://grid.ncsa.illinois.edu/myproxy/oauth/server/manuals/getting-started.xhtml here].
  
 
== Apache ==
 
== Apache ==
  
The Apache server holds configurations relating to the SSL setup of the Master Portal and to the proxy setup talking to Tomcat. In case you want to tweak either of these options, you should look into '''/etc/httpd/conf.d/ssl.conf'''. Make sure to configure the [https://httpd.apache.org/docs/current/mod/mod_proxy.html mod_proxy] setting to allow for both MP Client and Server to pass through. For a list of valid endpoints on the Master Portal, you can consult [[Master_Portal_Internals#Endpoints | this]] page. Your configuration should contain something like the following:  
+
The Apache server has configuration relating to the SSL setup of the Master Portal and to the (reverse) proxy setup for Tomcat. Both are configured in '''/etc/httpd/conf.d/ssl.conf'''.<br>
 +
Make sure [https://httpd.apache.org/docs/current/mod/mod_proxy.html mod_proxy] is configured to allow all the relevant MP Client and Server endpoints to pass through, consult [[Master_Portal_Internals#Endpoints | this page]] for details.<br>
 +
Although it is better to explicitly proxy all the required endpoints, as a bare minimum, you can use something like the following:  
  
 
  ProxyPass /mp-oa2-server/authorized !
 
  ProxyPass /mp-oa2-server/authorized !
Line 62: Line 71:
 
  ProxyPass /mp-oa2-client ajp://127.0.0.1:8009/mp-oa2-client
 
  ProxyPass /mp-oa2-client ajp://127.0.0.1:8009/mp-oa2-client
  
'''Note!''' Take special care to disallow the /authorized endpoint to be forwarded, since it's a [http://grid.ncsa.illinois.edu/myproxy/oauth/server/manuals/authorized.xhtml private endpoint] meant only to be called from within the Master Portal. Make sure to define proxy rules in order of precedence from most restrictive to least restrictive!
+
'''Note!''' In particular the /authorized endpoint should '''NOT''' be forwarded: it is a [http://grid.ncsa.illinois.edu/myproxy/oauth/server/manuals/authorized.xhtml private endpoint] meant only to be called from within the Master Portal.  
 +
 
 +
Also make sure to define proxy rules in order of decreasing restrictiveness, i.e. with the most restrictive rule first.
  
 
== Tomcat ==
 
== Tomcat ==
  
In the Tomcat '''server.xml''' configuration make sure to have the AJP connector opened on port 8009. There are a couple of restrictions you should enable on this connector  
+
In the Tomcat '''server.xml''' configuration make sure to have the [http://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html AJP connector] opened on port 8009. There are a couple of restrictions you should enable on this connector:
  
 
* Setting address="127.0.0.1" will enforce local access only on the container. Without this a user could bypass the Apache server (and authentication) by directly accessing port 8009 from the browser. This should not be possible due to firewall restrictions in the first place, but even if the firewall fails, this configuration will still keep out any access not coming from localhost.
 
* Setting address="127.0.0.1" will enforce local access only on the container. Without this a user could bypass the Apache server (and authentication) by directly accessing port 8009 from the browser. This should not be possible due to firewall restrictions in the first place, but even if the firewall fails, this configuration will still keep out any access not coming from localhost.
* Setting tomcatAuthentication="false" will tell [http://tomcat.apache.org/tomcat-7.0-doc/config/ajp.html Tomcat] to trust the external authenticator (Apache).
+
* Setting tomcatAuthentication="false" will tell Tomcat to trust the external authenticator (Apache).
 +
* Setting URIEncoding="UTF-8" ensures incoming GET requests will be processed as UTF-8, instead of ISO-8859-1.
 +
 
 +
<Connector port="8009" URIEncoding="UTF-8" address="127.0.0.1" tomcatAuthentication="false" protocol="AJP/1.3" redirectPort="8443"/>
 +
 
 +
Since the Master Portal is made up of two separate components, a [[Master_Portal_Internals#Structure | MP Server and Client ]], they run as two independent web applications inside the web container. In order to be able to forward requests internally between the two (without going through the browser) the [https://tomcat.apache.org/tomcat-7.0-doc/config/context.html crossContext] flag has to be enabled in '''context.xml'''.
 +
 
 +
<Context crossContext="true">   
 +
...
 +
</Context>
 +
 
 +
In order to allow the MasterPortal to send notifications upon new client registration requests, you have to also add the following resource to the context.xml file:
  
  <Connector port="8009" address="127.0.0.1" tomcatAuthentication="false" protocol="AJP/1.3"/>
+
  <Resource name="mail/Session" type="javax.mail.Session" auth="Container"></Resource>
  
* tomcat cross context , disable tomcat_auth, set listening address
+
this requires additional configuration in the [[#Configuration | config file]], see [http://grid.ncsa.illinois.edu/myproxy/oauth/server/configuration/server-email.xhtml email notifications] for details.
  
 
== MP Server Config ==
 
== MP Server Config ==
 +
 +
The MP Server is a web application called '''mp-oa2-server.war''' living in the web container. It uses a set of files under '''/var/www/server''' structured in the following way:
 +
 +
* '''/var/www/server/conf''' holds server configuration files
 +
* '''/var/www/server/log''' holds the server logs produced by the web application
 +
* '''/var/www/server/storage''' is only used if using file system for storage backend; when using a database this directory is unused
 +
* '''/var/www/server/tools''' holds external tools, such as the [http://grid.ncsa.illinois.edu/myproxy/oauth/server/manuals/cli.xhtml client approval cli].
 +
 +
=== Configuration ===
 +
 +
The MP Server configuration file can be found at '''/var/www/server/conf/cfg.xml'''. The configuration file is XML formatted and follows the structure give by the [http://grid.ncsa.illinois.edu/myproxy/oauth/server/index.xhtml OA4MP Server]. For more information about the standard OA4MP Server configuration, consult [http://grid.ncsa.illinois.edu/myproxy/oauth/server/configuration/config-index.xhtml this] page. Here we will describe the additional configuration options that are specific to the MP Server implementation.
 +
 +
      <sshkeys max="5"
 +
                scope="eu.rcauth.sshkeys"
 +
      />
 +
 +
      ...
 +
 +
      <scopes>
 +
          <scope local="true">eu.rcauth.sshkeys</scope>
 +
      </scopes>
 +
 +
      ...
 +
 +
      <myproxy host="credstore_host" port="credstore_port" password="credstore_pw">
 +
 +
                ...
 +
 +
                <defaultLifetime>43200</defaultLifetime>
 +
 +
                <validators>
 +
                        <validator handler="eu.rcauth.masterportal.server.validators.DNValidator">
 +
                                <input name="input_claim">cert_subject_dn</input>
 +
                        </validator>
 +
                        <validator handler="eu.rcauth.masterportal.server.validators.LifetimeValidator">
 +
                                <!--
 +
                                    max_proxy_lifetime is set in the myproxy-server.conf (converted into seconds)
 +
                                -->
 +
                                <input name="max_proxy_lifetime">950400</input>
 +
                                <!--
 +
                                    the tolerance is a small timeframe in seconds (usually a day) which prevents
 +
                                    the Delegation Server from being flooded with requests.
 +
                                -->
 +
                                <input name="tolerance">86400</input>
 +
                        </validator>
 +
                </validators>
 +
 +
      </myproxy>
 +
 +
The '''sshkeys''' section configures the ssh key upload API, see [[Master_Portal_sshkey_endpoint#Configuration | here]] for configuration details.
 +
 +
In order to allow configuring scopes locally for the MasterPortal (such as the scope for allowing the [[Master_Portal_sshkey_endpoint#Configuration | ssh key upload API]]), without forwarding them to the Delegation Server, we extend the [http://grid.ncsa.illinois.edu/myproxy/oauth/common/configuration/scopes.xhtml standard] '''scopes''' configuration with a boolean '''local''' attribute. We need this functionality since unknown scopes are not ignored but treated as error.
 +
 +
The [http://grid.ncsa.illinois.edu/myproxy/oauth/server/configuration/server-myproxy.xhtml standard] '''myproxy''' configuration only allows for setting the host and port, but not the password to use when talking to a MyProxy Server. The MP Server adds the '''password''' attribute to the myproxy configuration tag. The configured password will be used when GET-ing a proxy of the stored credential. Note that this password should be the same as the password configured for myproxy in the [[#MP_Client_Config | MP Client configuration]], otherwise the MP Server will be unable to retrieve proxies inserted by the MP Client.
 +
 +
The Master Portal expects a proxy lifetime value to be provided with a [[OAuth_for_MyProxy_GetProxy_Endpoint#Protocol_Specification | /getproxy]] request. This parameter, however, is optional. In case no proxy lifetime value is provided in the request, the Master Portal will default to the configured value in '''defaultLifetime'''. The '''defaultLifetime''' is set in seconds and its recommended value is 12 hours.
 +
 +
The MP Server introduces '''validators'''. Validators are executed during a [[OAuth_for_MyProxy_GetProxy_Endpoint | /getproxy]] call in order to decide whether a stored credential is still valid, or not. The validators defined in the configuration file are executed in the order in which the appear in the file. Every validator gets as input the current transaction object, request object and the response of the MyProxy INFO command on the requested credentials. A validator can either pass or fail. In case ''ALL'' validators pass, the stored credential is deemed valid, the /getproxy call resumes and a proxy is returned. In case ''ANY'' validator fails, the stored credential is treated as invalid in which case the Master Portal will renew it by asking for a new certificate from the Delegation Server. Once the certificate is renewed, the /getproxy call resumes and a proxy is returned. The MP Server comes with two validator implementations:
 +
 +
* '''DNValidator''' : A user's certificate DN is constructed from the attributes released by his IdP. The way in which DNs are constructed is outlined in the RCauth.eu [https://www.rcauth.eu/policy/DutchGrid-RCauth-Pilot-ICA-G1-CPCPS.pdf CP/CPS]. In case the attributes released by a user's IdP change, the DN constructed for the user also changes with it. Since the Master Portal is a caching service, it caches a user's proxy certificate for its full lifetime, but it will not account for attribute/DN changes by default. Using the '''DNValidator''' will make sure that the DN of the cached certificate matches that of the DN generated from the current user attributes (from the current session). This validator takes one input:
 +
*# '''input_claim''' : The Master Portal does not know how to construct a DN from user attributes (that is determined by the Delegation Server), therefore the DN of the cached certificate will be compared with the DN released in the current set of claims (from the current session) by the Delegation Server. Normally this should be the [http://goo.gl/VnMKXS#h.9om6a4erbsvr cert_subject_dn] claim, but it can be configured here to the name of the actual claim used for comparison.
 +
* '''LifetimeValidator''' : Validates the requested proxy lifetime (from the [[OAuth_for_MyProxy_GetProxy_Endpoint#Protocol_Specification | /getproxy ]] call) against the actual proxy lifetime remaining in the cached credential. This validator will check against invalid proxy lifetime requests that exceed server maximum, and against proxy lifetime requests that are larger than the time left in the cached proxy. The maximum proxy lifetime used by this validator is constructed by ''max_proxy_lifetime - tolerance'' from the following two inputs:
 +
*# '''max_proxy_lifetime''' : This is a Master Portal server side maximum value in seconds imposed on the proxies being retrieved via /getproxy. Recommended value is 11 days.
 +
*# '''tolerance''' : This is a tolerance applied to the '''max_proxy_lifetime'''. It is a small timeframe in seconds (usually a day) which prevents the Delegation Server from being flooded with requests. Without the tolerance value, consecutive /getproxy requests that keep asking for same proxy with the maximum allowed lifetime ('''max_proxy_lifetime''') would trigger consecutive credential renewals via the Delegation Server.
 +
 +
In case you are interested in writing your own validators, you will need to implement the [https://github.com/rcauth-eu/aarc-master-portal/blob/master/master-portal/master-portal-server/src/main/java/eu/rcauth/masterportal/server/validators/GetProxyRequestValidator.java GetProxyRequestValidator] interface, and [[RCauth_Delegation_Server_%26_MasterPortal_-_Building_from_Source#Building_the_Master_Portal | rebuild]] the Master Portal from source.
 +
 +
Apart from these additions, it is also possible to add a boolean attribute '''enableAutoRegisterEndpoint''' to the 'service' node itself. This endpoint (/autoregister, disabled by default) allows clients to register without the need for manual approval. Such functionality must '''NOT''' be exposed directly to end-users, but can be used as building block, via localhost, in the construction of an [https://openid.net/specs/openid-connect-federation-1_0.html OIDCfed] enabled OP.
 +
Alternatively, the v0.2.0 and up [https://github.com/rcauth-eu/aarc-master-portal/blob/0.2.0-release/RELEASE-NOTES.md#client-management-api client management API] is a more complex but probably more secure way of doing that.
 +
 +
=== Database ===
 +
 +
Using the database backend will require you to prepare the database and its tables before using the web application. The OA4MP Server comes with a set of defined [https://github.com/rcauth-eu/OA4MP/blob/v4.2-RCauth-1/oa4mp-server-admin-oauth2/src/main/resources/oauth2-mysql.sql schemas] used by the application. The Master Portal implementation uses an [https://github.com/rcauth-eu/aarc-ansible-master-portal/blob/0.2-devel/roles/masterportal/templates/mp-oa2-server-mysql.sql.j2 updated set of schemas] which introduces two additional columns in the '''transactions''' table:
 +
 +
{| class="wikitable" style="width: 55%;"
 +
|| '''Column Name'''
 +
|| '''Column Type'''
 +
 +
|-
 +
|| claims
 +
|| TEXT CHARACTER SET 'utf8'
 +
 +
|-
 +
|| mp_client_session_identifier
 +
|| TEXT
 +
 +
|}
 +
 +
The '''claims''' column is used to store the set of released claims of the current transaction in JSON format. The '''mp_client_session_identifier''' column is used to store the session identifier used by the MP Client for the current transaction. See more about session keeping inside the Master Portal [[Master_Portal_Internals#Session_keeping | here]].
 +
 +
In addition, it add a new table '''ssh_keys''' used for storing ssh public keys via the [[Master_Portal_sshkey_endpoint | ssh key upload API]]:
 +
 +
{| class="wikitable" style="width: 55%;"
 +
|| '''Column Name'''
 +
|| '''Column Type'''
 +
|| '''Primary key'''
 +
 +
|-
 +
|| username
 +
|| VARCHAR(255) NOT NULL
 +
|| y
 +
|-
 +
|| label
 +
|| VARCHAR(255) NOT NULL
 +
|| y
 +
|-
 +
|| pub_key
 +
|| TEXT NOT NULL
 +
|| n
 +
|-
 +
|| description
 +
|| TEXT
 +
|| n
 +
|-
 +
|| import_time
 +
|| TIMESTAMP
 +
|| n
 +
|}
 +
 +
=== CLI ===
 +
 +
The client approval cli, '''oa2-cli''', can be found deployed under the /var/www/server/tools directory. OA4MP is built with self-registration support. Clients can access a special /register endpoint on the server for self-registration. However, before a client is accepted by the Master Portal, it has to be approved. The '''oa2-cli''' tool is used to approve pending self-registrations from clients. The clients in this context are the portals (Science Gateways). For more information about this cli, consult [http://grid.ncsa.illinois.edu/myproxy/oauth/server/manuals/cli.xhtml this] page.<br>
 +
The cli can also be used to create the JSON web keys, see the [https://github.com/rcauth-eu/aarc-ansible-master-portal/blob/0.2-devel/README README for the Ansible scripts] for details.
  
 
== MP Client Config ==
 
== MP Client Config ==
  
* DB tables  
+
The MP Client is a web application called '''mp-oa2-client.war''' living in the web container. It uses a set of files under '''/var/www/client''' structured in the following way:
 +
 
 +
'''/var/www/client/conf''' holds client configuration files
 +
'''/var/www/client/log''' holds the client logs produced by the web application
 +
'''/var/www/client/storage''' is only used if using file system for storage backend; when using a database this directory is unused
 +
 
 +
=== Configuration ===
 +
 
 +
The MP Client configuration file can be found at '''/var/www/client/conf/cfg.xml'''. The configuration file is XML formatted and follows the structure give by the [http://grid.ncsa.illinois.edu/myproxy/oauth/client/index.xhtml OA4MP Client]. For more information about the standard OA4MP Client configuration, consult [http://grid.ncsa.illinois.edu/myproxy/oauth/client/configuration/ this] page. Here we will describe the additional configuration options that are specific to the MP Client implementation.
 +
 
 +
          <myproxy host="credstore_host" port="credstore_port" password="credstore_pw">
 +
 
 +
              ...
 +
 +
          </myproxy>
 +
 
 +
Just like for the [[#MP_Server_Config | MP Server configuration]], the MP Client also needs a '''password''' attribute in the myproxy configuration node. The configured password will be used to STORE/PUT proxy certificates in the credential store. Note that this password should be the same as the password configured for myproxy in the [[#MP_Server_Config | MP Server configuration]], otherwise the MP Server will be unable to retrieve proxies inserted by the MP Client.
 +
 
 +
=== Database ===
 +
 
 +
Using the database backend will require you to prepare the database and its tables before using the web application. The original OA4MP Client comes with a set of defined [http://svn.code.sf.net/p/cilogon/code/tags/latest/client/oauth2-mysql.sql schemas] used by the application. The Master Portal implementation uses an [https://github.com/rcauth-eu/aarc-ansible-master-portal/blob/0.2-devel/roles/masterportal/templates/mp-oa2-client-mysql.sql.j2 updated set of schemas] which introduces two additional columns in the '''assets''' table:
 +
 
 +
{| class="wikitable" style="width: 55%;"
 +
|| '''Column Name'''
 +
|| '''Column Type'''
 +
 
 +
|-
 +
|| mp_server_request_state
 +
|| TEXT
 +
 
 +
|-
 +
|| mp_server_request_code
 +
|| TEXT
  
* oa4mp server config
+
|}
  
* oa4mp client config
+
The '''mp_server_request_code''' & '''mp_server_request_state''' are the same as the code & state parameters received by the MP Server during the current session. These two variables are there to keep session between the MP Server and Client. See more about session keeping inside the Master Portal [[Master_Portal_Internals#Session_keeping | here]].
  
 
= Logs =
 
= Logs =
 +
 +
General log messages produced by the Master Portal server and client go to their respective log files:
 +
 +
/var/www/server/log/mp-oa2-server.log.0
 +
/var/www/client/log/mp-oa2-client.log.0
 +
 +
By default logs are appended to the existing logs which are rotated when they reach the maximum logsize.
 +
See further [[#Configuration | above]] and at the [http://grid.ncsa.illinois.edu/myproxy/oauth/common/configuration/logging.xhtml logging] documentation.
 +
 +
The cli tool logs can be found at:
 +
 +
  /var/www/server/log/oa2-cli.log
 +
 +
Exceptions and stack traces thrown by the Master Portal application will end up in the logs of the web container (Tomcat):
 +
 +
# on CentOS6
 +
less /var/log/tomcat/catalina.out
 +
 +
# on CentOS7
 +
journalctl -l -u tomcat.service
 +
 +
The MyProxy Server (Credential Store) logs in syslog:
 +
 +
# on CentOS6
 +
less /var/log/messages
 +
 +
# on CentOS7
 +
journalctl -l -u myproxy-server.service

Latest revision as of 08:50, 5 September 2019

Introduction

The Master Portal is a central component in the RCauth architecture. The Master Portal caches long lived user proxies into its Credential Store, and returns short lived proxies on demand to a trusted portal (Science Gateway) for authenticated users. This page is intended for Master Portal operators and administrators who wish to configure their Master Portals.

In case you want to deploy a Master Portal, see this guide for using our Ansible deployment scripts. This page highlights the important parts of the configuration.

Credential Store Configuration

The Master Portal cannot be operated without its Credential Store. Missing the Credential Store from your setup will render your Master Portal unable to cache and retrieve any proxy certificates. Without the Credential Store a Master Portal could still be used for user authentication (via OIDC/OA4MP), but its GetProxy Endpoint will be unable to retrieve proxy certificates.

MyProxy Server

The Credential Store is just a MyProxy Server installation running in Credential Store mode. It being an unmodified MyProxy Server, you can tailor its configuration to your needs following the official MyProxy Server admin guide. In order to run it as a Credential Store backend for the Master Portal, you will need the following set of configurations:

accepted_credentials            <Master Portal Host DN>
authorized_retrievers           <Master Portal Host DN>
 
cert_dir                        /etc/grid-security/certificates
max_proxy_lifetime              264

You should not need to also set authorized_renewers, default_authorized_renewers or trusted_retrievers.

The Master Portal authenticates to the MyProxy Server via its host certificates. Make sure to configure the DN of the host certificate of the Master Portal (<Master Portal Host DN>) as the authorized party in the MyProxy Server configuration snippet above. The meaning of each of the individual authorizations can be found in the man page of myproxy-server.config.
The cert_dir has to point to the set of trust roots trusted by the MyProxy Server.
The max_proxy_lifetime determines the maximum lifetime in hours that a returned proxy credential can have. I.e. one can request shorter-lived proxies, not longer-lived. Since the Master Portal is only allowed to release short lived proxy certificates, a lifetime of 264 hours (11 days) is imposed by default (additionally, the current RCauth online CA produces EECs with a lifetime of 11 days making it meaningless to create proxies with longer lifetime).

VOMS Support

The MyProxy Server returns returns plain (i.e. non-VOMS) proxies by default, but it has support for releasing VOMS-ified proxies as well. Asking for VOMS-ified proxies is also supported in the Master Portal via the GetProxy Endpoint. In order to enable VOMS support in the MyProxy Server, set allow_voms_attribute_requests

allow_voms_attribute_requests   True

Additionally, one can add local configuration for supported VOs, by adding something like

voms_userconf                   /etc/vomses/

this prevents the need for sending 'vomses' information to the myproxy server (see GetProxy Endpoint).

Self-Authorized Retrieval

By default, the MyProxy Server disallows self renewal of certificates, but this function could be enabled. By enabling this, users possessing a valid proxy certificate could in principle renew it by directly talking to the MyProxy Server, and without the intervention of any of the web components (such as the Master Portal). Of course they can only renew up to the lifetime of the long-lived proxy certificate cached in the myproxy-server.
Note that even if self-authorization is configured, configuration will NOT allow users to renew other user's credentials, because there are per-credential policies in place that restrict the access of a single proxy certificate to its own full "DN". These policies are added to each credential during their upload by the Master Portal.
In practice such a setup is hard to make secure, as it would require storing the long-lived proxy certificates without a passphrase. Additionally it is not required for the delegation scenario involving the Master Portal, therefore it is better leave this unset. A much safer alternative is the use of ssh key authentication.

Trust Root

Deploying the trust root of the Online CA (typically RCauth.eu, see also Online CA of detailed achitecture) is an important step. Make sure to install it in the directory pointed by the cert_dir above, otherwise the MyProxy Server will not allow the MasterPortal to upload the generated user proxies. Make sure that the trust root has its signing policy, crl, and subject hash links in order.

Extras

Purge Script

The MyProxy Server does not automatically remove expired or revoked credentials. Furthermore, although the Master Portal can detect if a proxy certificate stored in the MyProxy Server is expired (it can then replace it with a new one) it can not detect whether a certificate in the chain is revoked (the myproxy server does not provide that information). We therefore developed a simple myproxy-purge.sh script which is called periodically from a cron job. It will check every proxy certificate stored in the MyProxy Server and remove the ones that have expired or have been revoked. By default this script is called every 6 hours, matching the period of fetch-crl. You can change the frequency by editing /etc/cron.d/myproxy_purge.

Master Portal Configuration

The Master Portal is made up internally of a OA4MP Server and Client, as described here. Both OA4MP Server and Client are web applications packaged in .war files. In order to deploy these you will need a web container. The setup used for this includes Tomcat as the web container (following the suggestions given by OA4MP Server), with an Apache httpd server running in front of the web container, acting as a reverse http(s) proxy. Details on the reverse proxy setup can be found here and here.

Apache

The Apache server has configuration relating to the SSL setup of the Master Portal and to the (reverse) proxy setup for Tomcat. Both are configured in /etc/httpd/conf.d/ssl.conf.
Make sure mod_proxy is configured to allow all the relevant MP Client and Server endpoints to pass through, consult this page for details.
Although it is better to explicitly proxy all the required endpoints, as a bare minimum, you can use something like the following:

ProxyPass /mp-oa2-server/authorized !
ProxyPass /mp-oa2-server ajp://127.0.0.1:8009/mp-oa2-server

ProxyPass /mp-oa2-client/forwardgetcert !
ProxyPass /mp-oa2-client/startRequest !
ProxyPass /mp-oa2-client ajp://127.0.0.1:8009/mp-oa2-client

Note! In particular the /authorized endpoint should NOT be forwarded: it is a private endpoint meant only to be called from within the Master Portal.

Also make sure to define proxy rules in order of decreasing restrictiveness, i.e. with the most restrictive rule first.

Tomcat

In the Tomcat server.xml configuration make sure to have the AJP connector opened on port 8009. There are a couple of restrictions you should enable on this connector:

  • Setting address="127.0.0.1" will enforce local access only on the container. Without this a user could bypass the Apache server (and authentication) by directly accessing port 8009 from the browser. This should not be possible due to firewall restrictions in the first place, but even if the firewall fails, this configuration will still keep out any access not coming from localhost.
  • Setting tomcatAuthentication="false" will tell Tomcat to trust the external authenticator (Apache).
  • Setting URIEncoding="UTF-8" ensures incoming GET requests will be processed as UTF-8, instead of ISO-8859-1.
<Connector port="8009" URIEncoding="UTF-8" address="127.0.0.1" tomcatAuthentication="false" protocol="AJP/1.3" redirectPort="8443"/>

Since the Master Portal is made up of two separate components, a MP Server and Client , they run as two independent web applications inside the web container. In order to be able to forward requests internally between the two (without going through the browser) the crossContext flag has to be enabled in context.xml.

<Context crossContext="true">     
...
</Context>

In order to allow the MasterPortal to send notifications upon new client registration requests, you have to also add the following resource to the context.xml file:

<Resource name="mail/Session" type="javax.mail.Session" auth="Container"></Resource>

this requires additional configuration in the config file, see email notifications for details.

MP Server Config

The MP Server is a web application called mp-oa2-server.war living in the web container. It uses a set of files under /var/www/server structured in the following way:

  • /var/www/server/conf holds server configuration files
  • /var/www/server/log holds the server logs produced by the web application
  • /var/www/server/storage is only used if using file system for storage backend; when using a database this directory is unused
  • /var/www/server/tools holds external tools, such as the client approval cli.

Configuration

The MP Server configuration file can be found at /var/www/server/conf/cfg.xml. The configuration file is XML formatted and follows the structure give by the OA4MP Server. For more information about the standard OA4MP Server configuration, consult this page. Here we will describe the additional configuration options that are specific to the MP Server implementation.

      <sshkeys max="5"
               scope="eu.rcauth.sshkeys"
      />

      ...

      <scopes>
          <scope local="true">eu.rcauth.sshkeys</scope>
      </scopes>

      ...

      <myproxy host="credstore_host" port="credstore_port" password="credstore_pw">

                ...

                <defaultLifetime>43200</defaultLifetime>

                <validators>
                       <validator handler="eu.rcauth.masterportal.server.validators.DNValidator">
                               <input name="input_claim">cert_subject_dn</input>
                       </validator>
                       <validator handler="eu.rcauth.masterportal.server.validators.LifetimeValidator">
                               <input name="max_proxy_lifetime">950400</input>
                               <input name="tolerance">86400</input>
                       </validator>
                </validators>

      </myproxy>

The sshkeys section configures the ssh key upload API, see here for configuration details.

In order to allow configuring scopes locally for the MasterPortal (such as the scope for allowing the ssh key upload API), without forwarding them to the Delegation Server, we extend the standard scopes configuration with a boolean local attribute. We need this functionality since unknown scopes are not ignored but treated as error.

The standard myproxy configuration only allows for setting the host and port, but not the password to use when talking to a MyProxy Server. The MP Server adds the password attribute to the myproxy configuration tag. The configured password will be used when GET-ing a proxy of the stored credential. Note that this password should be the same as the password configured for myproxy in the MP Client configuration, otherwise the MP Server will be unable to retrieve proxies inserted by the MP Client.

The Master Portal expects a proxy lifetime value to be provided with a /getproxy request. This parameter, however, is optional. In case no proxy lifetime value is provided in the request, the Master Portal will default to the configured value in defaultLifetime. The defaultLifetime is set in seconds and its recommended value is 12 hours.

The MP Server introduces validators. Validators are executed during a /getproxy call in order to decide whether a stored credential is still valid, or not. The validators defined in the configuration file are executed in the order in which the appear in the file. Every validator gets as input the current transaction object, request object and the response of the MyProxy INFO command on the requested credentials. A validator can either pass or fail. In case ALL validators pass, the stored credential is deemed valid, the /getproxy call resumes and a proxy is returned. In case ANY validator fails, the stored credential is treated as invalid in which case the Master Portal will renew it by asking for a new certificate from the Delegation Server. Once the certificate is renewed, the /getproxy call resumes and a proxy is returned. The MP Server comes with two validator implementations:

  • DNValidator : A user's certificate DN is constructed from the attributes released by his IdP. The way in which DNs are constructed is outlined in the RCauth.eu CP/CPS. In case the attributes released by a user's IdP change, the DN constructed for the user also changes with it. Since the Master Portal is a caching service, it caches a user's proxy certificate for its full lifetime, but it will not account for attribute/DN changes by default. Using the DNValidator will make sure that the DN of the cached certificate matches that of the DN generated from the current user attributes (from the current session). This validator takes one input:
    1. input_claim : The Master Portal does not know how to construct a DN from user attributes (that is determined by the Delegation Server), therefore the DN of the cached certificate will be compared with the DN released in the current set of claims (from the current session) by the Delegation Server. Normally this should be the cert_subject_dn claim, but it can be configured here to the name of the actual claim used for comparison.
  • LifetimeValidator : Validates the requested proxy lifetime (from the /getproxy call) against the actual proxy lifetime remaining in the cached credential. This validator will check against invalid proxy lifetime requests that exceed server maximum, and against proxy lifetime requests that are larger than the time left in the cached proxy. The maximum proxy lifetime used by this validator is constructed by max_proxy_lifetime - tolerance from the following two inputs:
    1. max_proxy_lifetime : This is a Master Portal server side maximum value in seconds imposed on the proxies being retrieved via /getproxy. Recommended value is 11 days.
    2. tolerance : This is a tolerance applied to the max_proxy_lifetime. It is a small timeframe in seconds (usually a day) which prevents the Delegation Server from being flooded with requests. Without the tolerance value, consecutive /getproxy requests that keep asking for same proxy with the maximum allowed lifetime (max_proxy_lifetime) would trigger consecutive credential renewals via the Delegation Server.

In case you are interested in writing your own validators, you will need to implement the GetProxyRequestValidator interface, and rebuild the Master Portal from source.

Apart from these additions, it is also possible to add a boolean attribute enableAutoRegisterEndpoint to the 'service' node itself. This endpoint (/autoregister, disabled by default) allows clients to register without the need for manual approval. Such functionality must NOT be exposed directly to end-users, but can be used as building block, via localhost, in the construction of an OIDCfed enabled OP. Alternatively, the v0.2.0 and up client management API is a more complex but probably more secure way of doing that.

Database

Using the database backend will require you to prepare the database and its tables before using the web application. The OA4MP Server comes with a set of defined schemas used by the application. The Master Portal implementation uses an updated set of schemas which introduces two additional columns in the transactions table:

Column Name Column Type
claims TEXT CHARACTER SET 'utf8'
mp_client_session_identifier TEXT

The claims column is used to store the set of released claims of the current transaction in JSON format. The mp_client_session_identifier column is used to store the session identifier used by the MP Client for the current transaction. See more about session keeping inside the Master Portal here.

In addition, it add a new table ssh_keys used for storing ssh public keys via the ssh key upload API:

Column Name Column Type Primary key
username VARCHAR(255) NOT NULL y
label VARCHAR(255) NOT NULL y
pub_key TEXT NOT NULL n
description TEXT n
import_time TIMESTAMP n

CLI

The client approval cli, oa2-cli, can be found deployed under the /var/www/server/tools directory. OA4MP is built with self-registration support. Clients can access a special /register endpoint on the server for self-registration. However, before a client is accepted by the Master Portal, it has to be approved. The oa2-cli tool is used to approve pending self-registrations from clients. The clients in this context are the portals (Science Gateways). For more information about this cli, consult this page.
The cli can also be used to create the JSON web keys, see the README for the Ansible scripts for details.

MP Client Config

The MP Client is a web application called mp-oa2-client.war living in the web container. It uses a set of files under /var/www/client structured in the following way:

/var/www/client/conf holds client configuration files
/var/www/client/log holds the client logs produced by the web application
/var/www/client/storage is only used if using file system for storage backend; when using a database this directory is unused

Configuration

The MP Client configuration file can be found at /var/www/client/conf/cfg.xml. The configuration file is XML formatted and follows the structure give by the OA4MP Client. For more information about the standard OA4MP Client configuration, consult this page. Here we will describe the additional configuration options that are specific to the MP Client implementation.

          <myproxy host="credstore_host" port="credstore_port" password="credstore_pw">
  
             ...

          </myproxy>

Just like for the MP Server configuration, the MP Client also needs a password attribute in the myproxy configuration node. The configured password will be used to STORE/PUT proxy certificates in the credential store. Note that this password should be the same as the password configured for myproxy in the MP Server configuration, otherwise the MP Server will be unable to retrieve proxies inserted by the MP Client.

Database

Using the database backend will require you to prepare the database and its tables before using the web application. The original OA4MP Client comes with a set of defined schemas used by the application. The Master Portal implementation uses an updated set of schemas which introduces two additional columns in the assets table:

Column Name Column Type
mp_server_request_state TEXT
mp_server_request_code TEXT

The mp_server_request_code & mp_server_request_state are the same as the code & state parameters received by the MP Server during the current session. These two variables are there to keep session between the MP Server and Client. See more about session keeping inside the Master Portal here.

Logs

General log messages produced by the Master Portal server and client go to their respective log files:

/var/www/server/log/mp-oa2-server.log.0
/var/www/client/log/mp-oa2-client.log.0

By default logs are appended to the existing logs which are rotated when they reach the maximum logsize. See further above and at the logging documentation.

The cli tool logs can be found at:

 /var/www/server/log/oa2-cli.log

Exceptions and stack traces thrown by the Master Portal application will end up in the logs of the web container (Tomcat):

# on CentOS6
less /var/log/tomcat/catalina.out

# on CentOS7
journalctl -l -u tomcat.service

The MyProxy Server (Credential Store) logs in syslog:

# on CentOS6
less /var/log/messages

# on CentOS7
journalctl -l -u myproxy-server.service