View source for Argus Global Banning Setup Overview

= Introduction =
This page is a *DRAFT*. Furthermore, this page is not intended to give a full overview of how to install an Argus service, see instead [https://twiki.cern.ch/twiki/bin/view/EGEE/ArgusEMIDeployment https://twiki.cern.ch/twiki/bin/view/EGEE/ArgusEMIDeployment] for information on general setup.

= Architecture / Components =
; [[#Central wLCG Argus|Central wLCG instance]] : for the purpose of this wiki an Argus PAP is sufficient (no PDP or PEPd)
; [[#NGI Argus|NGI Argus instance]] : unless all its sites run an Argus, a full Argus (with PDP and PEPd) is needed.
; [[#Site|Site services]] : Site either runs its own Argus, or defines local mapping rules combined with NGI-based banning.

= Setup =
== Setting up a basic site or NGI Argus ==
The general installation and configuration of an Argus is outside of the scope of this page. For further information see
* [http://www.eu-emi.eu/products/-/asset_publisher/1gkD/content/argus-2 http://www.eu-emi.eu/products/-/asset_publisher/1gkD/content/argus-2]
* [https://twiki.cern.ch/twiki/bin/view/EGEE/ArgusEMIDeployment https://twiki.cern.ch/twiki/bin/view/EGEE/ArgusEMIDeployment]
* [https://twiki.cern.ch/twiki/bin/view/EGEE/AuthorizationFramework https://twiki.cern.ch/twiki/bin/view/EGEE/AuthorizationFramework]

== Configuring a PAP ==
An Argus PAP can be configured using either the <tt>pap-admin</tt> cli tool, see [https://twiki.cern.ch/twiki/bin/view/EGEE/AuthZPAPCLI https://twiki.cern.ch/twiki/bin/view/EGEE/AuthZPAPCLI] or directly via the ini files <tt>pap_authorization.ini</tt> (for the ACL) and <tt>pap_configuration.ini</tt> (for setting remote PAPs), both located in <tt>etc/argus/pap</tt>.

For setting the policies, the <tt>pap-admin</tt> is the only option.

'''Note''': When using YAIM to configure Argus, it will be left without any policy.

== Central Banning Argus ==
* The Central Banning Argus needs to be configured to have a policy which *only* bans (/deny entries) subjects and/or FQANs. 
* In addition it needs to have an appropriate ACL which permits all the NGI Argus PAPs read access to its policy.

=== Configuring the ACL in the central banning PAP ===
The central banning Argus PAP needs to have a access control entry for each of the NGI Argus instances. Each of these must have read access to the "local" policy in the central Argus PAP. 

Adding a new ACE for argus-ngi with the LDAP notation for its DN:
 pap-admin add-ace 'CN=argus-ngi.example.org, OU=example, O=hosts, O=Example CA' POLICY_READ_LOCAL
(replace the DN with the DN of the NGI Argus host).

The DN will be internally translated into OpenSSL format.

Check the new list of ACEs, note that the output uses the OpenSSL notation, which cannot be used as input.
 pap-admin list-acl

Alternatively a line can be added to the <tt>[dn]</tt> section in the <tt>/etc/argus/pap/pap_authorization.ini</tt> file, e.g.:
 [dn]
 "/O=Example CA/O=hosts/OU=example/CN=argus-ngi.example.org" : POLICY_READ_LOCAL

When updating the <tt>pap_authorization.ini</tt> file directly, the service needs to be restarted. The CLI tool on the other hand, does not require a reboot and will update the ini file.

=== Setting up the policy in the PAP ===
The central banning policy will be very simple. Normally, it should be valid for any resource and action ID (see [https://twiki.cern.ch/twiki/bin/view/EGEE/AuthZIntro#Identifiers_within_Argus https://twiki.cern.ch/twiki/bin/view/EGEE/AuthZIntro#Identifiers_within_Argus] for background information), i.e. for ".*".
Banning a user:
 pap-admin ban subject "CN=John Doe, O=nikhef, O=users, O=dutchgrid"
Which will create a policy in the 
 default (local):
section containing:
 resource ".*" {
 
     action ".*" {
         rule deny { subject="CN=John Doe, O=nikhef, O=users, O=dutchgrid" }
     }
 }

For banning a large list of subjects (or FQANs) it is probably useful to create a file containing a policy such as above with many deny lines, and use the command
 pap-admin add-policies-from-file <ban-policy-file>
where ban-policy-file contains the banning policy. add-policies-from-file can be abbreviated to apf.

Unbanning can be done using
 pap-admin unban subject "CN=John Doe, O=nikhef, O=users, O=dutchgrid"

The active policy can be viewed with
 pap-admin list-policies
where list-policies can be abbreviated to lp. Note that policies are parsed in the order printed, until a match is found.

== NGI Argus ==
=== Adding the central banning Argus PAP to the list of PAPs ===
The NGI Argus PAP needs to use the central banning Argus as input. It can be added using the CLI by specifying a alias, its hostname and DN. In this case the DN *must* be in OpenSSL notation. Also note that it needs to be added 'public' since the sites need to be able to read the central Argus' policy:
 pap-admin add-pap --public centralbanning lcg-argus.cern.ch "/DC=ch/DC=cern/OU=computers/CN=argus.cern.ch"
After this, the new PAP needs to be enabled:
 pap-admin enable-pap centralbanning
And the central banning PAP needs to be ordered first:
 pap-admin set-paps-order centralbanning default

Alternatively the configuration can be added to the <tt>/etc/argus/pap/pap_configuration.ini</tt> file:
 [paps]
 centralbanning.type = remote
 centralbanning.enabled = true
 centralbanning.dn = /DC=ch/DC=cern/OU=computers/CN=argus.cern.ch
 centralbanning.hostname = lcg-argus.cern.ch
 centralbanning.port = 8150
 centralbanning.path = /pap/services/
 centralbanning.protocol = https
 centralbanning.public = true

 [paps:properties]
 poll_interval = 3600 # polling time in seconds.
 ordering = centralbanning
 ordering = default

followed by a restart of the PAP service.
A forced retrieval of the remote policies can be triggered using
 pap-admin refresh-cache

'''TODO''': figure out the best polling interval. Defaults is 14400 (i.e. 4 hours).

=== Configuring the ACL in the NGI PAP ===
The NGI Argus PAP needs to be setup with ACEs in the same way as the Central Argus, except it needs to allow read access (also) for remote policies.
In case the NGI would also like to keep a local list of banned users, the ACE should be setup as in
 pap-admin add-ace 'CN=argus-site.example.org, OU=example, O=hosts, O=Example CA' 'POLICY_READ_LOCAL|POLICY_READ_REMOTE'

Alternatively a line can be added to the <tt>[dn]</tt> section in the <tt>/etc/argus/pap/pap_authorization.ini</tt> file, e.g.:
 [dn]
 "/O=Example CA/O=hosts/OU=example/CN=argus-site.example.org" : POLICY_READ_LOCAL|POLICY_READ_REMOTE
Followed by a restart of the PAP service.

=== Policies in the NGI Argus ===

The NGI Argus can add local banning rules just as the Central Banning Argus, and these will be also taken into account by the site Argus instances. Also in this case it is advisable not to use a specific resource ID but just use the catch-all ".*".

The banning Argus PAPs, either central or NGI, should **not** put obligations in their policies, unless they really know what they are doing. Especially the "http://glite.org/xacml/obligation/local-environment-map" mapping oligation will lead to problems. In addition, the NGI Argus should have no obligationHandlers configured in the corresponding PEPd configuration file <tt>/etc/argus/pepd/pepd.ini</tt> as a further protection.

In case the NGI Argus is also used to provide the actual mappings for the sites (this is a difficult and not-advisable setup) this could be done in non-generic resource ID entries, where the banning is done in the catch-all ".*" resource ID section.

'''TODO''': Need to check that the less-generic policy is still used when the more-generic has no match.

=== NGI PDP configuration ===

When a site will not run its own Argus service, it will most likely want to query the NGI Argus via the PEPd frontend and a complete PAP+PDP+PEPd Argus will be setup by the NGI.
The NGI PDP will query the NGI PAP from time to time, and cache the policy. This retentionInterval time defaults to 240 minutes (i.e. 4 hours) and can be set in <tt>/etc/argus/pdp/pdp.ini</tt> in the <tt>[POLICY]</tt> section. After an update, the NGI PDP needs to be restarted.

A update can be triggered by hand using
 /etc/init.d/argus-pdp reloadpolicy

A reasonable time needs to be set, the default 4 hours is much too long.

== Site ==

=== Site runs own Argus ===
When the site runs its own Argus instance, it should be sufficient to add the NGI Argus as a remote PAP and configure it to be the leading PAP, following the same commands as above:
 pap-admin add-pap ngi argus-ngi.example.org "CN=argus-ngi.example.org,OU=example,O=hosts,O=VL-e P4"
 pap-admin enable-pap ngi
 pap-admin set-paps-order ngi default

Additionally, the polling interval needs to be set to a reasonable value, '''to be decided''':
 pap-admin set-polling-interval 3600
And also the retentionInterval for the site PDP, in the pdp.ini file, as described above.

Note that the only reason the site would not directly use the central banning PAP is the difficulty in maintaining the ACL in the central banning PAP.

After adding the NGI PAP, the resulting combined policy can be viewed using
 pap-admin lp --all
The additional <tt>--all</tt> is necessary to include the remote policies.

=== Site runs no Argus ===

There are roughly two options available:

# The site downloads e.g. via cron, the banlist from NGI (Argus). An [[Media:Policy_to_banmapfile_v2.sh|example script (note: new version)]] is provided. It uses this with, for example, the LCMAPS ''lcmaps_ban_dn'' plugin (part of lcmaps-plugins-basic-1.5.1 or higher, EMI-3/UMD-3), the ''lcmaps_ban_fqan'' (part of lcmaps-plugins-voms 1.5.4 or higher, EMI-3/UMD-3) or the LCAS ''lcas_userban'' plugin. See [[#Using the lcmaps_ban_dn and/or lcmaps_ban_fqan plugins|below]] for examples.
# The site runs first the LCMAPS C-PEP plugin which call out to the NGI Argus and then uses other LCMAPS plugins (e.g. SCAS or local mappings) for obtaining the user mapping. There are some complications with this, see [[#Using lcmaps-plugins-c-pep, problems and solutions|below]]. Also note that the SCAS itself cannot use the LCMAPS C-PEP plugin since the SCAS only receives the separate attributes, not the entire certificate; the [https://edms.cern.ch/document/929867 AuthZ interoperability profile] defines the optional attribute http://authz-interop.org/xacml/subject/cert-chain, but this is currently not implemented in the SCAS or LCMAPS SCAS-client plugin.

==== Using the ''lcmaps_ban_dn'' and/or ''lcmaps_ban_fqan'' plugins ====

In the LCMAPS DB file, one would typically use something like:
 [..]
 
 ban_dn = "lcmaps_ban_dn.mod"
          "-banmapfile /etc/grid-security/ban_users.db"
          "-disablewildcard" # This disables the use of wildcards
 
 ban_fqan = "lcmaps_ban_fqan.mod"
          "-banmapfile /etc/grid-security/ban_users.db"
 
 [..]
 
 default:
 verify_proxy -> ban_dn
 ban_dn -> ban_fqan
 ban_fqan -> ... # fill in the normal site-local policy

This policy would then be used either on e.g. individual workernodes, a central SCAS service, etc.<br>
'''NOTE''': the current UMD-3 release of the ban_fqan plugin also fails in case there are ''no'' VOMS credentials in the proxy (see https://bugzilla.nikhef.nl/show_bug.cgi?id=17), an update has been released into EMI-3, version lcmaps-plugins-voms-1.6.4-1, and will move to the UMD-3. It can also be found in the Nikhef repository https://software.nikhef.nl/dist/mwsec/rpm/

==== Using lcmaps-plugins-c-pep, problems and solutions ====

The first complication comes from the use of an Argus ''only'' for banning. In such case the result of the combined effective Argus policy will be either ''Deny'' (user is on banlist) or ''Not Applicable'' (user is not on banlist). On the other hand, Argus is designed to typically result in either a ''Deny'' or ''Permit''. The results ''Not Applicable'' (no match) and ''Indeterminate'' (parse error) are treated as failure, see https://twiki.cern.ch/twiki/bin/view/EGEE/AuthZIntro#Not_Applicable_and_Indeterminate. The second problem with such a scenario would be the high load on the NGI Argus: all the WNs of all the non-Argus sites would call the single NGI Argus. It seems this cannot be properly solved.

===== Solution 1: special policy and change in attribute-mappings.ini =====
A solution could be to define a policy that ends with an explicit permit line catching everyone not on the banlist. This is non-trivial:
<ol>
<li> After the first match the rest of the policy is not being parsed, so this cannot be naively done on the NGI Argus if the same policy is being imported by site Argus instances. A possible workaround would be to put the "permit" line under a special "resource" definition, while the banlist is under the catch-all resource ".*". The LCMAPS C-PEP plugin would then specify the special resource, while sites with their own Argus would use their own resource identifier.
<li> By default, it is impossible to define a 'catch-all' policy: Argus normally does an exact match for subject values. However, this can be adjusted in the <tt>/etc/argus/pap/attribute-mappings.ini</tt> via the following change, but unfortunately, we are still bitten by the bug described below in the Warning.
<pre>
 --- /etc/argus/pap/attribute-mappings.ini.orig  2013-02-13 15:15:36.000000000 +0100
 +++ /etc/argus/pap/attribute-mappings.ini       2013-09-03 14:48:52.000000000 +0200
 @@ -5,7 +5,8 @@
  subject.xacml-id = urn:oasis:names:tc:xacml:1.0:subject:subject-id
  subject.xacml-datatype = urn:oasis:names:tc:xacml:1.0:data-type:x500Name
  subject.xacml-target-element = subject
 -subject.xacml-match-function = urn:oasis:names:tc:xacml:1.0:function:x500Name-equal
 +subject.xacml-match-function = urn:oasis:names:tc:xacml:2.0:function:x500Name-regexp-match
 +subject.xacml-match-function-datatype = http://www.w3.org/2001/XMLSchema#string
 
  id = subject-issuer
  subject-issuer.xacml-id = http://glite.org/xacml/attribute/subject-issuer
</pre>
With many thanks to Valery Tschopp and Andrea Ceccanti for help, see https://groups.google.com/forum/#!topic/argus-support/jLgcttOBMtg

This would need to be changed ''only'' on the PAP where the catch-all subject is locally defined, typically the NGI Argus.

The resulting policy on the NGI Argus would then become something like:
 # First optionally add NGI banned users
 resource ".*" {
     action ".*" {
         rule deny { subject="CN=John Doe" }
         rule deny { subject="CN=Jane Doe" }
     }
 }
 
 # Add a special resource for non-Argus sites
 resource "banlist_only" {
     action ".*" {
         rule permit { subject=".*" }
     }
 }
while the typical definition in the lcmaps.db file would be something like:
  pepc        = "lcmaps_c_pep.mod"
              "--pep-daemon-endpoint-url https://argus-ngi.example.org:8154/authz"
              '''"--resourceid banlist_only"'''
              "--actionid anything"
              "--capath /etc/grid-security/certificates"
              "--pep-certificate-mode implicit"
              "--use-pilot-proxy-as-cafile" # Add this on RHEL 6 based systems
<li>'''''WARNING''''' unfortunately it turns out that this solution currently has issues due to a bug in the XACML parsing library used by Argus as described on the same argus-support mailthread linked above: when a complete DN is matched using the above rules, it only matches when extra spaces are added after the , separator between the fields, e.g. one would need to specify in the Argus policy <pre>CN=Mischa Salle, O=nikhef, O=users, O=dutchgrid</pre> instead of <pre>CN=Mischa Salle,O=nikhef,O=users,O=dutchgrid</pre>
</ol>

===== Solution 2: new lcmaps-plugins-c-pep-1.2.4 or higher =====
Since having such a catch-all policy and having regular expression matching in the subject are more prone to mistakes and still has issues, as an alternative a new option has been introduced into the lcmaps-plugins-c-pep-1.2.4. When specified, it will interpret a ''Not Applicable'' as success. The option can be called either <tt>--banning-only-mode</tt> or <tt>--treat-notapplicable-as-success</tt>.

The plugin '''has''' been released as update for EMI-3 (version 1.2.6) and will move to the UMD-3. It can also be found in the Nikhef repository,
 http://software.nikhef.nl/dist/mwsec/rpm/
available for RHEL5 and 6, Fedora 18, 19 and 20.

The typical definition in the lcmaps.db file would be something like:
  pepc        = "lcmaps_c_pep.mod"
              "--pep-daemon-endpoint-url https://argus-ngi.example.org:8154/authz"
              "--resourceid anything"
              "--actionid something"
              "--capath /etc/grid-security/certificates"
              "--pep-certificate-mode implicit"
              "--use-pilot-proxy-as-cafile" # Add this on RHEL 6 based systems
              '''"--banning-only-mode" # use *ONLY* when Argus is used for banning only'''