Difference between revisions of "Creating Pool Accounts With LDAP"

From PDP/Grid Wiki
Jump to navigationJump to search
m
m
Line 28: Line 28:
 
== Creating accounts for a new VO ==
 
== Creating accounts for a new VO ==
  
−
To use the scripts, login on the fileserver <tt>hooimijt.nikhef.nl</tt>, and make sure that <tt>/export/perm/adm/bin</tt> is in your path (it contains all the relevant scripts), or go there. Also, make sure you know your LDAP manager password.
+
The <tt>ndpfpooladd</tt> script is part of the ndpfuseradd package from version 1.1 and higher.
 +
You can run it from any host where this package is installed, provided that
  
−
To create accounts for a new VO, or for a new group of an existing VO, you need to:
+
* you can connect securely to the ldap server (currently teugel.nikhef.nl port 636)
−
<ol>
+
* you have root access on the NFS server and gridmapdir server via ssk keys, and you access
−
<li>add the accounts to the LDAP directory</li>
+
  these through an ssh agent
−
<li>create the homedirectories for these users on hooimijt</li>
 
−
<li>add the inodes to the gridmapdir</li>
 
−
</ol>
 
−
(and of course add the VO itself to the proper Quattor profiles for the selected facilities, but this is outside the scope of this page).
 
  
−
'''Note''': do not forget to create inodes in the gridmapdir on the DPM and the LFC hosts.
+
The documentation is not complete but the help is usable. Anyway, it's virtually all
 +
automatic now :-)
 +
The pool accounts created have an immutable .ssh directory, so that pool users cannot
 +
add their own ssh keys and come back later!
  
−
=== Generating the LDIF ===
+
The ndpfpooladd utility creates new pools of accounts for VOs in the NDPF. It
 +
handles all of the LDAP, NFS and gridmapdir interactions from a single host,
 +
provided you have your SSH public keys loaded in an agent (using passwords
 +
is too error-prone and enticing).
  
−
Adding users to the directory needs two commands (or one pipe). The <tt>gen_poolacc_ldif</tt> script generates <i>ldif</i> files, that need to be piped in to "ldapadd" to be inserted in the directory.
+
Usage: ndpfpooladd [--help] [-c|--config configfile] [-v[v[v]]] [-H ldapuri]
 +
  [--base ldapbaseDIT] [--automountbase DIT] [--ldapuidbase DIT]
 +
  [--homedirhost FQDN] [--homedirdirname dir]
 +
  [--gridmapdirhost FQDN] [--gridmapdirdirname dir]
 +
  [--(no-)updateldap] [--(no-)updateNFS] [--(no-)updateGMD]
 +
  [-u|--updatecn RDN]
  
−
Its use is best explained by an example:
+
  [-g unixgid] [--vo|--poolname poolname] [-b|--baseuid uidNumber]
 +
  [-l|--length numerals] [-n|--naccounts n] [-s|--startserial serial]
  
−
  ./gen_poolacc_ldif -c --vo atlas -g 2002 -b 43000 -n 200
+
  Examples:
 +
  Create 100 accounts for the new "atltst" VO, from uid 90100:
 +
  (frst create a new Unix group, e.g. atlastst with gidNumber e.g. 2099)
  
−
will spit out lots of ldif, like
+
    ndpfpooladd -u "David Groep" -g atlastst -b 90100 \
 +
            -l 2 --vo atltst -n 100
  
−
dn: uid=atlas000, ou=PoolAccounts, dc=farmnet,dc=nikhef,dc=nl
+
  To add 50 extra accounts to this pool, from a fresh uidNumber range:
−
objectclass: top
 
−
objectclass: posixAccount
 
−
cn: PoolAccount 0 of atlas
 
−
uid: atlas000
 
−
uidNumber: 43000
 
−
gidNumber: 2002
 
−
homeDirectory: /home/atlas000
 
−
...
 
−
 
 
−
The <tt>-c</tt> option (for "create") additionally generates the organizationalUnit definitions inside the poolaccounts tree under which the new <tt>uid</tt>s will be created. If you are extending an existing set of poolaccounts, this options must be omitted. But, if you leave it in by accident, the <tt>ldapadd</tt> command below will trigger an error and terminate without damage being done (and, conversely, omitting -c on initial creation will also trigger an error).
 
−
 
 
−
The LDIF output must be added to the directory with the <tt>ldapadd</tt> command:
 
−
 
 
−
ldapadd -H ldaps://teugel.nikhef.nl/ -W -x -D "cn=<i>My Name</i>,ou=managers,dc=farmnet,dc=nikhef,dc=nl"
 
−
 
 
−
Valid managers are "David Groep", "Jeff Templon@nikhef.nl", "Ronald Starink", "Sven Gabriel", "Tristan Suerink" and "backup" (the last one can only read, though). The two commands can be combined in a single pipe.
 
−
 
 
−
In due course, the new accounts will appear on the farm, and you can check their presence with the "id" and "ls" commands:
 
−
 
 
−
id -a atlas000
 
−
ls -l /home/atlas000/
 
−
 
 
−
There may be a slight delay if the system you are trying this on is running <tt>nscd</tt>, or is looking at a slave LDAP server (stalkaars-0[13].farm.nikhef.nl, hooimijt, vlaai instead of teugel).
 
−
 
 
−
The number of digits appended to the vo name is three (3) by default, but can be changed with the <tt>-l</tt> option. And, of course, the "voname" specified here is completely unrelated to the VO name in the information system or used in the GlueAccessControlBaseRules.
 
−
 
 
−
==== Extending an existing poolaccount range ====
 
−
 
 
−
You can also extend an existing range, by specifying a "start" value to <tt>gen_poolacc_ldif</tt>, but remember: the "base" value <b>must remain the same</b>. So, to generate an additional 100 atlas accounts, the command would be
 
−
 
 
−
./gen_poolacc_ldif --vo atlas -g 2002 -b 43000 -n 300 -s 200
 
−
 
 
−
to start at 200 and ensure that there are 300 accounts in total.
 
−
 
 
−
== Generating the home directories ==
 
−
 
 
−
Once the accounts have been added to the directory, you can create the home directories on <tt>schuur.nikhef.nl</tt>. This must be done as root, and on schuur itself.
 
−
The command to use is <tt>make_poolacc_dir</tt> (present under <tt>/root/bin</tt>), which takes one argument: the
 
−
uid prefix to select on. By default, it will generate a shell script that tries to (re)create the home directories for <i>all</i> poolaccounts, so beware.
 
−
 
 
−
'''Note''': at the moment, the script <tt>make_poolacc_dir</tt> is found at schuur in <tt>/root/bin</tt>.
 
  
−
To generate the home directories for the "phicos" VO, use:
+
    ndpfpooladd -u "David Groep" -g atlastst -b 91300 \
 +
            -l 2 --vo atltst -n 50 -s 100
  
−
./make_poolacc_dir --uid=phicos > /tmp/schapen
+
  note that the uidNumber ranges need not be contiguous.
−
sh /tmp/schapen
 
  
−
To so the same for the additional 100 atlas accounts created from "atlas200" to
+
The rest of the commandline options are set to reasonable defaults in
−
"atlas299", use:
+
the source, but can be overridden in a configuration file specified
 +
with the "-c" option, or in $HOME/.ndpfpooladdrc
  
−
  ./make_poolacc_dir --uid=atlas2 > /tmp/lam
+
  Notes:
−
  sh /tmp/lam
+
- some basic checks on uidNumber availability are performed: the first
 +
  and last uid of a range must not be in use
 +
- if one part fails (e.g. NFS), start by disabling the previous steps
 +
  with the --no-updateXXX options
 +
  So, if LDAP succeeded but creating the directories failed, retry with
 +
  "--no-updateldap"
 +
- new Unix groups MUST be manually created in the LDAP directory
 +
  - If you get bored with the "-u" option, add a .ndpfpooladdrc file in
 +
  your $HOME with the content:
 +
    $updatecn="David Groep"
  
−
The current version of <tt>make_poolacc_dir</tt> ensures that the .ssh directory
 
−
contains an empty "authorized_keys" file and is immutable (<tt>"chattr =i .ssh"</tt>).
 
−
 
−
== Creating the inodes ==
 
−
 
−
Creating the inodes is done with <tt>populate_gridmapdir</tt>. Run this script as root on <tt>vlaai.nikhef.nl</tt>. The <tt>populate_gridmapdir</tt> script is even more trivial than the other two: it extracts all uid's from the <tt>ou=Poolaccounts,dc=farmnet,dc=nikhef,dc=nl</tt> tree and prepends it with "/export/perm/share/gridmapdir":
 
−
 
−
./populate_gridmapdir
 
−
 
−
results in
 
−
 
−
/export/perm/share/gridmapdir/alice000
 
−
/export/perm/share/gridmapdir/alice001
 
−
/export/perm/share/gridmapdir/alice002
 
−
/export/perm/share/gridmapdir/alice003
 
−
/export/perm/share/gridmapdir/alice004
 
−
...
 
−
 
−
To filter out the new ones, use grep, and pipe the results through xargs so as to touch the files:
 
−
 
−
./populate_gridmapdir | grep atlas2 | xargs touch
 
−
 
−
will do the job for the 100 additional atlas accounts, for instance.
 
  
 
== Repairing an empty gridmapdir ==
 
== Repairing an empty gridmapdir ==
Line 146: Line 104:
  
 
and watch the results.
 
and watch the results.
 +
 +
This utility is part of the manage-gridmap package!
  
 
== Migrating the poolaccounts in the LDAP directory ==
 
== Migrating the poolaccounts in the LDAP directory ==

Revision as of 19:01, 4 January 2009

IMPORTANT: THIS INFORMATION IS UPDATED BY THE NDPFPOOLADD SCRIPT NOW!

The LDAP directory structure

The list of valid users of the NDPF is kept in a central LDAP directory, currently hosted on teugel.nikhef.nl. This directory contains both the "local" users as well as all poolaccounts and all automount map entries. The structure of the directory is: 

+ dc=farmnet,dc=nikhef,dc=nl
  |
  + ou=Managers
  + ou=LocalGroups (contains all groups!)
  + ou=LocalUsers
  + ou=Poolaccounts
    |
    + ou=dteam
    + ou=ops
    ...
  + ou=automount
    |
    + ou=auto.home
    + ou=lcgprod
      |
      + ou=auto.sedata
      + ou=auto.share
      + ou=auto.stage
      + ou=auto.sedata2

The ou=Poolaccounts entry contains the hierarchy of pool account groups. Per pool account group, there is a separate ou, which contains the actual list of pool accounts. Each account is named by its uid, and is of objectClass "posixAccount". For each account named here, there should be a corresponding entry in the ou=pool,ou=auto.home,ou=automount branch of the tree as well (of objectClass "automount").

Creating accounts for a new VO

The ndpfpooladd script is part of the ndpfuseradd package from version 1.1 and higher. You can run it from any host where this package is installed, provided that

  • you can connect securely to the ldap server (currently teugel.nikhef.nl port 636)
  • you have root access on the NFS server and gridmapdir server via ssk keys, and you access
 these through an ssh agent

The documentation is not complete but the help is usable. Anyway, it's virtually all automatic now :-) The pool accounts created have an immutable .ssh directory, so that pool users cannot add their own ssh keys and come back later! 

The ndpfpooladd utility creates new pools of accounts for VOs in the NDPF. It
handles all of the LDAP, NFS and gridmapdir interactions from a single host,
provided you have your SSH public keys loaded in an agent (using passwords
is too error-prone and enticing).

Usage: ndpfpooladd [--help] [-c|--config configfile] [-v[v[v]]] [-H ldapuri]
  [--base ldapbaseDIT] [--automountbase DIT] [--ldapuidbase DIT]
  [--homedirhost FQDN] [--homedirdirname dir]
  [--gridmapdirhost FQDN] [--gridmapdirdirname dir]
  [--(no-)updateldap] [--(no-)updateNFS] [--(no-)updateGMD]
  [-u|--updatecn RDN]

  [-g unixgid] [--vo|--poolname poolname] [-b|--baseuid uidNumber]
  [-l|--length numerals] [-n|--naccounts n] [-s|--startserial serial]

Examples:
  Create 100 accounts for the new "atltst" VO, from uid 90100:
  (frst create a new Unix group, e.g. atlastst with gidNumber e.g. 2099)

    ndpfpooladd -u "David Groep" -g atlastst -b 90100 \
            -l 2 --vo atltst -n 100

  To add 50 extra accounts to this pool, from a fresh uidNumber range:

    ndpfpooladd -u "David Groep" -g atlastst -b 91300 \
            -l 2 --vo atltst -n 50 -s 100

  note that the uidNumber ranges need not be contiguous.

The rest of the commandline options are set to reasonable defaults in
the source, but can be overridden in a configuration file specified
with the "-c" option, or in $HOME/.ndpfpooladdrc

Notes:
- some basic checks on uidNumber availability are performed: the first
  and last uid of a range must not be in use
- if one part fails (e.g. NFS), start by disabling the previous steps
  with the --no-updateXXX options
  So, if LDAP succeeded but creating the directories failed, retry with
  "--no-updateldap"
- new Unix groups MUST be manually created in the LDAP directory
- If you get bored with the "-u" option, add a .ndpfpooladdrc file in
  your $HOME with the content:
    $updatecn="David Groep"


Repairing an empty gridmapdir

For this you need the backup file that's generated nightly by the poolmaplog script from cron. The file format is simple: 

uid   subjectDN_in_lowercase
...

btu for use in the gridmapdir the special chars (so painstackingly converted to readable format by poolmaplog) must be concerted back. This is the task of the repair-pool script. As far as I know, these are the special characters: 

% / <space> = ( ) - . @

the repair-pool script will translate these to URL-escaped characters (ie. "=" becomes "%3D" -- note that we must thus convert any %-signs first!)

The script will automatically relink the poolaccounts to the proper DN for those accounts that were in use (i.e. has a DN assigned to them). You should only attempt repair if the pooldir is empty! 

./repair-pool < /export/perm/share/gridmapdir/.poolmap.20050816

and watch the results.

This utility is part of the manage-gridmap package!

Migrating the poolaccounts in the LDAP directory

When the poolaccounts are migrated to a new system (schuur.nikhef.nl), the LDAP directory needs to be updated to reflect the new location. To this end, a new script is now available on hooimijt:/export/perm/adm/bin: 

./migrate_poolacc_dir_ldap [--uid=uidpattern]

which generates LDIF output with the directory updates. You can apply the LDIF modification with the command 

ldapmodify -x -W -D "cn=Your Name,ou=Managers,dc=farmnet,dc=nikhef,dc=nl" -H ldap://trog.nikhef.nl/ -f ldif_file

The default is to expect the poolaccount home to be on 

schuur.nikhef.nl:/project/share/pool/prefix/loginname

It will look for appropriately named poolaccounts under ou=pool,ou=auto.home,ou=automount,dc=farmnet,dc=nikhef,dc=nl

PS: also the gen_poolacc_dir script has been changed to use the new poolaccount homedirectory format. The output of this gen_poolacc_dir scfipt is a shell script to be executed on the physical filesystem-hosting node.

Retrieved from "https://wiki.nikhef.nl/grid/index.php?title=Creating_Pool_Accounts_With_LDAP&oldid=2151"