Access control for dCache storage elements

This page is part of an investigation on How to control access rights for LFC/SRM files .

Finding out which storage systems are available

To list the storage systems to which you have access use

 lcg-infosites --vo <YOUR-VO> se

which results in

Avail Space(Kb) Used Space(Kb)  Type   SEs
----------------------------------------------------------
12078           108             n.a    srm.grid.rug.nl
12078           108             n.a    srm.grid.rug.nl
730582644       681194097       n.a    gb-se-amc.amc.nl
8226695519985   23304480014     n.a    srm.grid.sara.nl
605355546       806421195       n.a    gb-se-nki.els.sara.nl
6575746866      20920246        n.a    carme.htc.biggrid.nl
152913518       115521938       n.a    se.grid.rug.nl
248345185       1166074827      n.a    gb-se-ams.els.sara.nl
355230761       1056545980      n.a    gb-se-uu.science.uu.nl
1266740857      145035883       n.a    gb-se-wur.els.sara.nl
337812899       1076607113      n.a    gb-se-kun.els.sara.nl
2195706454      3048365         n.a    tbn18.nikhef.nl
771834491       620488567       n.a    gb-se-lumc.lumc.nl

Note #1 the current version of the lcg-infosites command does not use your grid proxy at all !

Note #2 before using the srm* commands make sure that the SRM client is at least 2.1.0 as otherwise most, if not all, srm* commands will fail:

$ srmping -version
Storage Resource Manager (SRM) Client version 2.1.0
Copyright (c) 2002-2008 Fermi National Accelerator Laborator

Note #3 The glite 3.2 UI up to version 3.2.4 contains an older (actually, too old !) version of the srm-client than the glite 3.1 UI !

You can use the srmping command to get some basic information about a storage system

$ srmping -2  srm://srm.grid.sara.nl
VersionInfo : v2.2
backend_type:dCache
backend_version:production-1.9.3-3

So host srm.grid.sara.nl is a SRMv2.2 compliant storage system based on dCache.

Finding out how storage is organized

You can use the srmls command to figure out how the storage is organized:

$ srmls srm://srm.grid.sara.nl
 512 //
     [SNIP]
     4096 //pnfs/
     [SNIP]

This listing tells us that this storage system uses a PNFS file system, which means that we're talking to a dCache SRM with underlying PNFS file system.

If this command returned an error:

$ srmls srm://se.grid.rug.nl
srm client error:
java.lang.Exception: Return status:
 - Status code:  SRM_FAILURE
 - Explanation:  All ls requests failed in some way or another

then (my guess is that) we're talking to a dCache SRM with underlying Chimera file system. We will blindy try to list the /pnfs directory in the next example:

Subsequent srmls commands then give us:

$ srmls srm://srm.grid.sara.nl/pnfs/
 512 /pnfs//
     512 /pnfs//grid.sara.nl/
     [SNIP]

and for the Chimera-based dCache:

$  srmls srm://se.grid.rug.nl/pnfs/
 512 /pnfs/
     512 /pnfs/grid.rug.nl/

Heeey! So even though we're talking to a Chimera-based dCache the starting directory is still /pnfs.

$ srmls srm://srm.grid.sara.nl/pnfs/grid.sara.nl
 512 /pnfs/grid.sara.nl/
     512 /pnfs/grid.sara.nl/disk/
     512 /pnfs/grid.sara.nl/tapetests/
     512 /pnfs/grid.sara.nl/data/
     512 /pnfs/grid.sara.nl/disktests/

$ srmls srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data
 0 /pnfs/grid.sara.nl/data/
     [SNIP}
     512 /pnfs/grid.sara.nl/data/pvier/
     512 /pnfs/grid.sara.nl/data/vlemed/

Hey, we are at the VO level now. Here I've listed the two VOs which will be used throughout this page.

Creating your own directory in SRM-space

Before we copy a file to the dCache SRM we first create our own directory. If we do not do this then SRM will store the files in generated directories, over which we have little or no control.

srmmkdir srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/

The URL for this directory will be used throughout the rest of this page, hence we abbreviate it to

SRM=srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust

Copying and registering your file

Next we will copy a file to our SRM directory and register it in the LFC in one go:

$ lcg-cr -d $SRM/myfile -l lfn:/grid/pvier/janjust/my-dcache-file file://$PWD/myfile
guid:6b4c060f-cd95-4360-9e1b-a29d023d49b8

which returns the LFC GUID for the file upon success.

(For details on how to find out how the LFC directory space is organized see Access control for the LFC)

We could also have copied the file to the SRM only, bypassing the LFC registration, using the command

lcg-cp file://$PWD/myfile $SRM/myfile

or even

srmcp -globus_tcp_port_range=20000,25000 file://$PWD/myfile $SRM/myfile

but as we needed the LFC entry as well we used the (preferred) lcg-cr command. Please also note that the srmcp command is not very well supported.

Looking at the permissions

For a file that is copied to an SRM and that is registered in the LFC there are 2 sets of permissions:

1. SRM-level
2. LFC-level

These permissions are not directly related to each other and need to be modified separately. In this section we explain how to modify the SRM-level permissions. The LFC-level permissions are explained in Access control for the LFC.

$ srm-get-permissions $SRM/myfile
# file  : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/myfile
# owner : 18010
owner:18010:RW
user:18010:RW
group:1276:R
other:R

Just for the fun of it, let's list a file which we did not create ourselves:

$ srm-get-permissions srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/ronstestdir3
# file  : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/ronstestdir3
# owner : 18010
owner:18010:RWX
user:18010:RWX
group:1276:NONE
other:NONE

Wait a second... that file has exactly the same user (18010) and group (1276) !

Now let's check a directory that was created using a different VOMS proxy, this time one for VO vlemed:

$ srm-get-permissions srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/vlemed/generated/2007-11-07
# file  : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/vlemed/generated/2007-11-07
# owner : 18002
owner:18002:RWX
user:18002:RWX
group:1276:RX
other:RX

This time the user is different (18002) but the group is still 1276 ! We will have to keep this in mind when we want to limit access to our files.

Note: this limitation is due to the default setup of dCache. pnfs does not support user-level access control. An alternative is to use Chimera as the underlying file system, which does support user-level ACLs. However, the default Chimera settings after a migration from pnfs still map all users withing a single VO to the same dCache userid.

Modifying the permissions

Use

srm-set-permissions -h

to get a list of options for this command.

Let's remove all permissions for group and other on $SRM/myfile:

$ srm-set-permissions -type=CHANGE -group=NONE -other=NONE $SRM/myfile

(this command does not return any output on success)

Note that we have set the group permissions to <NONE> . If we do not do this then users from other VOs can still read the file.

Verifying access control

As the original user:

$ srm-get-permissions $SRM/myfile
# file  : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/myfile
# owner : 18010
owner:18010:RW
user:18010:RW
group:1276:NONE
other:NONE

Now if we switch to another VO:

$ voms-proxy-init --voms vlemed
Enter GRID pass phrase:
Your identity: /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser
Creating temporary proxy .......................................... Done
Contacting  voms.grid.sara.nl:30003 [/O=dutchgrid/O=hosts/OU=sara.nl/CN=voms.grid.sara.nl] "vlemed" Done
Creating proxy ............................. Done
Your proxy is valid until Thu Aug  6 23:45:56 2009

and then try to copy the file

$ srmcp $SRM/myfile file:///$PWD/blah
Thu Aug 06 11:46:41 CEST 2009: rs.state = Failed rs.error =  at Thu Aug 06 11:46:36 CEST 2009 state Pending : 
created RequestFileStatus#-2083337326 failed with error:[  at Thu Aug 06 11:46:36 CEST 2009 state Failed : 
user AR:3209444168817257963   vlemed /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser 18002 
read-write 0 / / < 1 groupLists :   GL:/vlemed 1 groups : [1276,]; >cannot read 00190000000000000018C718]

Thu Aug 06 11:46:41 CEST 2009: java.io.IOException: rs.state = Failed rs.error =  at Thu Aug 06 11:46:36 
CEST 2009 state Pending : created
RequestFileStatus#-2083337326 failed with error:[  at Thu Aug 06 11:46:36 CEST 2009 state Failed : 
user AR:3209444168817257963 vlemed /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser 18002
read-write 0 / / < 1 groupLists :   GL:/vlemed 1 groups : [1276,]; >cannot read 00190000000000000018C718]

srm copy of at least one file failed or not completed

We can no longer access the file: success!

Notes

• it currently is not possible with dCache PNFS to restrict access to a file for your fellow VO-members.
• it currently is also not known how it can be done using dCache Chimera.
• the dCache server has mapped the voms proxy to a different user (18002), which causes the srmcp request to fail. Thus dCache user+group mappings are slightly different from UNIX user+group mappings:
  • user+group1 = dcache_user1
  • user+group2 = dcache_user2

whereas on a regular UNIX filesystem you can always access a file that is owned by you, regardless of the groupid of the file.

Making files inaccessible to your VO

Here's a quick guide on how to make file inaccessible to your VO, including yourself:

• Change the permissions of the file to author=NONE, group=RWX:

srm-set-permissions -type=CHANGE -group=RWX -owner=NONE $SRM/myfile

• Verify:

$ srm-get-permissions $SRM/myfile
# file  : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/myfile
# owner : 18010
owner:18010:NONE
user:18010:NONE
group:1276:RWX
other:NONE

• Now try to read it:

$ srmcp -globus_tcp_port_range=20000,25000 $SRM/myfile file:///$PWD/blah
org.globus.ftp.exception.ServerException: Server refused performing the request. 
Custom message:  (error code 1) 
 [Nested exception message: Custom message: Unexpected reply: 550 Permission denied].
Nested exception is org.globus.ftp.exception.UnexpectedReplyCodeException:
Custom message: Unexpected reply: 550 Permission denied
       at org.globus.ftp.vanilla.TransferMonitor.run(TransferMonitor.java:195)
       at org.globus.ftp.vanilla.TransferMonitor.start(TransferMonitor.java:109)
       at org.globus.ftp.FTPClient.transferRunSingleThread(FTPClient.java:1457)
       at org.globus.ftp.FTPClient.get2(FTPClient.java:1724)
       at org.dcache.srm.util.GridftpClient$TransferThread.run(GridftpClient.java:951)
       at java.lang.Thread.run(Thread.java:619)
GridftpClient:  transfer exception
org.globus.ftp.exception.ServerException: Server refused performing the request. 
[SNIP]

so we can no longer read our own file. Now if we switch to another VO:

$ voms-proxy-init --voms vlemed
Enter GRID pass phrase:
Your identity: /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser
Creating temporary proxy .......................................... Done
Contacting  voms.grid.sara.nl:30003 [/O=dutchgrid/O=hosts/OU=sara.nl/CN=voms.grid.sara.nl] "vlemed" Done
Creating proxy ............................. Done
Your proxy is valid until Thu Aug  6 23:45:56 2009

and then try to copy the file

$ srmcp -globus_tcp_port_range=20000,25000 $SRM/myfile file:///$PWD/blah

we do get a copy of the file.

Conclusion

Access control in dCache is functional, with some notes:

• the srm-{get,set}-permissions work quite well
• making a file inaccessible to people outside of a VO is possible
• with PNFS it is NOT possible to control group-access to your files: all members of a VO can read/modify/delete all files.
• even after a migration from PNFS to Chimera it is still NOT possible for a "normal" user to control group-access to your files: all members of a VO can read/modify/delete all files. The same userid-sqash is used as with the PNFS setup.