Difference between revisions of "Access control for dCache storage elements"
From PDP/Grid Wiki
Jump to navigationJump to search| (24 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
This page is part of an investigation on [[How to control access rights for LFC/SRM files]] . | This page is part of an investigation on [[How to control access rights for LFC/SRM files]] . | ||
| − | = Finding out | + | = Finding out which storage systems are available = |
To list the storage systems to which you have access use | To list the storage systems to which you have access use | ||
| Line 22: | Line 22: | ||
771834491 620488567 n.a gb-se-lumc.lumc.nl | 771834491 620488567 n.a gb-se-lumc.lumc.nl | ||
| − | '''Note''' the current version of the <tt>lcg-infosites</tt> command does not use your grid proxy at all ! | + | '''Note #1''' the current version of the <tt>lcg-infosites</tt> command does not use your grid proxy at all ! |
| + | |||
| + | '''Note #2''' before using the <tt>srm*</tt> commands make sure that the SRM client is at least '''2.1.0''' as otherwise most, if not all, <tt>srm*</tt> commands will fail: | ||
| + | $ srmping -version | ||
| + | Storage Resource Manager (SRM) Client version 2.1.0 | ||
| + | Copyright (c) 2002-2008 Fermi National Accelerator Laborator | ||
| + | '''Note #3''' The glite 3.2 UI up to version 3.2.4 contains an '''older''' (actually, '''too old''' !) version of the srm-client than the glite 3.1 UI ! | ||
| + | |||
| + | |||
| + | You can use the <tt>srmping</tt> command to get some basic information about a storage system | ||
| + | $ srmping -2 srm://srm.grid.sara.nl | ||
| + | VersionInfo : v2.2 | ||
| + | backend_type:dCache | ||
| + | backend_version:production-1.9.3-3 | ||
| + | |||
| + | So host <tt>srm.grid.sara.nl</tt> is a SRMv2.2 compliant storage system based on dCache. | ||
| + | |||
| + | = Finding out how storage is organized = | ||
| − | You can | + | You can use the <tt>srmls</tt> command to figure out how the storage is organized: |
$ srmls srm://srm.grid.sara.nl | $ srmls srm://srm.grid.sara.nl | ||
512 // | 512 // | ||
| Line 30: | Line 47: | ||
4096 //pnfs/ | 4096 //pnfs/ | ||
[SNIP] | [SNIP] | ||
| − | This listing tells us that this storage system uses a PNFS file system, which means that we're talking to a dCache SRM. Subsequent <tt>srmls</tt> commands then give us: | + | This listing tells us that this storage system uses a PNFS file system, which means that we're talking to a dCache SRM with underlying PNFS file system. |
| + | |||
| + | If this command returned an error: | ||
| + | $ srmls srm://se.grid.rug.nl | ||
| + | srm client error: | ||
| + | java.lang.Exception: Return status: | ||
| + | - Status code: SRM_FAILURE | ||
| + | - Explanation: All ls requests failed in some way or another | ||
| + | then (my guess is that) we're talking to a dCache SRM with underlying Chimera file system. We will blindy try to list the <tt>/pnfs</tt> directory in the next example: | ||
| + | |||
| + | Subsequent <tt>srmls</tt> commands then give us: | ||
$ srmls srm://srm.grid.sara.nl/pnfs/ | $ srmls srm://srm.grid.sara.nl/pnfs/ | ||
512 /pnfs// | 512 /pnfs// | ||
512 /pnfs//grid.sara.nl/ | 512 /pnfs//grid.sara.nl/ | ||
[SNIP] | [SNIP] | ||
| − | + | and for the Chimera-based dCache: | |
| + | $ srmls srm://se.grid.rug.nl/pnfs/ | ||
| + | 512 /pnfs/ | ||
| + | 512 /pnfs/grid.rug.nl/ | ||
| + | Heeey! So even though we're talking to a Chimera-based dCache the starting directory is still <tt>/pnfs</tt>. | ||
| + | |||
$ srmls srm://srm.grid.sara.nl/pnfs/grid.sara.nl | $ srmls srm://srm.grid.sara.nl/pnfs/grid.sara.nl | ||
512 /pnfs/grid.sara.nl/ | 512 /pnfs/grid.sara.nl/ | ||
| Line 62: | Line 94: | ||
Next we will copy a file to our SRM directory and register it in the LFC in one go: | Next we will copy a file to our SRM directory and register it in the LFC in one go: | ||
| − | lcg-cr -d $SRM/myfile -l lfn:/grid/pvier/janjust/ | + | $ lcg-cr -d $SRM/myfile -l lfn:/grid/pvier/janjust/my-dcache-file file://$PWD/myfile |
| − | |||
guid:6b4c060f-cd95-4360-9e1b-a29d023d49b8 | guid:6b4c060f-cd95-4360-9e1b-a29d023d49b8 | ||
| + | which returns the LFC GUID for the file upon success. | ||
| − | (For details on how to find out how the LFC directory space is organized see | + | (For details on how to find out how the LFC directory space is organized see [[Access control for the LFC]]) |
| + | |||
| + | |||
| + | We could also have copied the file to the SRM only, bypassing the LFC registration, using the command | ||
| + | lcg-cp file://$PWD/myfile $SRM/myfile | ||
| + | or even | ||
| + | srmcp -globus_tcp_port_range=20000,25000 file://$PWD/myfile $SRM/myfile | ||
| + | but as we needed the LFC entry as well we used the (preferred) <tt>lcg-cr</tt> command. Please also note | ||
| + | that the <tt>srmcp</tt> command is not very well supported. | ||
= Looking at the permissions = | = Looking at the permissions = | ||
| − | For a file that is copied to SRM and that is registered in the LFC there are 2 sets of permissions: | + | For a file that is copied to an SRM and that is registered in the LFC there are 2 sets of permissions: |
# SRM-level | # SRM-level | ||
# LFC-level | # LFC-level | ||
| Line 105: | Line 145: | ||
We will have to keep this in mind when we want to limit access to our files. | We will have to keep this in mind when we want to limit access to our files. | ||
| − | '''Note''': this limitation is due to the setup of | + | '''Note''': this limitation is due to the default setup of dCache. |
| − | + | <tt>pnfs</tt> does not support user-level access control. An alternative is to use <tt>Chimera</tt> as the underlying | |
| − | <tt>pnfs</tt> does not support user-level access control. An alternative is to use <tt> | + | file system, which does support user-level ACLs. However, the default Chimera settings after a migration from <tt>pnfs</tt> |
| − | file system, which does support user-level ACLs, | + | '''still''' map all users withing a single VO to the same dCache userid. |
| − | |||
= Modifying the permissions = | = Modifying the permissions = | ||
| Line 116: | Line 155: | ||
srm-set-permissions -h | srm-set-permissions -h | ||
to get a list of options for this command. | to get a list of options for this command. | ||
| + | |||
| + | Let's remove all permissions for <tt>group</tt> and <tt>other</tt> on $SRM/myfile: | ||
$ srm-set-permissions -type=CHANGE -group=NONE -other=NONE $SRM/myfile | $ srm-set-permissions -type=CHANGE -group=NONE -other=NONE $SRM/myfile | ||
(this command does not return any output on success) | (this command does not return any output on success) | ||
| Line 142: | Line 183: | ||
Your proxy is valid until Thu Aug 6 23:45:56 2009 | Your proxy is valid until Thu Aug 6 23:45:56 2009 | ||
and then try to copy the file | and then try to copy the file | ||
| − | $ srmcp $SRM/myfile file://$PWD/blah | + | $ srmcp $SRM/myfile file:///$PWD/blah |
Thu Aug 06 11:46:41 CEST 2009: rs.state = Failed rs.error = at Thu Aug 06 11:46:36 CEST 2009 state Pending : | Thu Aug 06 11:46:41 CEST 2009: rs.state = Failed rs.error = at Thu Aug 06 11:46:36 CEST 2009 state Pending : | ||
created RequestFileStatus#-2083337326 failed with error:[ at Thu Aug 06 11:46:36 CEST 2009 state Failed : | created RequestFileStatus#-2083337326 failed with error:[ at Thu Aug 06 11:46:36 CEST 2009 state Failed : | ||
| Line 160: | Line 201: | ||
'''Notes''' | '''Notes''' | ||
* it currently is '''not''' possible with dCache PNFS to restrict access to a file for your fellow VO-members. | * it currently is '''not''' possible with dCache PNFS to restrict access to a file for your fellow VO-members. | ||
| + | * it currently is also not known how it can be done using dCache Chimera. | ||
* the dCache server has mapped the voms proxy to a different user (18002), which causes the <tt>srmcp</tt> request to fail. Thus dCache user+group mappings are slightly different from UNIX user+group mappings: | * the dCache server has mapped the voms proxy to a different user (18002), which causes the <tt>srmcp</tt> request to fail. Thus dCache user+group mappings are slightly different from UNIX user+group mappings: | ||
** user+group1 = dcache_user1 | ** user+group1 = dcache_user1 | ||
| Line 165: | Line 207: | ||
whereas on a regular UNIX filesystem you can always access a file that is owned by you, regardless of the | whereas on a regular UNIX filesystem you can always access a file that is owned by you, regardless of the | ||
groupid of the file. | groupid of the file. | ||
| + | |||
| + | = Making files inaccessible to your VO = | ||
| + | Here's a quick guide on how to make file inaccessible to your VO, including yourself: | ||
| + | * Change the permissions of the file to '''author=NONE, group=RWX''': | ||
| + | srm-set-permissions -type=CHANGE -group=RWX -owner=NONE $SRM/myfile | ||
| + | * Verify: | ||
| + | $ srm-get-permissions $SRM/myfile | ||
| + | # file : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/myfile | ||
| + | # owner : 18010 | ||
| + | owner:18010:NONE | ||
| + | user:18010:NONE | ||
| + | group:1276:RWX | ||
| + | other:NONE | ||
| + | * Now try to read it: | ||
| + | $ srmcp -globus_tcp_port_range=20000,25000 $SRM/myfile file:///$PWD/blah | ||
| + | org.globus.ftp.exception.ServerException: Server refused performing the request. | ||
| + | Custom message: (error code 1) | ||
| + | [Nested exception message: Custom message: Unexpected reply: 550 Permission denied]. | ||
| + | Nested exception is org.globus.ftp.exception.UnexpectedReplyCodeException: | ||
| + | Custom message: Unexpected reply: 550 Permission denied | ||
| + | at org.globus.ftp.vanilla.TransferMonitor.run(TransferMonitor.java:195) | ||
| + | at org.globus.ftp.vanilla.TransferMonitor.start(TransferMonitor.java:109) | ||
| + | at org.globus.ftp.FTPClient.transferRunSingleThread(FTPClient.java:1457) | ||
| + | at org.globus.ftp.FTPClient.get2(FTPClient.java:1724) | ||
| + | at org.dcache.srm.util.GridftpClient$TransferThread.run(GridftpClient.java:951) | ||
| + | at java.lang.Thread.run(Thread.java:619) | ||
| + | GridftpClient: transfer exception | ||
| + | org.globus.ftp.exception.ServerException: Server refused performing the request. | ||
| + | [SNIP] | ||
| + | so we can no longer read our own file. Now if we switch to another VO: | ||
| + | $ voms-proxy-init --voms vlemed | ||
| + | Enter GRID pass phrase: | ||
| + | Your identity: /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser | ||
| + | Creating temporary proxy .......................................... Done | ||
| + | Contacting voms.grid.sara.nl:30003 [/O=dutchgrid/O=hosts/OU=sara.nl/CN=voms.grid.sara.nl] "vlemed" Done | ||
| + | Creating proxy ............................. Done | ||
| + | Your proxy is valid until Thu Aug 6 23:45:56 2009 | ||
| + | and then try to copy the file | ||
| + | $ srmcp -globus_tcp_port_range=20000,25000 $SRM/myfile file:///$PWD/blah | ||
| + | we '''do''' get a copy of the file. | ||
| + | |||
| + | = Conclusion = | ||
| + | Access control in dCache is functional, with some notes: | ||
| + | * the <tt>srm-{get,set}-permissions</tt> work quite well | ||
| + | * making a file inaccessible to people outside of a VO is possible | ||
| + | * with PNFS it is '''NOT''' possible to control group-access to your files: all members of a VO can read/modify/delete all files. | ||
| + | * even after a migration from PNFS to Chimera it is still '''NOT''' possible for a "normal" user to control group-access to your files: all members of a VO can read/modify/delete all files. The same userid-sqash is used as with the PNFS setup. | ||
Latest revision as of 09:32, 4 November 2010
This page is part of an investigation on How to control access rights for LFC/SRM files .
Finding out which storage systems are available
To list the storage systems to which you have access use
lcg-infosites --vo <YOUR-VO> se
which results in
Avail Space(Kb) Used Space(Kb) Type SEs ---------------------------------------------------------- 12078 108 n.a srm.grid.rug.nl 12078 108 n.a srm.grid.rug.nl 730582644 681194097 n.a gb-se-amc.amc.nl 8226695519985 23304480014 n.a srm.grid.sara.nl 605355546 806421195 n.a gb-se-nki.els.sara.nl 6575746866 20920246 n.a carme.htc.biggrid.nl 152913518 115521938 n.a se.grid.rug.nl 248345185 1166074827 n.a gb-se-ams.els.sara.nl 355230761 1056545980 n.a gb-se-uu.science.uu.nl 1266740857 145035883 n.a gb-se-wur.els.sara.nl 337812899 1076607113 n.a gb-se-kun.els.sara.nl 2195706454 3048365 n.a tbn18.nikhef.nl 771834491 620488567 n.a gb-se-lumc.lumc.nl
Note #1 the current version of the lcg-infosites command does not use your grid proxy at all !
Note #2 before using the srm* commands make sure that the SRM client is at least 2.1.0 as otherwise most, if not all, srm* commands will fail:
$ srmping -version Storage Resource Manager (SRM) Client version 2.1.0 Copyright (c) 2002-2008 Fermi National Accelerator Laborator
Note #3 The glite 3.2 UI up to version 3.2.4 contains an older (actually, too old !) version of the srm-client than the glite 3.1 UI !
You can use the srmping command to get some basic information about a storage system
$ srmping -2 srm://srm.grid.sara.nl VersionInfo : v2.2 backend_type:dCache backend_version:production-1.9.3-3
So host srm.grid.sara.nl is a SRMv2.2 compliant storage system based on dCache.
Finding out how storage is organized
You can use the srmls command to figure out how the storage is organized:
$ srmls srm://srm.grid.sara.nl
512 //
[SNIP]
4096 //pnfs/
[SNIP]
This listing tells us that this storage system uses a PNFS file system, which means that we're talking to a dCache SRM with underlying PNFS file system.
If this command returned an error:
$ srmls srm://se.grid.rug.nl srm client error: java.lang.Exception: Return status: - Status code: SRM_FAILURE - Explanation: All ls requests failed in some way or another
then (my guess is that) we're talking to a dCache SRM with underlying Chimera file system. We will blindy try to list the /pnfs directory in the next example:
Subsequent srmls commands then give us:
$ srmls srm://srm.grid.sara.nl/pnfs/
512 /pnfs//
512 /pnfs//grid.sara.nl/
[SNIP]
and for the Chimera-based dCache:
$ srmls srm://se.grid.rug.nl/pnfs/
512 /pnfs/
512 /pnfs/grid.rug.nl/
Heeey! So even though we're talking to a Chimera-based dCache the starting directory is still /pnfs.
$ srmls srm://srm.grid.sara.nl/pnfs/grid.sara.nl
512 /pnfs/grid.sara.nl/
512 /pnfs/grid.sara.nl/disk/
512 /pnfs/grid.sara.nl/tapetests/
512 /pnfs/grid.sara.nl/data/
512 /pnfs/grid.sara.nl/disktests/
$ srmls srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data
0 /pnfs/grid.sara.nl/data/
[SNIP}
512 /pnfs/grid.sara.nl/data/pvier/
512 /pnfs/grid.sara.nl/data/vlemed/
Hey, we are at the VO level now. Here I've listed the two VOs which will be used throughout this page.
Creating your own directory in SRM-space
Before we copy a file to the dCache SRM we first create our own directory. If we do not do this then SRM will store the files in generated directories, over which we have little or no control.
srmmkdir srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/
The URL for this directory will be used throughout the rest of this page, hence we abbreviate it to
SRM=srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust
Copying and registering your file
Next we will copy a file to our SRM directory and register it in the LFC in one go:
$ lcg-cr -d $SRM/myfile -l lfn:/grid/pvier/janjust/my-dcache-file file://$PWD/myfile guid:6b4c060f-cd95-4360-9e1b-a29d023d49b8
which returns the LFC GUID for the file upon success.
(For details on how to find out how the LFC directory space is organized see Access control for the LFC)
We could also have copied the file to the SRM only, bypassing the LFC registration, using the command
lcg-cp file://$PWD/myfile $SRM/myfile
or even
srmcp -globus_tcp_port_range=20000,25000 file://$PWD/myfile $SRM/myfile
but as we needed the LFC entry as well we used the (preferred) lcg-cr command. Please also note that the srmcp command is not very well supported.
Looking at the permissions
For a file that is copied to an SRM and that is registered in the LFC there are 2 sets of permissions:
- SRM-level
- LFC-level
These permissions are not directly related to each other and need to be modified separately. In this section we explain how to modify the SRM-level permissions. The LFC-level permissions are explained in Access control for the LFC.
$ srm-get-permissions $SRM/myfile # file : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/myfile # owner : 18010 owner:18010:RW user:18010:RW group:1276:R other:R
Just for the fun of it, let's list a file which we did not create ourselves:
$ srm-get-permissions srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/ronstestdir3 # file : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/ronstestdir3 # owner : 18010 owner:18010:RWX user:18010:RWX group:1276:NONE other:NONE
Wait a second... that file has exactly the same user (18010) and group (1276) !
Now let's check a directory that was created using a different VOMS proxy, this time one for VO vlemed:
$ srm-get-permissions srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/vlemed/generated/2007-11-07 # file : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/vlemed/generated/2007-11-07 # owner : 18002 owner:18002:RWX user:18002:RWX group:1276:RX other:RX
This time the user is different (18002) but the group is still 1276 ! We will have to keep this in mind when we want to limit access to our files.
Note: this limitation is due to the default setup of dCache. pnfs does not support user-level access control. An alternative is to use Chimera as the underlying file system, which does support user-level ACLs. However, the default Chimera settings after a migration from pnfs still map all users withing a single VO to the same dCache userid.
Modifying the permissions
Use
srm-set-permissions -h
to get a list of options for this command.
Let's remove all permissions for group and other on $SRM/myfile:
$ srm-set-permissions -type=CHANGE -group=NONE -other=NONE $SRM/myfile
(this command does not return any output on success)
Note that we have set the group permissions to <NONE> . If we do not do this then users from other VOs can still read the file.
Verifying access control
As the original user:
$ srm-get-permissions $SRM/myfile # file : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/myfile # owner : 18010 owner:18010:RW user:18010:RW group:1276:NONE other:NONE
Now if we switch to another VO:
$ voms-proxy-init --voms vlemed Enter GRID pass phrase: Your identity: /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser Creating temporary proxy .......................................... Done Contacting voms.grid.sara.nl:30003 [/O=dutchgrid/O=hosts/OU=sara.nl/CN=voms.grid.sara.nl] "vlemed" Done Creating proxy ............................. Done Your proxy is valid until Thu Aug 6 23:45:56 2009
and then try to copy the file
$ srmcp $SRM/myfile file:///$PWD/blah Thu Aug 06 11:46:41 CEST 2009: rs.state = Failed rs.error = at Thu Aug 06 11:46:36 CEST 2009 state Pending : created RequestFileStatus#-2083337326 failed with error:[ at Thu Aug 06 11:46:36 CEST 2009 state Failed : user AR:3209444168817257963 vlemed /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser 18002 read-write 0 / / < 1 groupLists : GL:/vlemed 1 groups : [1276,]; >cannot read 00190000000000000018C718] Thu Aug 06 11:46:41 CEST 2009: java.io.IOException: rs.state = Failed rs.error = at Thu Aug 06 11:46:36 CEST 2009 state Pending : created RequestFileStatus#-2083337326 failed with error:[ at Thu Aug 06 11:46:36 CEST 2009 state Failed : user AR:3209444168817257963 vlemed /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser 18002 read-write 0 / / < 1 groupLists : GL:/vlemed 1 groups : [1276,]; >cannot read 00190000000000000018C718] srm copy of at least one file failed or not completed
We can no longer access the file: success!
Notes
- it currently is not possible with dCache PNFS to restrict access to a file for your fellow VO-members.
- it currently is also not known how it can be done using dCache Chimera.
- the dCache server has mapped the voms proxy to a different user (18002), which causes the srmcp request to fail. Thus dCache user+group mappings are slightly different from UNIX user+group mappings:
- user+group1 = dcache_user1
- user+group2 = dcache_user2
whereas on a regular UNIX filesystem you can always access a file that is owned by you, regardless of the groupid of the file.
Making files inaccessible to your VO
Here's a quick guide on how to make file inaccessible to your VO, including yourself:
- Change the permissions of the file to author=NONE, group=RWX:
srm-set-permissions -type=CHANGE -group=RWX -owner=NONE $SRM/myfile
- Verify:
$ srm-get-permissions $SRM/myfile # file : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/myfile # owner : 18010 owner:18010:NONE user:18010:NONE group:1276:RWX other:NONE
- Now try to read it:
$ srmcp -globus_tcp_port_range=20000,25000 $SRM/myfile file:///$PWD/blah
org.globus.ftp.exception.ServerException: Server refused performing the request.
Custom message: (error code 1)
[Nested exception message: Custom message: Unexpected reply: 550 Permission denied].
Nested exception is org.globus.ftp.exception.UnexpectedReplyCodeException:
Custom message: Unexpected reply: 550 Permission denied
at org.globus.ftp.vanilla.TransferMonitor.run(TransferMonitor.java:195)
at org.globus.ftp.vanilla.TransferMonitor.start(TransferMonitor.java:109)
at org.globus.ftp.FTPClient.transferRunSingleThread(FTPClient.java:1457)
at org.globus.ftp.FTPClient.get2(FTPClient.java:1724)
at org.dcache.srm.util.GridftpClient$TransferThread.run(GridftpClient.java:951)
at java.lang.Thread.run(Thread.java:619)
GridftpClient: transfer exception
org.globus.ftp.exception.ServerException: Server refused performing the request.
[SNIP]
so we can no longer read our own file. Now if we switch to another VO:
$ voms-proxy-init --voms vlemed Enter GRID pass phrase: Your identity: /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser Creating temporary proxy .......................................... Done Contacting voms.grid.sara.nl:30003 [/O=dutchgrid/O=hosts/OU=sara.nl/CN=voms.grid.sara.nl] "vlemed" Done Creating proxy ............................. Done Your proxy is valid until Thu Aug 6 23:45:56 2009
and then try to copy the file
$ srmcp -globus_tcp_port_range=20000,25000 $SRM/myfile file:///$PWD/blah
we do get a copy of the file.
Conclusion
Access control in dCache is functional, with some notes:
- the srm-{get,set}-permissions work quite well
- making a file inaccessible to people outside of a VO is possible
- with PNFS it is NOT possible to control group-access to your files: all members of a VO can read/modify/delete all files.
- even after a migration from PNFS to Chimera it is still NOT possible for a "normal" user to control group-access to your files: all members of a VO can read/modify/delete all files. The same userid-sqash is used as with the PNFS setup.
