Latest revision as of 15:48, 10 March 2010

This page is part of an investigation on How to control access rights for LFC/SRM files .

Finding out which storage systems are available

To list the storage systems to which you have access use

 lcg-infosites --vo <YOUR-VO> se

which results in

Avail Space(Kb) Used Space(Kb)  Type   SEs
----------------------------------------------------------
12078           108             n.a    srm.grid.rug.nl
12078           108             n.a    srm.grid.rug.nl
730582644       681194097       n.a    gb-se-amc.amc.nl
8226695519985   23304480014     n.a    srm.grid.sara.nl
605355546       806421195       n.a    gb-se-nki.els.sara.nl
6575746866      20920246        n.a    carme.htc.biggrid.nl
152913518       115521938       n.a    se.grid.rug.nl
248345185       1166074827      n.a    gb-se-ams.els.sara.nl
355230761       1056545980      n.a    gb-se-uu.science.uu.nl
1266740857      145035883       n.a    gb-se-wur.els.sara.nl
337812899       1076607113      n.a    gb-se-kun.els.sara.nl
2195706454      3048365         n.a    tbn18.nikhef.nl
771834491       620488567       n.a    gb-se-lumc.lumc.nl

Note #1 the current version of the lcg-infosites command does not use your grid proxy at all !

Note #2 before using the srm* commands make sure that the SRM client is at least 2.1.0 as otherwise most, if not all, srm* commands will fail:

$ srmping -version
Storage Resource Manager (SRM) Client version 2.1.0
Copyright (c) 2002-2008 Fermi National Accelerator Laborator

Note #3 The glite 3.2 UI up to version 3.2.4 contains an older (actually, too old !) version of the srm-client than the glite 3.1 UI !

You can use the srmping command to get some basic information about a storage system

$ srmping -2  srm://srm.grid.rug.nl
SRMClientV2 : srmPing: try # 0 failed with error
SRMClientV2 : ; nested exception is:
        java.net.NoRouteToHostException: No route to host
SRMClientV2 : srmPing: try again

Ah, the SRM server is listening on a different port. Storm documentation suggests using port 8444:

$ srmping -2  srm://srm.grid.rug.nl:8444
VersionInfo : v2.2
backend_type:StoRM
backend_version:<FE:1.5.0-1.sl4><BE:1.5.1-2.sl4>

So host srm.grid.rug.nl is a SRMv2.2 compliant storage system based on StoRM

Finding out how storage is organized

You can use the srmls command to figure out how the storage is organized:

$ srmls -l srm://srm.grid.rug.nl:8444/
srm client error:
java.lang.Exception: Return status:
 - Status code:  SRM_FAILURE
 - Explanation:  All requests failed

It seems that StoRM does not allow the listing of arbitary remote directories. By looking at a file that was previously stored on srm.grid.rug.nl (using lcg-lr) we guess that the directory structure starts with the VO:

$ srmls srm://srm.grid.rug.nl:8444/pvier/
 0 /pvier/
     0 /pvier/vletgenerated
     0 /pvier/generated

Indeed, we're at the VO level.

Creating your own directory in SRM-space

Before we copy a file to the StoRM SRM we first create our own directory. If we do not do this then SRM will store the files in generated directories, over which we have little or no control.

srmmkdir srm://srm.grid.rug.nl:8444/pvier/janjust

The URL for this directory will be used throughout the rest of this page, hence we abbreviate it to

SRM=srm://srm.grid.rug.nl:8444/pvier/janjust

Copying and registering your file

Next we will copy a file to our SRM directory and register it in the LFC in one go:

$ lcg-cr -d $SRM/myfile -l lfn:/grid/pvier/janjust/my-storm-file file://$PWD/myfile
guid:1601bad4-c236-4f01-a4fb-54e12c7dc0a9

which returns the LFC GUID for the file upon success.

(For details on how to find out how the LFC directory space is organized see Access control for the LFC)

We could also have copied the file to the SRM only, bypassing the LFC registration, using the command

lcg-cp file://$PWD/myfile $SRM/myfile

or even

srmcp -globus_tcp_port_range=20000,25000 file://$PWD/myfile $SRM/myfile

but as we needed the LFC entry as well we used the (preferred) lcg-cr command. Please also note that the srmcp command is not very well supported.

Looking at the permissions

For a file that is copied to an SRM and that is registered in the LFC there are 2 sets of permissions:

SRM-level
LFC-level

These permissions are not directly related to each other and need to be modified separately. In this section we explain how to modify the SRM-level permissions. The LFC-level permissions are explained in Access control for the LFC.

$ srm-get-permissions $SRM/myfile
srm-get-permissions $SRM/myfile
Return code: SRM_NOT_SUPPORTED
Explanation: Not supported

permissions array is null

Uh-oh... it looks like StoRM does not support permissions or access control

Modifying the permissions

Not supported

Verifying access control

Not supported

Conclusion

StoRM does not seem to support access control at all, at least not on the SRM-level.

Difference between revisions of "Access control for StoRM storage elements"

Latest revision as of 15:48, 10 March 2010

Contents

Finding out which storage systems are available

Finding out how storage is organized

Creating your own directory in SRM-space

Copying and registering your file

Looking at the permissions

Modifying the permissions

Verifying access control

Conclusion

Navigation menu

Page actions

Page actions

Personal tools

Navigation

Search

Tools

@@ Line 1: / Line 1: @@
 This page is part of an investigation on [[How to control access rights for LFC/SRM files]] .
-= Finding out how storage is organized =
+= Finding out which storage systems are available =
 To list the storage systems to which you have access use
@@ Line 22: / Line 22: @@
   771834491       620488567       n.a    gb-se-lumc.lumc.nl
-'''Note''' the current version of the <tt>lcg-infosites</tt> command does not use your grid proxy at all !
+'''Note #1''' the current version of the <tt>lcg-infosites</tt> command does not use your grid proxy at all !
+'''Note #2''' before using the <tt>srm*</tt> commands make sure that the SRM client is at least '''2.1.0''' as otherwise most, if not all, <tt>srm*</tt> commands will fail:
+ $ srmping -version
+ Storage Resource Manager (SRM) Client version 2.1.0
+ Copyright (c) 2002-2008 Fermi National Accelerator Laborator
+'''Note #3''' The glite 3.2 UI up to version 3.2.4 contains an '''older''' (actually, '''too old''' !) version of the srm-client than the glite 3.1 UI !
-You can then try to use the <tt>srmls</tt> command to figure out how the storage is organized:
+You can use the <tt>srmping</tt> command to get some basic information about a storage system
-  $ srmls -l srm://srm.grid.rug.nl/
+  $ srmping -2  srm://srm.grid.rug.nl
-  SRMClientV2 : srmLs: try # 0 failed with error
+  SRMClientV2 : srmPing: try # 0 failed with error
   SRMClientV2 : ; nested exception is:
           java.net.NoRouteToHostException: No route to host
-  SRMClientV2 : srmLs: try again
+  SRMClientV2 : srmPing: try again
 Ah, the SRM server is listening on a different port. Storm documentation suggests using port 8444:
+ $ srmping -2  srm://srm.grid.rug.nl:8444
+ VersionInfo : v2.2
+ backend_type:StoRM
+ backend_version:<FE:1.5.0-1.sl4><BE:1.5.1-2.sl4>
+So host <tt>srm.grid.rug.nl</tt> is a SRMv2.2 compliant storage system based on StoRM
+= Finding out how storage is organized =
+You can use the <tt>srmls</tt> command to figure out how the storage is organized:
   $ srmls -l srm://srm.grid.rug.nl:8444/
   srm client error:
@@ Line 36: / Line 52: @@
    - Status code:  SRM_FAILURE
    - Explanation:  All requests failed
-Progress, more or less...
+It seems that StoRM does not allow the listing of arbitary remote directories.
 By looking at a file that was previously stored on srm.grid.rug.nl (using <tt>lcg-lr</tt>) we guess that the directory structure starts with the VO:
   $ srmls srm://srm.grid.rug.nl:8444/pvier/
@@ Line 44: / Line 59: @@
 /pvier/generated
 Indeed, we're at the VO level.
 = Creating your own directory in SRM-space =
-Before we copy a file to the dCache SRM we first create our own directory. If we do not do this then SRM will
+Before we copy a file to the StoRM SRM we first create our own directory. If we do not do this then SRM will
 store the files in <tt>generated</tt> directories, over which we have little or no control.
   srmmkdir srm://srm.grid.rug.nl:8444/pvier/janjust
@@ Line 78: / Line 94: @@
 in [[Access control for the LFC]].
   $ srm-get-permissions $SRM/myfile
-  # file  : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/myfile
+  srm-get-permissions $SRM/myfile
-  # owner : 18010
+  Return code: SRM_NOT_SUPPORTED
- owner:18010:RW
+  Explanation: Not supported
- user:18010:RW
- group:1276:R
-  other:R
-Just for the fun of it, let's list a file which we did not create ourselves:
+  permissions array is null
-  $ srm-get-permissions srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/ronstestdir3
- # file  : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/ronstestdir3
- # owner : 18010
- owner:18010:RWX
- user:18010:RWX
- group:1276:NONE
- other:NONE
-Wait a second... that file has '''exactly''' the same user (18010) and group (1276) !
-Now let's check a directory that was created using a different VOMS proxy, this time one for VO <tt>vlemed</tt>:
- $ srm-get-permissions srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/vlemed/generated/2007-11-07
- # file  : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/vlemed/generated/2007-11-07
- # owner : 18002
- owner:18002:RWX
- user:18002:RWX
- group:1276:RX
- other:RX
-This time the user is different (18002) but the group is still 1276 !
-We will have to keep this in mind when we want to limit access to our files.
-'''Note''': this limitation is due to the setup of the dCache storage system at SARA, where <tt>pnfs</tt> is used as
+'''Uh-oh... it looks like StoRM does not support permissions or access control'''
-the underlying file system.
-<tt>pnfs</tt> does not support user-level access control. An alternative is to use <tt>chimera</tt> as the underlying
-file system, which does support user-level ACLs, but this would require a migration of '''all''' data currently in
-dCache, as well as a (manual) re-synchronization between dCache and all LFCs that have entries pointing to dCache.
 = Modifying the permissions =
-Use
+''' Not supported '''
- srm-set-permissions -h
-to get a list of options for this command.
- $ srm-set-permissions -type=CHANGE -group=NONE -other=NONE $SRM/myfile
-(this command does not return any output on success)
-'''Note''' that we have set the group permissions to <NONE> . If we do not do this then users from other VOs can still read the file.
 = Verifying access control =
-As the original user:
+''' Not supported '''
- $ srm-get-permissions $SRM/myfile
- # file  : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/myfile
- # owner : 18010
- owner:18010:RW
- user:18010:RW
- group:1276:NONE
- other:NONE
-Now if we switch to another VO:
- $ voms-proxy-init --voms vlemed
+= Conclusion =
- Enter GRID pass phrase:
+StoRM does not seem to support access control at all, at least not on the SRM-level.
- Your identity: /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser
- Creating temporary proxy .......................................... Done
- Contacting  voms.grid.sara.nl:30003 [/O=dutchgrid/O=hosts/OU=sara.nl/CN=voms.grid.sara.nl] "vlemed" Done
- Creating proxy ............................. Done
- Your proxy is valid until Thu Aug  6 23:45:56 2009
-and then try to copy the file
- $ srmcp $SRM/myfile file:///$PWD/blah
- Thu Aug 06 11:46:41 CEST 2009: rs.state = Failed rs.error =  at Thu Aug 06 11:46:36 CEST 2009 state Pending :
- created RequestFileStatus#-2083337326 failed with error:[  at Thu Aug 06 11:46:36 CEST 2009 state Failed :
- user AR:3209444168817257963   vlemed /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser 18002
- read-write 0 / / < 1 groupLists :   GL:/vlemed 1 groups : [1276,]; >cannot read 00190000000000000018C718]
- Thu Aug 06 11:46:41 CEST 2009: java.io.IOException: rs.state = Failed rs.error =  at Thu Aug 06 11:46:36
- CEST 2009 state Pending : created
- RequestFileStatus#-2083337326 failed with error:[  at Thu Aug 06 11:46:36 CEST 2009 state Failed :
- user AR:3209444168817257963 vlemed /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser 18002
- read-write 0 / / < 1 groupLists :   GL:/vlemed 1 groups : [1276,]; >cannot read 00190000000000000018C718]
- srm copy of at least one file failed or not completed
-We can no longer access the file: success!
-'''Notes'''
-* it currently is '''not''' possible with dCache PNFS to restrict access to a file for your fellow VO-members.
-* the dCache server has mapped the voms proxy to a different user (18002), which causes the <tt>srmcp</tt> request to fail. Thus dCache user+group mappings are slightly different from UNIX user+group mappings:
-** user+group1 = dcache_user1
-** user+group2 = dcache_user2
-whereas on a regular UNIX filesystem you can always access a file that is owned by you, regardless of the
-groupid of the file.
-= Making files inaccessible to your VO =
-Here's a quick guide on how to make file inaccessible to your VO, including yourself:
-* Change the permissions of the file to '''author=NONE, group=RWX''':
- srm-set-permissions -type=CHANGE -group=RWX -owner=NONE $SRM/myfile
-* Verify:
- $ srm-get-permissions $SRM/myfile
- # file  : srm://srm.grid.sara.nl/pnfs/grid.sara.nl/data/pvier/janjust/myfile
- # owner : 18010
- owner:18010:NONE
- user:18010:NONE
- group:1276:RWX
- other:NONE
-* Now try to read it:
- $ srmcp -globus_tcp_port_range=20000,25000 $SRM/myfile file:///$PWD/blah
- org.globus.ftp.exception.ServerException: Server refused performing the request.
- Custom message:  (error code 1)
-  [Nested exception message: Custom message: Unexpected reply: 550 Permission denied].
- Nested exception is org.globus.ftp.exception.UnexpectedReplyCodeException:
- Custom message: Unexpected reply: 550 Permission denied
-        at org.globus.ftp.vanilla.TransferMonitor.run(TransferMonitor.java:195)
-        at org.globus.ftp.vanilla.TransferMonitor.start(TransferMonitor.java:109)
-        at org.globus.ftp.FTPClient.transferRunSingleThread(FTPClient.java:1457)
-        at org.globus.ftp.FTPClient.get2(FTPClient.java:1724)
-        at org.dcache.srm.util.GridftpClient$TransferThread.run(GridftpClient.java:951)
-        at java.lang.Thread.run(Thread.java:619)
- GridftpClient:  transfer exception
- org.globus.ftp.exception.ServerException: Server refused performing the request.
- [SNIP]
-so we can no longer read our own file. Now if we switch to another VO:
- $ voms-proxy-init --voms vlemed
- Enter GRID pass phrase:
- Your identity: /O=dutchgrid/O=users/O=nikhef/CN=Jan Just Keijser
- Creating temporary proxy .......................................... Done
- Contacting  voms.grid.sara.nl:30003 [/O=dutchgrid/O=hosts/OU=sara.nl/CN=voms.grid.sara.nl] "vlemed" Done
- Creating proxy ............................. Done
- Your proxy is valid until Thu Aug  6 23:45:56 2009
-and then try to copy the file
- $ srmcp -globus_tcp_port_range=20000,25000 $SRM/myfile file:///$PWD/blah
-we '''do''' get a copy of the file.