Difference between revisions of "DCache"

From PDP/Grid Wiki
Jump to navigationJump to search
Line 5: Line 5:
 
= Usage =
 
= Usage =
  
dCache can be used pretty much as a normal file system. The main difference is that you can not change the contents of a file stored in dCache once the file is written. So for example you can't overwrite a file:
+
dCache can be used pretty much as a normal file system. Access to files is controlled by the usual unix file permissions and the main difference is that you can not change the contents of a file stored in dCache once the file is written. So for example you can't overwrite a file:
  
 
<code>
 
<code>
 
  <nowiki>
 
  <nowiki>
[andrewp@stbc-i1 /dcache/atlas/andrewp]$ echo 123 > test.file
+
[stbc-i1 /dcache/test]$ echo 123 > test.file
[andrewp@stbc-i1 /dcache/atlas/andrewp]$ echo 1234 > test.file
+
[stbc-i1 /dcache/test]$ echo 1234 > test.file
 
-bash: test.file: Permission denied</nowiki>
 
-bash: test.file: Permission denied</nowiki>
 
</code>
 
</code>
Line 18: Line 18:
 
<code>
 
<code>
 
  <nowiki>
 
  <nowiki>
[andrewp@stbc-i2 /dcache/atlas/andrewp]$ echo 123 > test.file
+
[stbc-i2 /dcache/test]$ echo 123 > test.file
[andrewp@stbc-i2 /dcache/atlas/andrewp]$ echo 4 >> test.file
+
[stbc-i2 /dcache/test]$ echo 4 >> test.file
 
-bash: test.file: Permission denied</nowiki>
 
-bash: test.file: Permission denied</nowiki>
 
</code>
 
</code>
Line 27: Line 27:
 
<code>
 
<code>
 
  <nowiki>
 
  <nowiki>
[andrewp@stbc-i2 /dcache/atlas/andrewp]$ ll test.file
+
[stbc-i2 /dcache/test]$ ll test.file
 
-rw-r--r--. 1 andrewp datagrid 4 May  4 09:57 test.file
 
-rw-r--r--. 1 andrewp datagrid 4 May  4 09:57 test.file
[andrewp@stbc-i2 /dcache/atlas/andrewp]$ rm test.file
+
[stbc-i2 /dcache/test]$ rm test.file
 
rm: remove write-protected regular file `test.file'? y</nowiki>
 
rm: remove write-protected regular file `test.file'? y</nowiki>
 
</code>
 
</code>
  
 
These limitations mean that dCache is ''not'' suited for such things as compiling code on, storing large files being actively edited (e.g. word documents) or storing files under a version control system.
 
These limitations mean that dCache is ''not'' suited for such things as compiling code on, storing large files being actively edited (e.g. word documents) or storing files under a version control system.
 +
 +
= Storage Space =
 +
 +
The total storage space on the dCache storage nodes is about 550 TB. This space is divided up amongst the various groups using the system and is organised into a directory hierarchy:
 +
<code>
 +
<nowiki>
 +
[andrewp@stbc-i4 ~]$ ll /dcache/
 +
total 2
 +
drwxr-xr-x. 4 root alice    512 Apr  1 11:46 alice
 +
drwxr-sr-x. 5 root atlas    512 May  4 09:48 atlas
 +
drwxrwsr-x. 3 root datagrid 512 Nov 13  2015 datagrid
 +
drwxr-xr-x. 3 root root    512 Jan 21 16:10 test/nowiki>
 +
</code>
 +
 +
User storage directories are under the appropriate group directory:
 +
 +
<code>
 +
<nowiki>
 +
[stbc-i4 ~]$ ll /dcache/datagrid/
 +
total 1
 +
drwxr-sr-x. 1094 andrewp datagrid 512 May  4 09:59 andrewp/nowiki>
 +
</code>
 +
 +
= Directory Allocation =
 +
 +
To get a storage directory created for yourself please send a request to <code>stbc-admin@nikhef.nl</code>. Don't forget to include which group the directory should be created in.

Revision as of 10:35, 20 May 2016

Introduction

dCache is filesystem designed for storing large amounts of data. The data is stored amongst many server nodes, accessed through a single virtual filesystem by a variety of methods. The dCache filesystem used on stoomboot can be accessed via on nfs mount point from any of the stoomboot interactive or worker nodes. All file stored in dCache can be found under the mount point /dcache.

Usage

dCache can be used pretty much as a normal file system. Access to files is controlled by the usual unix file permissions and the main difference is that you can not change the contents of a file stored in dCache once the file is written. So for example you can't overwrite a file:

[stbc-i1 /dcache/test]$ echo 123 > test.file
[stbc-i1 /dcache/test]$ echo 1234 > test.file
-bash: test.file: Permission denied

or append to a file:

[stbc-i2 /dcache/test]$ echo 123 > test.file
[stbc-i2 /dcache/test]$ echo 4 >> test.file
-bash: test.file: Permission denied

and when deleting a file that file is always treated as being write protected (even if the writable bit is set):

[stbc-i2 /dcache/test]$ ll test.file
-rw-r--r--. 1 andrewp datagrid 4 May  4 09:57 test.file
[stbc-i2 /dcache/test]$ rm test.file
rm: remove write-protected regular file `test.file'? y

These limitations mean that dCache is not suited for such things as compiling code on, storing large files being actively edited (e.g. word documents) or storing files under a version control system.

Storage Space

The total storage space on the dCache storage nodes is about 550 TB. This space is divided up amongst the various groups using the system and is organised into a directory hierarchy:

<nowiki>

[andrewp@stbc-i4 ~]$ ll /dcache/ total 2 drwxr-xr-x. 4 root alice 512 Apr 1 11:46 alice drwxr-sr-x. 5 root atlas 512 May 4 09:48 atlas drwxrwsr-x. 3 root datagrid 512 Nov 13 2015 datagrid drwxr-xr-x. 3 root root 512 Jan 21 16:10 test/nowiki>

User storage directories are under the appropriate group directory:

<nowiki>

[stbc-i4 ~]$ ll /dcache/datagrid/ total 1 drwxr-sr-x. 1094 andrewp datagrid 512 May 4 09:59 andrewp/nowiki>

Directory Allocation

To get a storage directory created for yourself please send a request to stbc-admin@nikhef.nl. Don't forget to include which group the directory should be created in.