Difference between revisions of "DCache"
Line 3: | Line 3: | ||
[http://www.dcache.org dCache] is filesystem designed for storing large amounts of data. The data is stored amongst many server nodes, accessed through a single virtual filesystem by a variety of methods. The dCache filesystem used on stoomboot can be accessed via on nfs mount point from any of the stoomboot interactive or worker nodes. All files stored in dCache can be found under the mount point <code>/dcache</code>. | [http://www.dcache.org dCache] is filesystem designed for storing large amounts of data. The data is stored amongst many server nodes, accessed through a single virtual filesystem by a variety of methods. The dCache filesystem used on stoomboot can be accessed via on nfs mount point from any of the stoomboot interactive or worker nodes. All files stored in dCache can be found under the mount point <code>/dcache</code>. | ||
− | There are no backups of made of the dcache file system. Issues | + | There are no backups of made of the dcache file system. |
+ | |||
+ | Issues and problems with the file system should be sent to <code>stbc-admin[AT]nikhef[DOT]nl</code>. | ||
+ | |||
+ | To get a personal directory within dCache created and start using dCache see [[#Directory Allocation]]. | ||
= Usage = | = Usage = |
Revision as of 09:53, 23 May 2016
Introduction
dCache is filesystem designed for storing large amounts of data. The data is stored amongst many server nodes, accessed through a single virtual filesystem by a variety of methods. The dCache filesystem used on stoomboot can be accessed via on nfs mount point from any of the stoomboot interactive or worker nodes. All files stored in dCache can be found under the mount point /dcache
.
There are no backups of made of the dcache file system.
Issues and problems with the file system should be sent to stbc-admin[AT]nikhef[DOT]nl
.
To get a personal directory within dCache created and start using dCache see #Directory Allocation.
Usage
dCache can be used pretty much as a normal file system. Access to files is controlled by the usual unix file permissions and the main difference is that you can not change the contents of a file stored in dCache once the file is written. So for example you can't overwrite a file:
[stbc-i1 /dcache/test]$ echo 123 > test.file
[stbc-i1 /dcache/test]$ echo 1234 > test.file
-bash: test.file: Permission denied
or append to a file:
[stbc-i2 /dcache/test]$ echo 123 > test.file
[stbc-i2 /dcache/test]$ echo 4 >> test.file
-bash: test.file: Permission denied
and when deleting a file that file is always treated as being write protected (even if the writable bit is set):
[stbc-i2 /dcache/test]$ ll test.file
-rw-r--r--. 1 andrewp datagrid 4 May 4 09:57 test.file
[stbc-i2 /dcache/test]$ rm test.file
rm: remove write-protected regular file `test.file'? y
These limitations mean that dCache is not suited for such things as compiling code on, storing large files being actively edited (e.g. word documents) or storing files under a version control system.
Directory Allocation
To get a storage directory created for yourself please send a request to stbc-admin[AT]nikhef[DOT]nl
. Don't forget to include which group you are working with as that's used to determine were in the storage hierarchy your directory is created.
Backups
NONE. There are no backups of files storage on the system.
Storage Space Organisation
The total storage space on the dCache storage nodes is about 550 TB. Each group has a directory under the main dcache directory:
[stbc-i4 ~]$ ll /dcache/
total 2
drwxr-xr-x. 4 root alice 512 Apr 1 11:46 alice
drwxr-sr-x. 5 root atlas 512 May 4 09:48 atlas
drwxrwsr-x. 3 root datagrid 512 Nov 13 2015 datagrid
User storage directories are under the appropriate group directory:
[stbc-i4 ~]$ ll /dcache/datagrid/
total 1
drwxr-sr-x. 1094 andrewp datagrid 512 May 4 09:59 andrewp
Storage space is assigned to each group under there group directory so all people within a group share the same group storage space allocation.
TO-DO: df doesn't give useful results for dcache so a web page with current data usage will be setup