Difference between revisions of "Localgroupdisk nikhef"

From Atlas Wiki
Jump to navigation Jump to search
Line 17: Line 17:
  
 
=== moving datasets to it ===
 
=== moving datasets to it ===
 +
If the datasets already exist on the grid, you can simply request the replication of them to the NIKHEF-ELPROD_LOCALGROUPDISK using the [[http://panda.cern.ch/server/pandamon/query?mode=ddm_req DaTRI interface]].
 +
 +
Placing dataset replication requests in DaTRI has some requirements:
 +
<ul>
 +
  <li>You have a valid grid certificate loaded in the browser.  When you apply a Dutch grid certification issued by TERENA, you should have the certificate properly loaded into the browser.  If not, following [[ this instruction.]]
 +
  <li>You have to register yourself in DaTRI service.  You can do it from [[http://panda.cern.ch/server/pandamon/query?mode=ddm_user here]].  If you are not sure whether or not this is done before, check your registration status [[http://panda.cern.ch/server/pandamon/query?mode=ddm_user&action=info here]].
 +
</ul>
 +
 +
Once you have everything setup, you can request data replications the [[http://panda.cern.ch/server/pandamon/query?mode=ddm_req DaTRI interface]].
 +
 +
The "Request Parameters" are mandatory.  The "Data Pattern" takes into account the wildcard symbol ("*") and the container symbol ("/" at the end of the name) .  For the "Destination Sites", chose "NL" cloud and the "NIKHEF-ELPROD_LOCALGROUPDISK".
 +
 +
For the "Control Parameters" simply put in "data analysis" into the "justification".
 +
 +
You will noticed that the data replication is not started immediately after you place the request.  The request needs an additional approval procedure.  If the dataset size of a request is smaller than 500 gigabytes, they are approved automatically if your grid certificate is already registered in the NL group in ATLAS (i.e. you are able to do voms-proxy-init -voms atlas:/atlas/nl).  For the requests with dataset size larger than 500 gigabytes, they have to be approved manually by the disk managers.  For the moment, the managers are Hurng-Chun.Lee@cern.ch and dgeerts@nikhef.nl.  Although the managers will be notified for every requests requiring additional manual approval, it would be appreciated if you could contact the managers in addition for quicker reactions.
  
 
=== creating your own datasets on it ===
 
=== creating your own datasets on it ===
  
 
=== accessing datasets/files on it ===
 
=== accessing datasets/files on it ===

Revision as of 09:34, 28 September 2012

What is the local group grid storage at NIKHEF?

The grid storage for the local group has been setup at NIKHEF. In terms of the DQ2 terminology, the DQ2 site name is called "NIKHEF-ELPROD_LOCALGROUPDISK". It has 20 terabytes of disk space and used previously already by few groups to host D3PD Ntuples.

Unlike other grid storage spaces (that are centrally managed by the ATLAS central grid operation), we are our own manager of the usage of the local group disk. We decide ourselves how to share the space among sub-groups (TOP, SUSY, HIGGS, etc.) at NIKHEF.

How much space left and who is using it?

The following plot shows the disk usage status in past 30 days. The distance between the green and the red curves can be read as the free space remaining on this space.

http://bourricot.cern.ch/dq2/media/fig/NIKHEF-ELPROD_LOCALGROUPDISK_30.png

More accounting information can be found on this [page].

Lists of datasets on the local group disk can be found on this [page]. From the "User" column you know who replicated/created the datasets.

How to use it?

moving datasets to it

If the datasets already exist on the grid, you can simply request the replication of them to the NIKHEF-ELPROD_LOCALGROUPDISK using the [DaTRI interface].

Placing dataset replication requests in DaTRI has some requirements:

  • You have a valid grid certificate loaded in the browser. When you apply a Dutch grid certification issued by TERENA, you should have the certificate properly loaded into the browser. If not, following this instruction.
  • You have to register yourself in DaTRI service. You can do it from [here]. If you are not sure whether or not this is done before, check your registration status [here].

Once you have everything setup, you can request data replications the [DaTRI interface].

The "Request Parameters" are mandatory. The "Data Pattern" takes into account the wildcard symbol ("*") and the container symbol ("/" at the end of the name) . For the "Destination Sites", chose "NL" cloud and the "NIKHEF-ELPROD_LOCALGROUPDISK".

For the "Control Parameters" simply put in "data analysis" into the "justification".

You will noticed that the data replication is not started immediately after you place the request. The request needs an additional approval procedure. If the dataset size of a request is smaller than 500 gigabytes, they are approved automatically if your grid certificate is already registered in the NL group in ATLAS (i.e. you are able to do voms-proxy-init -voms atlas:/atlas/nl). For the requests with dataset size larger than 500 gigabytes, they have to be approved manually by the disk managers. For the moment, the managers are Hurng-Chun.Lee@cern.ch and dgeerts@nikhef.nl. Although the managers will be notified for every requests requiring additional manual approval, it would be appreciated if you could contact the managers in addition for quicker reactions.

creating your own datasets on it

accessing datasets/files on it