Using the Storage Elements (_SE) of the Grid

Command line tools

The files are stored on Storage Elements on the Grid. These are computers with very large disk space (in the order of 10TB), located at different sites. Usually these have different physical file administration schemes. Therefore, a logical layer was built on top of them, which makes the file access uniform from the user point of view. To use this facility , use the environmental variable LCG_CATALOG_TYPE:

export LCG_CATALOG_TYPE=lfc

(for bash), or

setenv LCG_CATALOG_TYPE lfc

(for tcsh).

The Logical File Name (lfn) is resolved by the Logical File Catalogue (lfc) server. It can be thought of as the partitional table of a world-wide hard disk. You have to specify this database server, e.g.:

export LFC_HOST=grid155.kfki.hu

(for bash), or

setenv LFC_HOST grid155.kfki.hu

(for tcsh).

The name of the lfc server can be found out by:

> lcg-infosites  --vo your_vo  lfc (The lcg-infosites command is also useful for finding out other information. We refer to its man page.)

Now you can refer to files with their logical file names, which are site-independent. You do not have to care afterwords that to which site your files are actually written. The logical file names have to be unique.A very convenient way to generate them is to imitate the UNIX type file naming scheme. With the command lfc-mkdir one can make logical directories, like:

> lfc-mkdir /grid/hungrid/user_name/some_directory (All logical file names begin with /grid/virtual_organization. If you are a new user, the directory /grid/virtual_organization/user_name will not exist initially: you have to create it first, if you want an own directory.)

There are various commands to manage logical file names, all beginning with lfc-, like lfc-chmod, lfc-getacl, lfc-setacl, lfc-chown, lfc-ln, lfc-rename, lfc-ls, lfc-rm, lfc-mkdir etc. Refer to the man pages of these commands, but the usage of these should be self-explanatory, imitating their UNIX analogue.

As the given logical directory is ready, one can copy files into it. If you want to store a file in the Grid storage infrastructure permanently, copying a file onto a storage element is not enough: you have to register the file in the logical file catalogue. It is done like:

> lcg-cr  --vo hungrid  -d grid143.kfki.hu  -l lfn:/grid/hungrid/user_name/destionation/test_file.txt  \
                                                                         file:/home/user_name/source/test_file.txt

(Here, the -d destination_Storage_Element is optional, and is used to force the destination Storage Element to be destination_Storage_Element; otherwise the file may be copied to a Storage Element anywhere in the world.) This command will copy and register your file onto the Grid storage system. Once your file is there, you can get a copy of it with lcg-cp e.g. as an input for a job:

> lcg-cp  --vo hungrid  lfn:/grid/hungrid/user_name/source/test_file.txt  \
                                          file:/home/user_name/destination/test_file.txt

This command gets a copy of your file on the storage system to some local area.

There are various commands to manage the stored files (e.g. unregister and delete them, or create replicas), all beginning with lcg-, like lcg-rep, lcg-aa, lcg-rf, lcg-cp, lcg-cr, lcg-la, lcg-uf, lcg-del, lcg-lg, lcg-lr, lcg-gt, lcg-ra etc. Refer to the man pages of these commands, but most of them are self-explanatory.

It is possible to spare always typeing --vo your_vo, by setting the environmental variable LCG_GFAL_VO:

export LCG_GFAL_VO=your_vo

(for bash), or

setenv LCG_GFAL_VO your_vo

(for tcsh).

A list of Storage Elements may be obtained by:

> lcg-infosites  --vo your_vo  se (See man pages of lcg-infosites.)

Note1: unfortunately, I encountered that the lcg-cr action often fails (this may be remedied in future, or may be already solved). Therefore, it is convenient to put an lcg-cr command into a checker loop, like:

#!/bin/sh
RESULT=1
TRIES=0
while (( $RESULT != 0 )) ; do
    lcg-cr --vo cms -d grid100.kfki.hu \
        -l lfn:/grid/cms/alaszlo/destintation/test_file.txt \
            file:/afs/kfki.hu/home/alaszlo/source/test_file.txt
    RESULT=$?
    TRIES=$(($TRIES+1))
    if (( $TRIES >= 5 )) ; then echo "lcg-cr failed. Giving up
after 5 tries." ; exit 1 ; fi
    if (( $RESULT != 0 )) ; then sleep 1m ; fi
done

This precaution may also be recommended for lcg-cp or other lcg- commands.

Note2: Never restrict the lfc permissions of a file or a directory in such a way, that the group (VO) does not have write permissions. There is a very simple reason for it. It is not really you, who does the actual copying onto a storage element: some program copies it on behalf of you, which is assigned some more or less random local user identifier, which is mapped to your Grid user identifier temporarily. This is a rather questionable way of manging user identifiers, but that is the way how it is implemented in the current version. This fact has rahter dangerous implications. If you restrict the permissions, such that the group does not have write permissions, then at an other occasion (when your Grid user identifier is mapped to another local user identifier), you cannot write your files. Meanwhile, an other user (who is accidentally given that particular local user identifier) may still have write permission to your file, despite of your precaution. As a consequence, the lfc permissions should not be restricted in such a way that the group does not have write permissions. But this also has a consequence: practically anyone has write permissions to your data in your group, plus you have write permissions to the data of others in the group. Let's hope that this conceptional mistake will be corrected soon.

Read/Write C routines (the GFAL)

The Grid File Access Library (GFAL) is a collection of read/write C routines. It is imitating the C file read/write routines. We refer to the man pages of gfal.

-- AndrasLaszlo - 10 Jul 2009