Difference: WmsxExample (1 vs. 18)

find  /tmp/$USER  -print  -exec touch {} \;

Note3: If your home directory is located on AFS, it is advised to source the following shell script before logging on to the Grid:

### Function to export the grid stuff in $HOME to the /tmp/$USER:
exportlocal()
{
    # ... the above stuff comes here ...
}

### Alias to export the grid stuff in $HOME to the /tmp/$USER:
alias  exportlocal  \
# ... the above stuff comes here, but put ' ; \' instead of new line, and continue in next line ...

## Make new .globus directory and export it:
if [[ -d /tmp/$USER/globus ]] ; then chmod -R 755 /tmp/$USER/globus ; fi
rm -rf /tmp/$USER/globus
mkdir -p /tmp/$USER/globus
cp $HOME/.globus/usercert.pem /tmp/$USER/globus/
cp $HOME/.globus/userkey.pem /tmp/$USER/globus/
export X509_USER_CER=/tmp/$USER/globus/usercert.pem
export X509_USER_KEY=/tmp/$USER/globus/userkey.pem
## Make new .glite directory and export it:
if [[ -d /tmp/$USER/glite ]] ; then chmod -R 755 /tmp/$USER/glite ; fi
rm -rf /tmp/$USER/glite
mkdir -p /tmp/$USER/glite
cp -rd $HOME/.glite/vomses /tmp/$USER/glite/
export GLITE_USER_HOME=/tmp/$USER/glite

## Make new .globus directory and export it:
if ( -d /tmp/$USER/globus ) then
    chmod -R 755 /tmp/$USER/globus
endif
rm -rf /tmp/$USER/globus
mkdir -p /tmp/$USER/globus
cp $HOME/.globus/usercert.pem /tmp/$USER/globus/
cp $HOME/.globus/userkey.pem /tmp/$USER/globus/
setenv X509_USER_CER /tmp/$USER/globus/usercert.pem
setenv X509_USER_KEY /tmp/$USER/globus/userkey.pem
## Make new .glite directory and export it:
if ( -d /tmp/$USER/glite ) then
    chmod -R 755 /tmp/$USER/glite
endif
rm -rf /tmp/$USER/glite
mkdir -p /tmp/$USER/glite
cp -rd $HOME/.glite/vomses /tmp/$USER/glite/
setenv GLITE_USER_HOME /tmp/$USER/glite

-- AndrasLaszlo - 16 Apr 2008

Mass submission of independent jobs: parameter scan with short calculations

Mass submission of independent jobs: parameter scan with short (e.g. 1 day) calculations

Mass submission of chained jobs: parameter scan with long calculations

Simple job submission with traditional JDL files

You can find a prepared example for this task here: . To submit the example job, type the following commands.

Get the tarball:

> tar  -xzf  sample.tar.gz
> rm  -f  sample.tar.gz
> cd  sample

Log onto the Grid:

> voms-proxy-init  -voms your_vo
# If your jobs are expected to be long, also get long term authentication:
> myproxy-init  -n

Submit the jobs:

> wmsx-provider.sh  workdir
> wmsx-requestor.sh  -vo your_vo
# If your jobs are expected to be long, also make the WMSX remember your grid password:
> wmsx-requestor.sh  -remembergrid
# If your jobs are expected to be long and you are working on AFS, also make the WMSX remember your AFS password:
> wmsx-requestor.sh  -rememberafs
# If you want to submit many-many jobs, limit the number of concurrently running jobs e.g. to 100:
> wmsx-requestor.sh  -n 100
> wmsx-requestor.sh  -a arg.list

The file arg.list contains a list of jobs (together with its parameters) to be started. Your results shall be retrieved under the directory workdir/out. If your job was interactive (JobType = "Interactive" was set in the JDL file), the file containting StdOut / StdErr shall be updated on the fly.

If you are finished, you can stop WMSX, and destroy your Grid authentications:

> wmsx-requestor.sh  -k
# If myproxy-init was used, i.e. for long term jobs:
> myproxy-destroy
> voms-proxy-destroy

You can find a prepared example for this task here: . To submit the example job, type the following commands. (Similar to the previous example.)

Get the tarball:

> tar  -xzf  chainsample.tar.gz
> rm  -f  chainsample.tar.gz
> cd  chainsample

Log onto the Grid:

> voms-proxy-init  -voms your_vo
# If your jobs are expected to be long, also get long term authentication:
> myproxy-init  -n

> wmsx-provider.sh  workdir
> wmsx-requestor.sh  -vo your_vo
# If your jobs are expected to be long, also make the WMSX remember your grid password:
> wmsx-requestor.sh  -remembergrid
# If your jobs are expected to be long and you are working on AFS, also make the WMSX remember your AFS password:
> wmsx-requestor.sh  -rememberafs
# If you want to submit many-many jobs, limit the number of concurrently running jobs e.g. to 100:
> wmsx-requestor.sh  -n 100
> wmsx-requestor.sh  -a arg.list

> wmsx-requestor.sh  -k
# If myproxy-init was used, i.e. for long term jobs:
> myproxy-destroy
> voms-proxy-destroy

WMSX examples

You can find a prepared example for this task here: . To submit the example job, type the following commands.

Get the tarball:

> tar  -xzf  simplejob.tar.gz
> rm  -f  simplejob.tar.gz
> cd  simplejob

Log onto the Grid:

> voms-proxy-init  -voms your_vo
# If your job is expected to be long, also get long term authentication:
> myproxy-init  -n

Submit the job:

> wmsx-provider.sh  workdir
> wmsx-requestor.sh  -vo your_vo
# If your job is expected to be long, also make the WMSX remember your grid password:
> wmsx-requestor.sh  -remembergrid
# If your job is expected to be long and you are working on AFS, also make the WMSX remember your AFS password:
> wmsx-requestor.sh  -rememberafs
> wmsx-requestor.sh  -j simplejob.jdl  -o workdir/out/StdOut  -r workdir/out/resultdir

Your results shall be retrieved into the directory workdir/out/resultdir, and the file containting StdOut / StdErr shall be workdir/out/StdOut, which is retrieved on the fly, if JobType = "Interactive" was set in your JDL file.

If you are finished, you can stop WMSX, and destroy your Grid authentications:

> wmsx-requestor.sh  -k
# If myproxy-init was used:
> myproxy-destroy
> voms-proxy-destroy

WMSX Examples

Simple job

Mass submission of independent jobs: simple parameter scan

Mass submission of chained jobs: parameter scan with chained jobs

A real-life example

-- AndrasLaszlo - 28 Nov 2007