WmsxExample < RmiGrid

---++ WMSX examples

_Note1_: There is an installed copy of _WMSX_ software on the [[http://www.grid.kfki.hu/twiki/bin/view/KFKIAFS/AfsSoftwares#WMSX_The_Workload_Management_Sys][KFKI AFS]]. To activate it, follow the instructions of that homepage.

_Note2_: The WMSX can work without the User Interface environment. For this, you only need an installed copy of WMSX on your computer, the usual =~/.globus/usercert.pem=, =~/.globus/userkey.pem= files and the =~/.glite= directory as on any User Interface machine, plus you need to provide the =~/.globus/certificates= directory, which can be obtained from any User Interface machine from =/etc/grid-security/certificates=. These files and directories can also be relocated by the =X509_USER_CERT=, =X509_USER_KEY=, =GLITE_USER_HOME= and =X509_CERT_DIR= environmental variables.

_Note3_: If your home directory is located on AFS, it is advised to source the following shell script before logging on to the Grid:
<verbatim>
## Make new .globus directory and export it:
chmod  -R  755  /tmp/$USER/globus  >&/dev/null
rm  -rf  /tmp/$USER/globus
mkdir  -p  /tmp/$USER/globus
cp  $HOME/.globus/usercert.pem  /tmp/$USER/globus/
cp  $HOME/.globus/userkey.pem  /tmp/$USER/globus/
cp  $HOME/.globus/certificates  /tmp/$USER/globus/
export  X509_USER_CER=/tmp/$USER/globus/usercert.pem
export  X509_USER_KEY=/tmp/$USER/globus/userkey.pem
export  X509_CERT_DIR=/tmp/$USER/globus/certificates
## Make new .glite directory and export it:
chmod  -R  755  /tmp/$USER/glite  >&/dev/null
rm  -rf  /tmp/$USER/glite
mkdir  -p  /tmp/$USER/glite
cp  -rd  $HOME/.glite/* /tmp/$USER/glite/
export  GLITE_USER_HOME=/tmp/$USER/glite
</verbatim> (for _bash_), or <verbatim>
## Make new .globus directory and export it:
chmod  -R  755  /tmp/$USER/globus  >&/dev/null
rm  -rf  /tmp/$USER/globus
mkdir  -p  /tmp/$USER/globus
cp  $HOME/.globus/usercert.pem  /tmp/$USER/globus/
cp  $HOME/.globus/userkey.pem  /tmp/$USER/globus/
cp  $HOME/.globus/certificates  /tmp/$USER/globus/
setenv  X509_USER_CER  /tmp/$USER/globus/usercert.pem
setenv  X509_USER_KEY  /tmp/$USER/globus/userkey.pem
setenv  X509_CERT_DIR  /tmp/$USER/globus/certificates
## Make new .glite directory and export it:
chmod  -R  755  /tmp/$USER/glite  >&/dev/null
rm  -rf  /tmp/$USER/glite
mkdir  -p  /tmp/$USER/glite
cp  -rd  $HOME/.glite/* /tmp/$USER/glite/
setenv  GLITE_USER_HOME  /tmp/$USER/glite
</verbatim> (for _tcsh_). This ensures that the necessary files are accessible even after your AFS tickets expire (as they are copied to some local directory). You could also put these inside a shell function or alias in your =.bashrc= or =.tcshrc=, like:
<verbatim>
### Function to export the grid stuff in $HOME to the /tmp/$USER:
exportlocal()
{
    # ... the above stuff comes here ...
}
</verbatim> (for _bash_), or
<verbatim>
### Alias to export the grid stuff in $HOME to the /tmp/$USER:
alias  exportlocal  \
# ... the above stuff comes here, but put ' ; \' instead of new line, and continue in next line ...
</verbatim> (for _tcsh_).

---+++ Simple job submission with traditional JDL files

Job submission with =glite-wms-job-submit= is not very straightforward, as the generated job identifier has to be stored, and then the job output retrieval directory has to be resolved by this identifier. To save you from this typing exercise, the WMSX is capable of managing the output retrieval directory for simple jobs.

You can find a prepared example for this task here: [[%ATTACHURL%/simplejob.tar.gz][simplejob.tar.gz]]. To submit the example job, type the following commands.

Get the tarball:
<verbatim>
> tar  -xzf  simplejob.tar.gz
> rm  -f  simplejob.tar.gz
> cd  simplejob
</verbatim>

Log onto the Grid:
<verbatim>
> glite-voms-proxy-init  -voms your_vo
# If your job is expected to be long, also get long term authentication:
> myproxy-init  -d  -n
</verbatim>

Submit the job:
<verbatim>
> wmsx-provider.sh  workdir
> wmsx-requestor.sh  -vo your_vo
# If your job is expected to be long, also make the WMSX remember your grid password:
> wmsx-requestor.sh  -remembergrid
# If your job is expected to be long and you are working on AFS, also make the WMSX remember your AFS password:
> wmsx-requestor.sh  -rememberafs
> wmsx-requestor.sh  -j simplejob.jdl  -r workdir/out/resultdir
# Or, if JobType = "Interactive" was set in the JDL file, say instead:
> wmsx-requestor.sh  -j simplejob.jdl  -r workdir/out/resultdir  -o workdir/out/StdOutFile
</verbatim>

Your results shall be retrieved into the directory =workdir/out/resultdir=. If your job was interactive ( =JobType = "Interactive"= was set in the JDL file), the file containting !StdOut / !StdErr shall be =workdir/out/StdOutFile=, which shall be updated on the fly. Remark: if the =JobType = "Interactive"= was set, the fields =StdInput=, =StdOutput= and =StdError= cannot be set in the JDL file.

If you are finished, you can stop WMSX, and destroy your Grid authentications:
<verbatim>
> wmsx-requestor.sh  -k
# If myproxy-init was used, i.e. for long term jobs:
> myproxy-destroy  -d
> glite-voms-proxy-destroy
</verbatim>

An educative example for simple interactive job submission may be found here: [[%ATTACHURL%/interactive.tar.gz][interactive.tar.gz]]. This works in a similar way as the =simplejob.tar.gz= example.

---+++ Mass submission of independent jobs: parameter scan with short calculations

WMSX is a very convenient tool for mass submission of short, e.g. of about maximum 1 day lifetime, independent jobs. Possible application: parameter scan studies with relatively short calculations. (The job lifetime is limited by the typical allowed maximal job lifetimes, assigned by the Grid sites. A limit of 3 day running time is typical, so a maximum 1 day job is fine.) WMSX automatically manages output retrieval issues and the limiting of the number of concurrently running jobs. For this purpose, the jobs are described with a slightly extended JDL language, which is preprocessed by WMSX. If =JobType = "Interactive"= is set, the file containing !StdOut / !StdErr is updated on the fly, so you can see what your job is actually doing. A pre-execution script helps to prepare inputs, and a post-execution script helps to process outputs of the job.

You can find a prepared example for this task here: [[%ATTACHURL%/sample.tar.gz][sample.tar.gz]]. To submit the example job, type the following commands.

Get the tarball:
<verbatim>
> tar  -xzf  sample.tar.gz
> rm  -f  sample.tar.gz
> cd  sample
</verbatim>

Log onto the Grid:
<verbatim>
> glite-voms-proxy-init  -voms your_vo
# If your jobs are expected to be long, also get long term authentication:
> myproxy-init  -d  -n
</verbatim>

Submit the jobs:
<verbatim>
> wmsx-provider.sh  workdir
> wmsx-requestor.sh  -vo your_vo
# If your jobs are expected to be long, also make the WMSX remember your grid password:
> wmsx-requestor.sh  -remembergrid
# If your jobs are expected to be long and you are working on AFS, also make the WMSX remember your AFS password:
> wmsx-requestor.sh  -rememberafs
# If you want to submit many-many jobs, limit the number of concurrently running jobs e.g. to 100:
> wmsx-requestor.sh  -n 100
> wmsx-requestor.sh  -a arg.list
</verbatim>

The file =arg.list= contains a list of jobs (together with its parameters) to be started. Your results shall be retrieved under the directory =workdir/out=. If your job was interactive ( =JobType = "Interactive"= was set in the JDL file), the file containting !StdOut / !StdErr shall be updated on the fly.

If you are finished, you can stop WMSX, and destroy your Grid authentications:
<verbatim>
> wmsx-requestor.sh  -k
# If myproxy-init was used, i.e. for long term jobs:
> myproxy-destroy  -d
> glite-voms-proxy-destroy
</verbatim>

---+++ Mass submission of chained jobs: parameter scan with long calculations

The mass submission has an important feature: each job can decide whether a launch of a further job is needed. This decision is done by the post-execution script: if it returns with exitcode 1, the chain script is invoked. The lines of the standard output of the chain script is interpreted by WMSX as lines in the =arg.list= file (see previous example), so new jobs are launched. In this way, jobs can be chained (and the chains can also fork, as a job can also launch multiple jobs). In this way, a parameter study with long term calculations can be performed. The long term calculations are preformed by the job chains. A good example is numerical solution of partial differential equations: multiple chains are launched (possibly with different initial conditions), and each chain is imitating a long term job (the solution of the partial differential equation with a given initial condition), split up into subsequent shorter term jobs.

You can find a prepared example for this task here: [[%ATTACHURL%/chainsample.tar.gz][chainsample.tar.gz]]. To submit the example job, type the following commands. (Similar to the previous example.)

Get the tarball:
<verbatim>
> tar  -xzf  chainsample.tar.gz
> rm  -f  chainsample.tar.gz
> cd  chainsample
</verbatim>

Log onto the Grid:
<verbatim>
> glite-voms-proxy-init  -voms your_vo
# If your jobs are expected to be long, also get long term authentication:
> myproxy-init  -d  -n
</verbatim>

Submit the jobs:
<verbatim>
> wmsx-provider.sh  workdir
> wmsx-requestor.sh  -vo your_vo
# If your jobs are expected to be long, also make the WMSX remember your grid password:
> wmsx-requestor.sh  -remembergrid
# If your jobs are expected to be long and you are working on AFS, also make the WMSX remember your AFS password:
> wmsx-requestor.sh  -rememberafs
# If you want to submit many-many jobs, limit the number of concurrently running jobs e.g. to 100:
> wmsx-requestor.sh  -n 100
> wmsx-requestor.sh  -a arg.list
</verbatim>

The file =arg.list= contains a list of jobs (together with its parameters) to be started. Your results shall be retrieved under the directory =workdir/out=. If your job was interactive ( =JobType = "Interactive"= was set in the JDL file), the file containting !StdOut / !StdErr shall be updated on the fly.

If you are finished, you can stop WMSX, and destroy your Grid authentications:
<verbatim>
> wmsx-requestor.sh  -k
# If myproxy-init was used, i.e. for long term jobs:
> myproxy-destroy  -d
> glite-voms-proxy-destroy
</verbatim>

-- Main.AndrasLaszlo - 04 Nov 2008
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
gz	chainsample.tar.gz	r1	manage	1.6 K	2008-04-16 - 15:42	AndrasLaszlo	Chainsample example
gz	interactive.tar.gz	r1	manage	0.5 K	2008-04-16 - 15:41	AndrasLaszlo	Interactive example
gz	sample.tar.gz	r1	manage	1.3 K	2008-04-16 - 15:41	AndrasLaszlo	Sample example
gz	simplejob.tar.gz	r1	manage	0.5 K	2008-04-16 - 15:40	AndrasLaszlo	Simplejob example
This topic: RmiGrid > WebHome > WmsxSoftware > WmsxExample
Topic revision: r16 - 2009-07-10 - AndrasLaszlo
Copyright &© by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback