Click on the row to expand each case.
AMGA oracle
CE and BDII Site on the same node
If you want to install CE and site BDII on the same node you have to install both metapackages (after set repos as reported on the table above):
yum install <CE_metapackage> ig_BDII
where <CE_metapackage> can be:
ig_CE
ig_CE_torque
ig_CE_LSF
ig_CREAM
ig_CREAM_torque
ig_CREAM_LSF
CE and Batch Server configuration (access to log files)
Access to batch system log files
It doesn't matter what kind of deployment you have, batch-system master on a different machine than the CE (lcg-CE, CREAM, TORQUE or LSF) or on the same one, you have to be sure that you provide access to the batch system log files: You must set up a mechanism to transfer accounting logs to the CE:
through NFS (don't forget to set $BATCH_LOG_DIR and $DGAS_ACCT_DIR in <your-site-info.def> configuration file)
through a daily cron job to the directory defined in $BATCH_LOG_DIR and $DGAS_ACCT_DIR in <your-site-info.def> configuration file
CE_torque (multiple CE and single Batch Server)
This configuration requires particulare attention both during the metapackage installation and during the nodetype configuration.
Install the first CE (that will act also as Batch Server) following the usual procedure:
yum install ig_CE_torque
Install the other secondary CEs without batch server software as follows:
yum install ig_CE glite-TORQUE_utils
HLR
As HLR has a “strict” dependency:
mysql = 4.1.20-3.RHEL4.1.el4_6.1
mysql-server = 4.1.20-3.RHEL4.1.el4_6.1
In order to avoid installation/update errors, if you have “protected” your OS repositories, we recommend the installation of yum-versionlock (if not already present on your system), and add the mysql in the list of “locked” rpms:
# yum install yum-versionlock
# cat /etc/yum/pluginconf.d/versionlock.list
mysql-4.1.20-3.RHEL4.1.el4_6.1
mysql-server-4.1.20-3.RHEL4.1.el4_6.1
MON
glite-MON metapackage has not dependency on MySQL rpm packages any more since MySQL server is not necessary on the same node as RGMA server. In case that you want to install MySQL server on the same node, you need to install MySQL rpm packages by hand.
SE StoRM
The two StoRM node-types can be separately installed on different hosts in a cluster configuration or on a single host.
For a cluster with one or more frontends, one or more server gridftp and a backend, further details are available on ” StoRM official documentation - Cluster Configuration”.
Otherwise you can follow the standard installation procedure as described above.
Prerequisites
Please note that StoRM use the ACLs on file and directory to enforce permission. The ACL support is a fundamental feature the file system have to provide. If your StoRM installation works on standard POSIX file system (ext3) please be sure you have the option acl on the corresponding fstab row.
Check in your /etc/fstab, e.g.:
# <file system> <mount point> <type> <options> <dump> <pass>
[...]
/dev/hda1 /storage ext3 defaults,acl 0 1
This is valid for different file system types (ext3, xfs, gpfs and others).
If your storage partition is also the main system partition, e.g. / , you can modify the option without restarting the system using:
mount -o remount /
After that you can configure the profile using ”yaim”:
/opt/glite/yaim/bin/ig_yaim -c -s <your-site-info.def> -n <nodetype>
Where <nodetype> is one of those reported on the Released profiles table (Nodetypes column).
Please note that if your profile hosts more than one nodetype you have to configure them all at once with a command like this:
/opt/glite/yaim/bin/ig_yaim -c -s <your-site-info.def> -n <nodetype1> -n <nodetype2> ...
IMPORTANT: MPI Support
For enabling MPI support please read and set variables as suggested at MPI Customizations
Known Issues: edgusers.conf
Due to a problem on edgusers.conf, for ig-yaim ⇐ 4.0.7-4, you could get the following error when configuring:
INFO: Using locally defined function /opt/glite/yaim/functions/local/config_edgusers
[snip]
DEBUG: Executing... useradd -m -p *NP* -c EDG user -u 152 -g edguser -G infosys -d : edguser
ERROR: User edguser with uid '152' failed to be created
ERROR: Error when creating a user !
You should use the following corrected edgusers.conf - you have to copy it in the /opt/glite/yaim/examples/ directory and reconfigure.
Known Issues: /etc/init.d/tomcat5
Latest version of tomcat5 has a bug on its init script that arises the following error when the service is stopped/started:
/etc/rc.d/init.d/tomcat5: line 196: log_success_msg: command not found
You may fix it simply modifying the first line of /etc/init.d/tomcat5 file as follows:
#!/bin/sh
Click on row to expand each case.
CE LCG
Check if after the configuration the following services are correctly up and running:
service globus-job-manager-marshal status
service globus-gass-cache-marshal status
If ”globus-job-manager-marshal” is down nobody can be authorized on the CE. If ”globus-gass-cache-marshal” is down nobody can be authorized on the WNs.
CE (LCG or CREAM) and BDII Site on the same node
Pay attention to mention both nodetypes on the same command line:
/opt/glite/yaim/bin/ig_yaim -c -s <your-site-info.def> -n <CE_nodetype> -n BDII_site
where <CE_nodetype> can be:
ig_CE
ig_CE_torque
ig_CE_LSF
ig_CREAM
ig_CREAM_torque
ig_CREAM_LSF
CE Torque (multiple CE, single Batch Server)
Set the BATCH_SERVER variable to the hostname of the first CE (the one installed with ig_CE_torque metapackage).
Then configure the first CE as usual:
/opt/glite/yaim/bin/ig_yaim -c -s <your-site-info.def> -n ig_CE_torque
and the others as follows:
/opt/glite/yaim/bin/ig_yaim -c -s <your-site-info.def> -n ig_CE -n glite-TORQUE_utils
Post configuration
1) Export pbs log and grid mapping directories from Batch Server
On the Batch Server CE add a line like the following to /etc/exports for each secondary CE:
/var/spool/pbs <secondary-ce>.<my.domain>(rw,no_root_squash)
/etc/grid-security/gridmapdir <secondary-ce>.<my.domain>(rw,no_root_squash)
Then start NFS and refresh the exported directories with the following command:
service nfs start
exportfs -r
2) Mount pbs log and grid mapping directories on each secondary CEs
To mount those directories on each secondary CE, add a corresponding line in /etc/fstab:
<batch-server-ce>.<my.domain>:/var/spool/pbs /var/spool/pbs nfs rw,defaults 0 0
<batch-server-ce>.<my.domain>:/etc/grid-security/gridmapdir /etc/grid-security/gridmapdir nfs rw,defaults 0 0
Create the directories and mount them:
mkdir -p /var/spool/pbs
mkdir -p /etc/grid-security/gridmapdir
mount /var/spool/pbs
mount /etc/grid-security/gridmapdir
CE CREAM
Due to the bug #56518 BLAH blparser doesn't start after boot of the machine. For the moment you have to manually restart the service with the following commands:
service tomcat5 stop
/opt/glite/etc/init.d/glite-ce-blparser start
service tomcat5 start
HLR
Edit APEL Publisher configuration
Manually edit the file /opt/glite/etc/glite-apel-publisher/publisher-config.xml as see below. This file is not modified by yaim.
<?xml version="1.0" encoding="UTF-8"?>
[...]
ApelConfiguration enableDebugLogging="yes">
<SiteName>**SITE_NAME**</SiteName>
<DBURL>jdbc:mysql://localhost:3306/dgas2apel</DBURL>
<DBUsername>**DGAS_HLR_DB_USER**</DBUsername>
<DBPassword>**DGAS_HLR_DB_PASSWORD**</DBPassword>
[...]
<JoinProcessor publishGlobalUserName="yes" disableJoin="yes">
<Republish>missing</Republish>
</JoinProcessor>
</ApelConfiguration>
Resources registration
After configuration you have to register the resources that have to be accounted on your HLR.
There are two registration procedure: automated and manual. The dgas-support team suggests to use the automatic one, because it is easier and mainly human error free.
In this guide only the automated method will be explained; for the manual method please refer to the ”INFNGRID DGAS 3.4.0 Guide”.
Automated registration procedure
The automated registration procedure requires that the CE you would register is published by a top level BDII server or has a site BDII on top of it. The registration covers both local and grid queues. It also provides to create database internal group and vo ids.
Single CE
In this case ”DGAS_USE_CE_HOSTNAME” variable is not set on the site CE. In order to register all the site resources you only need to register the resources of the CE.
Suppose you have ce01.domain CE. This is the command you have to use:
/opt/glite/sbin/glite-dgas-hlr-bdiiresimport.pl -s "sitebdii.domain" -H "ce01.domain"
Torque case - more CEs, single Batch Master
In this case ”DGAS_USE_CE_HOSTNAME” variable is not set on the site CEs. In order to register all the site resources you only need to register the resources of each CE.
Suppose you have 2 CEs (ce01.domain, ce02.domain) and a single Batch Master. These are the commands you have to use:
/opt/glite/sbin/glite-dgas-hlr-bdiiresimport.pl -s "sitebdii.domain" -H "ce01.domain"
/opt/glite/sbin/glite-dgas-hlr-bdiiresimport.pl -s "sitebdii.domain" -H "ce02.domain"
LSF case - more CEs, single Batch Master
In this case ”DGAS_USE_CE_HOSTNAME” variable has to be set on the site CEs. In order to register all the site resources you have to register the resources of each CE and then the resources of the secondary CEs to the selected CE.
Suppose you have 3 CEs (ce01.domain [set in DGAS_USE_CE_HOSTNAME], ce02.domain, ce03.domain) and a single Batch Master. These are the commands you have to use:
/opt/glite/sbin/glite-dgas-hlr-bdiiresimport.pl -s "sitebdii.domain" -H "ce01.domain"
/opt/glite/sbin/glite-dgas-hlr-bdiiresimport.pl -s "sitebdii.domain" -H "ce02.domain"
/opt/glite/sbin/glite-dgas-hlr-bdiiresimport.pl -s "sitebdii.domain" -H "ce03.domain"
/opt/glite/sbin/glite-dgas-hlr-bdiiresimport.pl -s "sitebdii.domain" -H "ce02.domain" -U "ce01.domain"
/opt/glite/sbin/glite-dgas-hlr-bdiiresimport.pl -s "sitebdii.domain" -H "ce03.domain" -U "ce01.domain"
LB
There is need for extra configuration (special indexes) for certain Virtual Organizations, such as Atlas and CMS, on the LB and WMS services. For details, please see the document ” Specific configuration on gLite WMS and LB 3.1 nodes for each VO”.
As a (tomporary) solution for bug #53714 you should check the existence of, and if not create, the file /opt/glite/etc/LB-super-users containing the DNs of authorized WMSes. For example:
# cat /opt/glite/etc/LB-super-users
/C=IT/O=INFN/OU=Host/L=Padova/CN=prod-wms-01.pd.infn.it
You can retrieve the host certificate subject with the following command:
openssl x509 -in /etc/grid-security/hostcert.pem -noout -subject
If you need in the future to add some other authorized WMSes, edit this file and then restart LB server with the following command:
/opt/glite/etc/init.d/glite-lb-bkserverd restart
IMPORTANT: Known issues
As reported in the bug #36336, there are conflicts between different YAIM functions to configure GLITE_LOCATION_VAR causing that LB services can not be stopped and started by daemon scripts. A workaround is to define GLITE_LOCATION_VAR=/var/glite in the file /opt/glite/yaim/defaults/glite-lb.pre before configuring the LB service.
PX
There is a known issue on ”
/authorized_retrievers/” (look at
9275 and
#35341 bugs). Follow this temporary workaround:
print CONFIG "authorized_retrievers \"\*\"\n";
print CONFIG "authorized_key_retrievers \"\*\"\n";
service myproxy stop
service myproxy start
SE DPM
After configuration remember to manually run the script /etc/cron.monthly/create-default-dirs-DPM.sh as suggested by yaim log.
This script create and set the correct permissions on VO storage directories; it will be run monthly via cron.
SE StoRM
YAIM arguments order
If you decide to configure both StoRM frontend and backend on the same host, please ensure that yaim script '-n' arguments are in the right order (first backend, then frontend):
... -n ig_SE_storm_backend -n ig_SE_storm_frontend ...
Cluster installation
The StoRM profiles can be configured all together on one host or separately in a cluster of hosts (many frontend, one backend). For this non-minimal configuration please refer to the following documents, otherwise use the standard configuration procedure described above:
VOMS
WMS
There is need for extra configuration (special indexes) for certain Virtual Organizations, such as Atlas and CMS, on the LB and WMS services. For details, please see the document ” Specific configuration on gLite WMS and LB 3.1 nodes for each VO”.
IMPORTANT: Known issues
DAG job doesn't work;
Bug #36669: if the user proxy is mapped to a static account instead of a pool account, the job-list-match command would work, but the job-submit command would fail. As a workaround,
only pool accounts should be used. As a configuration example, please have a look into the
/opt/glite/yaim/examples/users.conf file provided by the glite-yaim-core package.
Bug #35357: if only one file is specified in the
OutputSandbox, the method
getOutputFileList of WMProxy
API Python returns an empty list instead of the file as expected.
Bug #36432: the execution of the
/etc/init.d/gLite start command modifies the terminal setting for the special character erase in the following way: before the execution: erase = ^?; after the execution: erase = ^H;
IMPORTANT: At ” WMS management tools” section some scripts are available to deal with Sandbox cleanup issues. Indeed it happens that synchronization between WMS and LB does not work so that LB database entries are cleaned before the corresponding SandboxDirs on WMS are removed. In this case those directories will remain forever. To avoid the SandboxDir saturation you may use the ”clean-sandbox.sh” script (thanks to CNAF guys).
