General Support

We have two listservs for support, one for announcements from the Tier-2 admins to the users, and one for complaints and questions from the users to the admins. Every user of the Tier-2 should subscribe to the announcement listserv.

Announcements from admins to users: cmst2 at physics dot ucsd dot edu*
*The archive for this list is here: cmst2 archive

Questions and complaints from users to admins: t2support at physics dot ucsd dot edu

Login Platforms

The Tier-2 center supports multiple computers for interactive login. Those are called uaf-X.t2.ucsd.edu with X running from 1 to 6. The numbers 3,4,5,6 are modern 8ways with loads of memory. The 1,2 are older machines. I'd stay away from 1 if I was you.

To get login access, send email with your ssh key and hypernews account name to t2support. To get write access to dCache into your own /store/user area, send email with your hypernews account name and the output from "voms-proxy-info" to t2support.

We support 1TB of space in /store/user for every person from UCSB, UCR, UCSD who is in CMS.

dedicated groups on uaf

To share directories on the uaf between multiple people in a group, we define groups and use ACLs. If you need this functionality, do the following:

• Request a group from t2support
• Once you have a group, you need the following commands to make a directory and define it as group writeable.

mkdir bla getfacl bla setfacl -R -m g:cms1:rwx bla setfacl -R -d -m g:cms1:rwx getfacl bla 

This sets the default for all files in this directory, and does so recursively. Only the person who wns the file or directory can execute the command on it.

Send email to t2support if you have problems.

Software Deployment

codefs.t2.ucsd.edu is used to centrally deploy the CMS software and tools that provide most of necessary CMSSW and grid environment for the user level physics analysis and data operation.

The CMSSW is deployed via Tier-2 software distribution across the whole USCMS tier-2 (and some tier-3 sites). In general only standard release of CMSSW will be deployed. The analysis and test based on pre-release will not be supported unless the specific request is made or the deployment of the software is available under the standard procedure.

To make a desktop machine similar to the tier-2 interactive analysis machine, for example uaf-1.t2.ucsd.edu, the codefs.t2.ucsd.edu:/code/osgcode needs to be mounted to the local directory /code/osgcode

CMSSW Environment

Access to CMSSW repository

      export CMS_PATH=/code/osgcode/cmssoft/cms        

      export SCRAM_ARCH=slc4_ia32_gcc345        

      source ${CMS_PATH}/cmsset_default.sh       

       or         

      setenv CMS_PATH /code/osgcode/cmssoft/cms        

      setenv SCRAM_ARCH slc4_ia32_gcc345        

      source ${CMS_PATH}/cmsset_default.csh 

Create CMSSW project area and setp environment

       cd your-work-directory        

       scramv1 project CMSSW CMSSW_1_6_12        

       eval `scramv1 runtime -(c)sh` 

If you don't like waiting for your code to compile, try out compiling in parallel on our 8ways:

 scramv1 b -j 8 

Grid Environment and Tools

Make sure you have .globus and .glite directories in the home directory. In the .glite, there is a file, vomses, needs to be there. You can get one from /code/osgcode/ucsdt2/etc.

• Setup Glite

Before initiating the glite environment, please make sure no other grid environment exists, especially by checking no VDT environment is sourced (the VDT environment is set up with "source /setup.(c)sh").

To setup the glite environment,

       source /code/osgcode/ucsdt2/gLite/etc/profile.d/grid_env.(c)sh 

The glite environment should allow you to get the proxy and proper role in order to run your grid jobs

       voms-proxy-init -valid 120:00 --voms cms:/cms/uscms/Role=cmsuser 

As an aside, in case you need to install gLite client at your home institution, i.e. anywhere outside the login environment of the Tier-2 at UCSD, you can do so following the instructions at FkwInstallgLite .

• Setup Condor and VDT

In uaf machines, the condor environment is already in the PATH of uaf machines. Combining glite and condor environment, you can send grid jobs (e.g. crab jobs) via condor_g.

If VDT is chosen to bring the grid environment to your analysis instead of glite in the uaf machines,

       source /date/tmp/vdt/setup.(c)sh  

Never mix VDT with glite environment.

• Setup CRAB

There are primarily two submission methods to send crab jobs, condor_g and glitecoll, which determines how crab is set up and used.

     1. setup CMSSW environment as described above      

     2. setup glite or condor environment as described above      

     3. source /code/osgcode/ucsdt2/Crab/etc/crab.(c)sh 

To check which crab version is actually used by "ls -l /code/osgcode/ucsdt2/Crab/etc/crab.(c)sh"

To publish the datasets to the DBS (here is an example of local DBS deployed at UCSD), in the [USER] section of crab.cfg, following configuration needs to be added

      publish_data = 1        

      publish_data_name = "Njet_test1"       

      dbs_url_for_publication = http://ming.ucsd.edu:8080/DBS/servlet/DBSServlet 

To run analysis to the local DBS (other than the global CMS DBS), in the [USER] section of crab.cfg, following configuration needs to be added

      dbs_url = http://ming.ucsd.edu:8080/DBS/servlet/DBSServlet 

• CRAB Server

A crab server is deployed at UCSD, which you can use for your crab submission. Following is the configuration in the [CRAB] section of crab.cfg file to specify the crab server and scheduler.

      server_name = ucsd 

      scheduler = glitecoll         

Late binding based Crabserver can be used as defined here Essentially:

      server_name = ucsd 

      scheduler =  glidein    

Job Monitoring

Locally submitted jobs to the T2 Condor batch system can be monitored using:

• Overview of jobs at UCSD Tier-2
• Detailed job monitoring for UCSD Tier-2

Jobs submitted to the Grid via Crabserver can be found at:

• http://glidein-mon.t2.ucsd.edu:8080/dashboard/.

Note: In some cases one need to have the grid certificate loaded into the browser

Moving data to UCSD

We encourage anybody to make data replication requests via the PhEDEx? pages. If you make a request, James Letts and fkw receive an email. One of them will approve the request as long as there is disk space at the Tier-2 available. When they approve it, you receive an email back acknowledging the approved request.

To keep track of all the data at the UCSD Tier-2, we have developed a simple accounting system. For this to work, you need to pick an account you want to charge your request to. This is done by adding the following to the comment field when making the PhEDEx? request:

|| acc = ucsb || 

The above would charge the request to the UCSB account. An account is an arbitrary string. It might be easiest if you simply pick one of the accounts that already exist in the accounting system.

Absolute path in the dcache system at UCSD

The interactive login nodes at UCSD allow you to do an ls on the directories in dCache for both the official as well as user data:

#for official data: ls /pnfs/t2.ucsd.edu/data3/cms/phedex/store #for private user data: ls /pnfs/t2.ucsd.edu/data4/cms/store/user 

To get the host and port for srm and dcap, please check out http://dcache.ucsd.edu
This page has a wealth of monitoring information in addition to listing the host and port for srm, dccp, etc.

MC Production and Local Scope DBS

• Production User Portal

For running user-level production, the URL to the portal is https://yuan.ucsd.edu/production_request. The x509 certificate needs to be stored in the web browser to enable the access. Normally you import PKCS#12 file of the certificate to the browser. If you can't find the PKCS#12 file when you first received the x509 certificate, you can use following command to create one (named MyCert? .p12) based on your x509 certficate the key

     openssl pkcs12 -export -in usercert.pem -inkey userkey.pem -out MyCert.p12 -name "my x509 cert" 

The portal allows to run small- and mid-scale production based on ProdAgent? and GlideinWMS system with full detector simulation and reconstruction. The MC production will be run on USCMS Tier-2 and a few Tier-3 sites. The output will be stored at UCSD storage system and published at local DBS deployed at UCSD.

The data discovery of the local DBS is http://ming.ucsd.edu/data_discovery. The instance to be published is "dbs_2009". To access the datasets published at this local DBS by crab, the crab configuration needs to refer to the local DBS interface, http://ming.ucsd.edu:8080/DBS1/servlet/DBSServlet.

• Local DBS

The local DBS is implemented to support data publication via Crab or ProdAgent? . For Crab, the publication or access the local DBS can be set up by adding following to the [USER] section in the crab.cfg

For old version of DBS (DBS_1_0_8)

        dbs_url_for_publication = http://ming.ucsd.edu:8080/DBS/servlet/DBSServlet       

        dbs_url = http://ming.ucsd.edu:8080/DBS/servlet/DBSServlet 

For new version of DBS (DBS_2_0_4)

        dbs_url_for_publication = http://ming.ucsd.edu:8080/DBS1/servlet/DBSServlet       

        dbs_url = http://ming.ucsd.edu:8080/DBS1/servlet/DBSServlet 

The data discovery of the local DBS is

http://ming.ucsd.edu/data_discovery 

To look at the datasets bookept by DBS, you need to choose "instance" in the selection manual, "dbs_2008" corresponds to the old DBS, "dbs_2009" corresponds to the new DBS.

-- HaifengPi - 02 Sep 2008

-- SanjayPadhi - 2009/03/08

-- FkW - 2009/09/07