GridPP Log Book

Introduction and Instructions
This Log Book will be used to finalise the GridPP2 work programme and to monitor progress on a quarterly basis (replacing quarterly reports). A Web available version should be maintained by the relevant area manager. 

The existing deliverables should be partitioned into two levels: 

-	Level-1 deliverables that will be included in the GridPP2 ProjectMap, and which will be monitored on a quarterly basis. For each of your main objectives we anticipate Level-1 milestones at about 6 monthly intervals. 
-	Level-2 deliverables that are at a more detailed level. These will be reviewed on a regular basis  by the work-area managers to guide progress and anticipate problems but only the Level-1 deliverables will be tracked in the ProjectMap.

The Level-1 deliverables should be kept up to date in this document and the GridPP Project Manager should be notified of changes to Level-1 deliverables by the completion of a ChangeForm as in GridPP1. 
The Level-2 deliverables should be available as a link from this document but may be in whatever format is most suitable to the particular area (it may be a Link to an EGEE document, for example, or an Experiment specific work plan).

Quarterly reporting will be performed by the work-area manager, defined in the table below, by updating this Log Book.

Work Area	Area Manager	PMB Coordinator	Reviewer(s)	
Atlas	R. Jones	

Applications Coordinator		
CMS	P. Hobson			
LHCb	G. Patrick			
BaBar	R. Barlow			
CDF	R. St. Denis			
DO	P. Ratoff			
PhenoGrid	N. Glover			
UKQCD	R. Kenway?			
Portal	D. Britton			
LHC Deployment	TBD	

Middleware  Coordinator		
MetaData	T. Doyle			
Data Storage	J. Jensen			
Security and VO services	A. McNab			
Information Services	R. Middleton			
Networking	P. Clarke			
Work Load Management	D. Colling			
Tier-1 Services	A. Sansum	Production   Manager		
Tier-2 Hardware Support	S. Lloyd			
Dissemination	S. Pearce	CB Chair		
GridPP Log Book

1. Project

Name:  CMS 
Manager: P. R. Hobson

2. High Level Objectives and Level-1 Deliverables

Objective I
(a)	Distributed Workload Management for CMS
(b)	 To enable CMS physicists to use the deployed Grid for event reconstruction and analysis 
(c)	The CMS collaboration 
(d)	Major contribution to the design, implementation and testing of the CMS workload management system. Metric is a combination of software, documentation and demonstration of functionality, using Tier 1 and Tier 2 centres, and the adoption by CMS of significant parts of the sub-systems developed and evaluated within the GRIDPP project.
(e)	 R1, R2, R3, R4, R6, R10, R13, R14, R15, R19, R20, R21, R31, R38
(f)	 Level-1 Deliverables:
a.	3.4.11, 05Q4: Deployment over the Grid of a CMS job runtime environment. This system is designed to insulate the CMS physicist from the complexities of defining the particular run-time environment needed for particular classes of job. Metric is a software release with associated documentation describing the software, the tests that were performed and the results of those tests.
b.	3.4.12, 05Q4: Development and evaluation of a job preparation and splitting system to enable a CMS physicist to define a task and data sets and then submit appropriate sub-jobs to the Grid. Metric is a software release with associated documentation describing the software, the tests that were performed and the results of those tests.
c.	3.4.13, 06Q2: Job monitoring, tracking and bookkeeping system released. Metric is a software release, associated documentation and the successful tracking of at least 300 jobs submitted over the Grid.
d.	3.4.14, 06Q4: System to provide job output retrieval and publication using a replication service. Metric is a software release, associated documentation and the successful retrieval and publication of at least 100 jobs submitted over the Grid.
e.	3.4.15, 07Q3: Release of a workload management system meeting the needs of CMS for data taking. Metric is a software release, associated documentation and the workload management of jobs submitted at a rate commensurate with full luminosity data taking over the Grid.

Objective II
(a)	Data Management
(b)	To deploy an efficient and reliable data storage, caching and replication environment 
(c)	The CMS Collaboration 
(d)	Reliably delivering data in Physics Analysis Exercises up to LHC startup 
(e)	 R1, R2, R3, R4, R6, R10, R13, R14, R15, R19, R20, R21, R31, R38
(f)	 Level-1 Deliverables:
a.	3.4.5, 05Q2: Analysis of local and wide-area data management approaches. Metric is the contribution to the relevant section of the CMS Computing TDR.
b.	3.4.6, 05Q4: Deployment of high-speed bulk data replication service at CMS T1 / T2. Metric: system achieves TDR specification.
c.	3.4.7, 06Q2: Provision of a data distribution system for Monte Carlo data to all CMS T1/T2 sites. Metric is a software release plus a document demonstrating that data was delivered to all functional T1/T2 sites
d.	3.4.8, 05Q4: Deployment and testing of streaming data management service at CMS T0 / T1. Metric is contributing data used for the Physics TDR analyses.
e.	3.4.9, 06Q4: Planning the enhanced data management system. Metric is a Requirements Specification Document. 
f.	3.4.10, 07Q3: Data management system ready for LHC startup. Metric is system sign-off by CMS.

Objective III
(a)	Monitoring / metadata management for CMS
(b)	To provide CMS physics analysis application software with metadata monitoring. 
(c)	The CMS collaboration 
(d)	Evaluation of RGMA for metadata monitoring and software contribution to the production of Physics analysis metadata for DC06.
(e)	R1, R2, R3, R4, R6, R10, R14, R15, R20, R31, R38
(f)	Level-1 Deliverables:
a.	3.4.1, 05Q1: Performance analysis of BOSS/RGMA on a production LCG Grid. Metric is the testing of the performance of RGMA using at least 5 simultaneous LCG resources, and the release of a performance analysis document.
b.	3.4.2, 05Q2: Metadata requirements analysis published in CMS computing TDR. Metric is the contribution to the relevant section of the CMS Computing TDR.
c.	3.4.3, 06Q2: Provision of metadata enabling Physics analysis using data from DC06. Metric is contribution to CMS software release.

Objective IV
(a)	Large-scale data challenges for CMS
(b)	To test, at full LHC data rate the production software suite of CMS on a computing Grid prior to the LHC startup.
(c)	The CMS collaboration 
(d)	To contribute significantly to the international CMS Data Challenges
(e)	R4, R6, R10, R13, R14, R15, R19, R20, R21, R22, R27, R28, R29, R31, R35, R37, R38, R40
(f)	Level-1 Deliverables:
a.	3.4.4a, 05Q3: UK participation in testing CMS production computing model at a level comparable to other CMS T1 and associated T2 sites. Metric is production of test data sets, and numbers of data sets replicated, at the appropriate UK fraction of the CMS computing infrastructure.
b.	3.4.4b, 06Q3: UK participation in CMS DC06 at a level comparable to other CMS T1 and associated T2 sites. Metric is production and replication of DC06 data sets at the appropriate UK fraction of the CMS computing infrastructure.
c.	3.4.4c, 07Q2: Final-scale deployment and testing of the CMS Grid computing system worldwide, in preparation for LHC start up. Metric: System meets data taking requirements for both scale and throughput as defined in Computing TDR.
Current status of L1 Deliverables
GRIDPP Project Map Deliverable	Title	Original Due Date	Current Due Date (if different)	Status	Associated L2 deliverables	
D3.4.1	BOSS/RGMA performance analysis study	1/4/05		Done	3.1, 3.2, 3.3	
D3.4.2	Metadata requirements analysis	1/7/05		Done	2.3, 3.4, 3.5	
D3.4.3	Production Metadata delivery system	1/7/06		Partially completed	3.6 (plus 3.7 & 3.8 which were retired)	
D3.4.4	CMS Data Challenges and Deployment	On-going		Done	2.9,4.1,4.2,4.3,4.4,4.5	
D3.4.5	Local and Wide Area data management study	1/7/05		Done	2.1,2.2	
D3.4.6	Deployment of high speed bulk data replication services	1/1/06	
31/3/06	Done	2.3,2.5,2.6	
D3.4.7	Data distribution system for Monte Carlo	1/7/06		Done	2.4	
D3.4.8	Streaming data management service	1/1/06	31/3/06	Done	2.7	
D3.4.9	Enhanced data management system planning	1/1/07		Done	2.8	
D3.4.10	Final data management system	31/8/07		Done	2.9	
D3.4.11	CMS job runtime environment system	1/1/06		Done	1.1,1.2,1.3	
D3.4.12	Job preparation and splitting system	1/1/06	
31/9/06	Done	1.4,1.5,1.6	
D3.4.13	Job Monitoring, tracking and bookkeeping system	1/7/06	  
    31/9/06	Done	1.7,1.8	
D3.4.14	Job output retrieval and publication	1/1/07		Done	1.9,1.10	
D3.4.15	WLMS for CMS	31/8/07		Done	1.1 – 1.10	

3. Level-2 Deliverables
This is currently maintained as a web-based MS-WORD document located at 
http://people.brunel.ac.uk/~eestprh/GRIDPP/Logbook/CMS_Logbook_L2_deliverables_V9.doc
4. Commentary
This section is filled in incrementally quarter by quarter as a means of documenting particular successes, failures, issues, problems and their resolution. It should be brief, but should provided a coherent record of the evolution of the work. It will be reviewed each quarter by the chair of the relevant board and by the Project Manager. It may be a hyper-link to an external document such as an EGEE quarterly report or a collaboration report. However, it should state explicitly which level-1 deliverables have been completed in the quarter and should comment explicitly on any level-1 deliverables that are overdue. In this case, a modified date should be agreed and a Change form should be sent to the Project Manager.

2004 Q2 Comments

2004 Q3 Comments: 
D3.1 completed. Loss of H Tallini from Imperial College.

T. Barrass (Bristol, 50% CMS WP8)
•	Working on the transition from the SRB based solution to data transfer (T0 to T1) which was used in DC04 to a new unified approach. 
•	I s now working on the development of PhEDEx version 2, which will be stress tested this Autumn.
•	T Barrass is one of the technical project leaders in CMS for this project
•	Contributed to a number of talks and presentations which have been accepted for the CHEP04 conference.

B. MacEvoy (Imperial College, 50% CMS WP8)

Barry MacEvoy has been working the GROSS analysis framework which
has now been released and is being tested by the Italians, the Americans
and the Taiwanese. This project is documented at

http://www.hep.ph.ic.ac.uk/e-science/projects/downloads.html

Barry has also started to look at the web services architecture employed
by the American Grid Analysis Environment (GAE) which is going to provide
a significant contribution to the CMS/ARDA. 

Contributed to a number of papers which will be presented at CHEP04

Barry has also maintained the LCG nodes (for CMS use) and maintains the
Imperial e-Science web pages.

ANO (Imperial College, to replace Hugh Tallini)
A replacement has been offered and accepted the post. The replacement will start work on 1 October 2004, subject to Work Permit not being delayed. Three months effort has been lost in recruitment delay.

J. J. Nebrensky (Brunel, 50% CMS WP8 + 30% HEFCE contribution)

•	Major contribution to enabling Brunel to join LCG2 in September 2004.
•	Is working on getting our LCG2 test cluster operational so that we can continue to evaluate the performance of RGMA and BOSS with real production jobs (setting up MON box etc.)
•	RGMA performance on LGC2 Grid abstract submitted to IEEE Rome conference (October 2004) and accepted for oral presentation.
•	Evaluating the performance of latest LCG2 version of RGMA on the LCG2 grid.

2004 Q4 Comments: 
Deliverable D2.1 has been completed
Y-J Zhang appointed on 1/11/2004 to replace H Tallini at Imperial College.

T. Barrass (Bristol, 50% FTE)
A significant contribution to the design and  development of the CMS data distribution infrastructure (now named 
PhEDEx): http://cms-project-phedex.web.cern.ch/cms-project-phedex/ Major functional components of this infrastructure are now deployed at RAL, and have exhibited sustained transfer rates of 15 MBps from CERN Castor (at the same time as several other Tier 1s). 

Within CMS a small working group (including Barrass) are examining the experiment's use of metadata with a view to rapidly creating a CMS dataset bookkeeping system. The first draft use case document for this project has been produced and Barrass has made CMS aware of the UK GridPP metadata group, and their recent summary use case document.
B. MacEvoy (Imperial College, 50% FTE) and Y-J Zhang (Imperial College, 100% FTE)
Our major activity (with  unfunded effort from David Colling) over this quarter is the unification of the Italian (Grape/Crab) and British (Gross) job submission mechanisms. This has entailed detailed reviews of both products and a great deal of work. This effort culminated in a meeting in Bologna on Friday the 17th of December where a plan for the long term unification was developed. 

Much of the core functionality of Gross will be transferred into a new version of BOSS (which may or may not keep that name). Imperial College is taking reponsibility for this re-engineering of BOSS in collaboration with the current BOSS developer (Claudio Grandi). We will also provide  Python as well as C++ APIs for the new BOSS. The Italian groups will then produce the a user interface written in Python which composes the tasks to be submitted to the new version of BOSS. This will include all interaction with the meta-data services etc. The new version of BOSS will become the default for MC production as well as for data analysis.

J. J. Nebrensky (Brunel, 50%FTE + 30% FTE HEFCE contribution)
Presented an Oral paper at IEEE NSS/MIC/SNPS and RTSD Conference (October 16-22, Rome Italy) on the performance of RGMA on the LCG2 test-bed. Worked on ensuring that RGMA was correctly deployed and configured on the LCG2 test-bed. Progress has been made towards deliverable D3.2 with 30000 messages transferred on the LCG2 testbed by 1/12/2004. Has taken part in discussion regarding CMS metadata management strategy.

2005 Q1 Comments
NOTE : L1 Deliverable III(a) is delayed, see report of Nebrensky
T. Barrass (Bristol, 100% FTE)
Towards Deliverables 2.3, 3.4, 3.5

The CMS data management group continue to lay the groundwork for a CMS  dataset bookkeeping service, with 'phone meetings every Tuesday. The  time-scale for producing a first prototype is now short (order a  month); discussions are focussing on the best hierarchical organization  of the data. Some prototype APIs and schemas have been proposed; also  proposed are methods for distributing some notional bookkeeping  database. Some members of the group, however, are keen to agree on a  simple categorization of data first and deploy a simple prototype  before thinking about more sophisticated, research issues.  The metadata use cases work has been somewhat subsumed into a larger  scale analysis of CMS workflows for the TDR which I've initiated, and  which is seen as important to the collaboration. This is a large task,  and in its early stages yet; it should take form over the next couple  of weeks. 
D2.2 Complete (ongoing process)
Data is now available at RAL for CMS analysis use, MC and DST production. Some small amount of work needs to undertaken to document the process required to run analysis jobs locally.
B. MacEvoy (Imperial College, 50% FTE) and Y-J Zhang (Imperial College, 100% FTE)
A final release of GROSS (V0_4_0 available from http://www.hep.ph.ic.ac.uk/e-science/projects/downloads.html) was produced. This was tested by a small group of physicists (CMS Higgs group) for processing the Monte Carlo DST as part of the analysis for the Physics TDR.  CRAB is now taking over as the sole user interface to the  LCG (as planned at the Bologna meeting).  The design of the new version of BOSS that incorporates many of the  elements of GROSS has been developed during this quarter and a note  produced. Work to install PhEdEX at Imperial to enable data transfer has been  successful (assisted by Tim Barrass).  Members of the group are contributing to the production of the Computing  TDR. 
J. J. Nebrensky (Brunel, 100%FTE)
Although futher work on the performance of RGMA on the LCG production Grid has taken place, and a report on the results drafted, a very significant fraction of Nebrensky’s time has been taken up with installing and getting to work properly recent roll-outs of new versions of LCG at Brunel. This work, which has only recently resulted in a stable system from which RGMA can be properly evaluated, has delayed the production of deliverable III(a) by about 4 weeks. We intend to release the performance evaluation document in early May 2005. Note that this risk (the dependency of the work on fully functional LCG Grid) was identified in advance.
Nebrensky has been in contact with Barrass on the current state of play with regard to CMS metadata and has attended a number of meetings related to the discussion of metadata for the forthcoming Computing TDR.
2005 Q2 Comments
T. Barrass (Bristol, 100% FTE)
Towards Deliverables 2.3, 3.4, 3.5
B. MacEvoy (Imperial College, 50% FTE) and Y-J Zhang (Imperial College, 100% FTE)
Significant contributing to the CMS Computing  TDR. 
J. J. Nebrensky (Brunel, 100%FTE)
Major work has been to attempt to complete the RGMA testing using LCG. A great deal of effort has gone into making RGMA work across LCG with the version upgrade. A contribution was made to both the LCG Computing Grid Technical Design Report and the CMS Computing TDR.
An extended abstract was submitted by Nebrensky (with a number of UK co-authors including MacEvoy) to the 2005 IEEE Nuclear Science Symposium (Puerto Rico, October, 2005). 

2005 Q3 Comments
T. Barrass (Bristol, 100% FTE)
Over the last quarter I have been focusing on getting RAL working as a  Tier-1 for CMS. This involves, for me: deploying a PhEDEx service to take  part in LCG SC3; deploying PubDB instances for Production and SC3 use.  I have also, however, kept up my activities in CMS wide projects at a  maintenance level. This means that I have actively answered general  PhEDEx development and deployment questions, taken part in  administrating PhEDEx for CMS (acting on requests for datasets;  managing the integration of new nodes to the network).  Running CMS software at RAL has been severely hampered this quarter.  After some intensive work the principle problem has been identified as  a buggy access library for dCache; a temporary workround is now in  place, and RAL is beginning to take part in SC3 for CMS. I have also  spent a great deal of time working with Jens Rehn and Anzar Afaq on  testing the CMS publication tool CMSGlide with specific reference to  it's coupling to PhEDEx, and with Nicola de Fillipis on developments of  the equivalent ValidationTools tool.  
I gave a talk on PhEDEx at the AHM2005 in Nottingham (http://www.allhands.org.uk/programme/index.html) and helped on the  GridPP stand. I have spent some time creating general PhEDEx posters  (A0, A1), the former of which was presented by Daniele Bonacorsi at the  Frontier Science 2005 conference at Milan  (http://frontierscience.mib.infn.it/home.php?id=12) and which is  available at  /afs/cern.ch/cms/aprom/phedex/Documents/PhEDEx-general-poster.ai or  PhEDEx-general-poster-20050901.pdf. I also put significant effort into  editing the final versions of two papers for this year's ACAT ("Towards  the operation of INFN Tier-1 for CMS: lessons learned from CMS Data  Challenge (DC04)" Bonacorsi, on behalf of ... and "The CMS analysis  chain in a distributed environment" de Fillipis as primary author.
B. MacEvoy (Imperial College, 50% FTE) and Y-J Zhang (Imperial College, 100% FTE)
During this quarter the funded effort of Barry MavEvoy was delivered by David Colling (with half of David's GridPP funded WMS work being delivered by Barry).  
During this quarter activity has continued on the development of  BOSS  version 4. This differs from previous versions of BOSS in several ways,  most notably in the ways that it handles collections of jobs. BOSS will be  used as the basis of job submission for MC production as well as analysis  with tools such as CRAB and ASAP building upon it. This is progress  towards Level 1 deliverable I.a. Within Imperial this work has been led by Yong-Jun Zhang. Yong attended the CMS WM workshop 14-15 July, 2005 - Padova, Italy.  http://cmsdoc.cern.ch/cms/cpt/Computing/Technical/workshops/wm/jul2005/   Gave a talk about BOSS (Batch Object Submission System) project in this  meeting. Took part in the 14th GridPP Collaboration Meeting, University of  Birmingham, 6-7 September 2005 . Has been focusing on developing the BOSS project  http://cmsdoc.cern.ch/cms/cpt/Computing/Technical/subproj/BOSS.html   BOSS is a CMS joint project between Imperial College London and  Bologna(Italy) university. It is a Grid based tool to provide the means to  log and track all user jobs. . Imperial College joined this project to develop a new kernel with new  architecture.  By many discussion and modification, the XML form is adopted to develop  the new kernel. . Some new functionality(with concept task/job/program) have been defined  and implemented with C++. . The new architecture has have a good shape and start to integrate with  the old components. . The new BOSS with new architecture and additional functionality is  scheduled to be released on November.   The ability to analyse data relies on more than just a framework in which  to submit jobs. It requires that sites be kept up to date and functioning,  that the data be at sites where it can be used and on many other factors.  In order to monitor how well the CMS computing system is providing this  information a an analysis task has been set up within the integration  task. David Colling has been taking an active role in this task. While  this task is new it can be considered to be a contribution to the data and  service challenges and so is a contribution to deliverable IV.a (and b).  
CMS MC production was also run over LCG from Imperial College in this  quarter. This was a time consuming occupation however, eventually all  parts of the MC production chain were achieved over the Grid. This effort  is also a contribution to IV.a. This effort was carried out by David  Colling and others not funded by GridPP.
J. J. Nebrensky (Brunel, 100%FTE)
With the demise of RGMA within CMS, we have re-aligned Nebrensky’s work to contribute to L1 Objective IV, deliverables 1a and 1b. The final work on RGMA in this quarter is purely associated with preparing the paper and poster for the IEEE NSS conference in October.
Maintaining Brunel cluster and upgrading to LCG 2.6.0. Testing of new cluster hardware with LHC@home. Preparing for the installation of SRM at Brunel in preparation for CMS SCA-06.
Technical contributions to the tender specification for the new Brunel Cluster (SRIF2 funded) which will contribute resources to LCG in 2006. 
Attended GRIDPP14 and was a Panel Member. Attending GRIDPP TB-Support monthly telephone meetings. Attending the London Tier-2 Technical Meetings. 
Upgraded BOSS/R-GMA test framework (hardware and software) to use LCG 2.6.0 
Acceptance of abstract for NSS '05: N14-207: (14:30)  Performance of R-GMA for Monitoring Grid Jobs for CMS  Data Production R. Byrom, D. J. Colling, S. M. Fisher, C. Grandi, P. R. Hobson, P.  Kyberd, B. MacEvoy, J. J. Nebrensky, S. Traylen http://www.nss-mic.org/2005/program/ListProgram.asp?session=N14 . Nebrensky is taking the lead role in writing the paper and preparing the poster which he will present at the NSS meeting in 2005 Q4.
Assisted Grid Ireland by providing LCFG configuration and update files. 
2005 Q4 Comments
OVERALL SUMMARY OF GRIDPP FUNDED ACTIVITIES
In this quarter the activity has been focusing on preparation for the CMS part of LHC Service Challenge 4 which will take place in 2006. Considerable effort has been made towards completing the new version 4 of the job submission and monitoring tool BOSS, although this is slightly delayed with respect to our original estimate of end year 2005. Some contribution at RAL Tier-1 and from Imperial College at London Tier-2 to was made to the Service Challenge 3 (2005), considerable efforts being expended due to the fragility of the CMS software environment. Work on developing PhEDEx continues with improvements to the file routing mechanism in this quarter. At Brunel significant CPU resources have finally been provided to LCG, and with the arrival of our new Tier-2 support staff in November Brunel is now concentrating on joining Imperial College as a CMS resource for Service Challenge 4 in 2006. The Brunel Tier-2 farm has contributed to running CMS jobs in this quarter. GRIDPP funded staff have attended IEEE NSS/MIC (Puerto Rico) to present GRIDPP funded work on RGMA testing, and a number of abstracts relating to GRIDPP funded activities have been accepted for oral and poster presentations at CHEP06. Four staff-months of unfunded effort have been provided to the project.

Report on activities of C Munro (PPARC funded e-Science student contributing to CMS computing) added here for completeness.
Munro is involved with a project which provides an overview of CMS jobs running on the Grid. The monitor provides a global view of resource usage and highlights any potential problems. Grid related information is gathered from R-GMA with CMS specific data and runtime information gathered from MonaLisa.  The data is gathered into a single database and a Web-based user interface is provided so that users can perform detailed queries using different parameters such as the user, site or dataset. See www-asap.cern.ch/dashboard

He is also working on enabling interactive analysis for CMS physicists using Proof.  The ARDA Grid Prototype (ASAP) already allows users to process large amounts of data in batch on the Grid.  The next step is to allow users to easily analyse this data. As much of this data is in the root format (and as the new CMS EDM is directly root-readable) Proof (Parallel Root Facility) provides a natural solution. ASAP has been extended to automatically (or on demand) move the users data to the new Proof Test Cluster at CERN to provide users with a fast interactive service. Four members of the CMS Higgs group have been introduced to the service. See www-asap.cern.ch/asap_proof.php
T. Barrass (Bristol, 100% FTE)
SUMMARY
Barrass spent most of the last quarter making a dedicated effort to get RAL up and running for CMS analysis. Now CMS physicists at RAL can use data at RAL to develop analysis applications, and CMS physicists worldwide can submit their jobs to run on RAL-hosted data via the Grid. RAL was able to take part in the CMS part of LCG SC3 to a limited extent. It demonstrated reasonable transfer rates sustained for a few days, but was unable to host analysis jobs because of ongoing blocking problems with publishing. This has not been a trivial process. Much of the CMS computing infrastructure is very fragile; some difficulties were reported to the CMS computing integration meeting on 24 November (http://agenda.cern.ch/askArchive.php?base=agenda&categ=a056557&id=a056557s1t2/transparencies). Most sites are suffering from the same fragility. Barrass has started to formalise the interaction between CMS and RAL by producing a weekly report that summarises ops status (archived at https://uimon.cern.ch/twiki/bin/view/CMS/RutherfordOperationsLogs). This has been a very useful mechanism for highlighting problems at RAL that block progress, as these are sometimes not clear without an overview of an experiment's overall processes. These weekly reports also go to a UK CMS computing list (uk-cms-software@jiscmail.ac.uk) set up by Simon Metson to which all CMS UK members are invited to subscribe. This is already proving a useful resource.
To further improve communication Barrass set up a CMS computing Wiki at https://uimon.cern.ch/twiki/bin/view/CMS/RutherfordLabs as a means to bootstrap the sharing of experience we have in the UK. This, and the mailing list, are being widely publicised now that analysis and other tasks are possible at RAL.
An abstract has been accepted as a poster for CHEP 2006 (Mumbai) Techniques for high-throughput, reliable transfer systems: break-down of PhEDEx design, Barras et al
Deliverable D2.3 (Contribution to CMS data management system) – L1 IIb and IIIb
Barras has been active in the development of PhEDEx, being continuously involved in the design process, and recently contributing significant code changes for core components (file routing mechanism). He has also attended the CMS data management meetings weekly to follow development of the CMS Dataset Bookkeeping and Location Systems (first trial started on 15 Dec 2005). 
Deliverable D 2.4 (Sustained transfer in and out of RAL for analysis and MC production) – L1 IIc
The CMS data allocated to RAL is currently 11TB, which doesn't allow for a truly sustained transfer activity. During SC3 we did see rates reaching 70MBs-1, and typically sustain rates of 30 MB s-1 for days, comparable to many peers. MC activity has been low this quarter. We did demonstrate that we can transfer to as well as from CERN, and to and from a number of peer T1 sites.
Deliverable D2.6 (Rollout of transfer mechanism to Tier 2 centres) – L1 IIb
Rollout of new release of PhEDEx to Imperial has been delayed because of manpower limitations. Rollout of PhEDEx to Brunel is now foreseen, but no date is defined yet for the first deployment (the schedule is dictated by Brunel hardware commissioning).
Deliverable D2.7 (Tuning of local transfer infrastructure) – L1 IId
Tuning of local infrastructure is only just becoming possible now because of the long time reqiured to get semi-stable operations up and running.
Deliverable D2.8 (Enhanced data management system planning) – L1 IIe
Barrass has started to formalise processes at RAL to form a stable base for operations. He has:
1) Shifted all operations software and tools to a widely available NFS mounted space owned by a CMS RAL operators UNIX group, so that operations are manageable if I'm not available (see /rutherford/cms-soft1/RALOperations, owned by group 'phedex').
2) Begun to provide formal documentation of RAL ops processes (see https://uimon.cern.ch/twiki/bin/view/CMS/RALOperations).
3) Developed tools to automate various operations workflows. The publication of data available at RAL is a very problematic process in CMS; I have developed tools that automate the process and allow an operations manager to more easily diagnose faults, errors etc, and retry mass publication. Use of these tools has already highlighted (quantitatively) possible inefficiencies in access to dCache disk and tape resources at RAL 
4) Started work on ops workflows that will become essential in the next 6 months. Currently CMS (at least) lacks a mechanism for scheduling hot data availability on the dCache disk resource, for example.
CONCERNS
Barrass states that "it is almost impossible for me to take part in any systems development tasks because of the time taken to manage CMS activities at RAL".
Rollout of PhEDEx to Tier2 other than Brunel and Imperial is not being considered because of time limitations. Extended tests of transfers between T1 and T2s have not been attempted because of time and manpower limitations.
B. MacEvoy (Imperial College, 50% FTE) and Y-J Zhang (Imperial College, 100% FTE)
SUMMARY
During this quarter the funded effort of Barry MavEvoy was delivered by David Colling (with half of David's GridPP funded WMS work being delivered by Barry). 
An number of abstracts have been accepted for CHEP 2006 (Mumbai) :
Evolution of BOSS, a tool for job submission and tracking, Grandi, Colling et al
A Statistical Analysis of Job Performance within LCG Grid., Aggarwal, Colling, Macevoy, der Aa
Failure Management in the London Distributed Tier 2, Aggarwal et al
Deliverables (D1.6, 1.8 and 1.9, CMS Job submission and monitoring) – L1 Ib, Id, Ie
BOSS is a CMS computing tool for bookkeeping, monitoring and job  submission. It is originally developed by Bologna (Italy) group, until  version 3. Imperial College joined the BOSS development team at end of the  last year and is taking part in develop BOSS version 4.
BOSS is being used in two scenarios, physics analysis and Monte Carlo data production. In the analysis scenario, BOSS collaborates with CRAB which  is a distributed analysis tool and can understand datasets. In a production scenario, BOSS will collaborate with MCPS. BOSS works across the Grid, jobs are submitted from User Interface (UI) side, and run on Worker Nodes (WN). On the UI side, BOSS provides a CLI to  the user, an API to other tools and a data base for bookkeeping. On the WN side, BOSS provides functionalities to chain the programs and to register the real  time information to a remote data base. The BOSS release schedule is available from http://cmsdoc.cern.ch/cms/cpt/Computing/Technical/subproj/BOSS.html. 
Information on the current pre-release version of BOSS-4 (code, documentation and some information on how to run it) is available from http://www.hep.ph.ic.ac.uk/~yjzhang/boss/index.html.
What we have done in this quarter are
(1) To standardize the information representation, we adopted XML for both internal and external data.
(2) To modulate the components, we have separated the data components from  the backend components, standardized their interfaces and written them into classes. 
(3) To implement one of the new features of BOSS version 4, multi job layers  submission, functions have been generalized to understand the XML file  structure. 
(4) To separate the functionalities of normal user from administrator, we  have written a prototype CLI and API  for both the user and administrator 
(5) To simplify the Schema definition, we have written data base schema  into an XML file, from which an administrator can easily understand and update  the schema.
Deliverable D4.1 (Contribution to testing prototype CMS production model) – L1 IVa
Working towards the an integrated analysis (lead by D Colling) over the Grid and on the monitoring of CMS jobs. Imperial has been a working analysis site for more than 6 months. We have Run CMS MC production over the LCG and contributed to the first phase of SC3 (this ended at the beginning of this quarter). The team approach to running CMS MC production is run as a rota with each person taking a batch of assignments as they come in. That person will run all the stages of that production i.e. the different assignments for CMKIN, OSCAR, reconstruction, DST production for these samples. However when the next batch come in a few days later the next person takes them up. This means that we have distributed expertise and that it is not a huge job for any one person. So far (in Q3 and Q4 of 2005) the Imperial cluster in London-Tier2 has generated 90000 events, run 518500 through the simulation and produced 10000 fully digitized.
We have started to produce the official user documentation for Grid based analysis for CMS. This is being done in the form of a DocBook.
CONCERNS
The release version of BOSS version 4 is now scheduled to be provided in February 2006. This is delayed with respect to our original estimate of end Q4, 2005, but this is not yet considered to be a significant delay.
J. J. Nebrensky (Brunel, 100%FTE)
SUMMARY
In this quarter Nebrensky made the transition from developing Brunel as a Tier-2 site on LCG to contributing to our four Level 1 objective, specifically L1 deliverable IV.b (UK participation in CMS DC06 at a level comparable to other CMS T1 and associated T2 sites). 
The first month of this quarter was spent dealing with the fallout of an unexplained failure of the power supplies on nearly 30% of our machines. It was suspected that this was connected with the extensive redevelopment of the Brunel campus, which has lead to problems with the mains supply. The machines were repaired and line monitoring equipment was installed. The problem, our work to identify its source and the remedial steps which have been taken delayed the commissioning of the main cluster by over two months. In November our new GRIDPP funded (50% FTE) Tier-2 manager took up his post and Neberensky has transferred his experience over that month.

This plot from GOC (GStat: 20:30:56 12/19/05 GMT) for the production cluster Argo shows that in early November Brunel finally provided significant CPU resources to LCG. This acheivement was in major part due to the work of Neberensky (plus unfunded effort from Kyberd). The increases in CPU from mid-November onwards are due to the work carried out by our Tier-2 support person (Duncan Rand).
Neberensky also contributed further information on configuration and testing of RGMA under LCG 2-6-0 (see web pages) and information on running two CE at one site to the GOC Wiki.
He attended the IEEE NSS/MIC conference in Puerto Rico in October and presented “Performance of R-GMA for Monitoring Grid Jobs for CMS Data Production”, R Byrom et al (POSTER). He wrote a paper for the IEEE Conference Record, which is available from http://people.brunel.ac.uk/~eestprh/GRIDPP/Documents/NSS05CNF.pdf.
He contributed to an abstract which has been accepted for CHEP 2006 : Failure Management in the London Distributed Tier 2, Aggarwal et al (with Kyberd, Neberensky and Rand from Brunel)
Deliverable D4.2 (Contribution to SC06) – L1 IVb
Since mid November Nebrensky has been working closely with Barrass and Colling to get the necessary software components (PhEDEx, pubdb …) installed prior to installing the CMS production software suite. He is increasing our local storage capacity (SE) to ~ 4 Tb. Our Tier-2 manager (Duncan Rand) has also contributed unfunded (i.e. HEFCE funded) effort to this task. In the last month of this quarter several hundred CMS jobs were submitted and run on both of the Brunel clusters (Argo and Green-Stripe).

CONCERNS
GRIDPP’s concern at the lack of contribution of Nebresnsky  to CMS in Q3 and part of Q4 has now been dealt with as we believe that with the arrival of our GRIDPP part-funded Tier-2 support that Neberensky has now transferred all of his responsibilities for the management of the cluster and can once again make a valuable contribution to CMS-specific deliverables in this part of the GRIDPP project. It should be noted that Nebrensky has made a very valuable contribution to the commissioning of the Brunel Tier-2 computing farms.

We have delays to three L1 deliverables (GRIDPP Map 3.4.6, 3.4.8 and 3.4.12). This is due to delays in getting stable operations at UK Tier-2 sites other than Imperial College, and a three month delay in releasing the version 4 of BOSS. Our current estimate is that the delay is of order one project quarter from the original delivery date.
2006 Q1 Comments
OVERALL SUMMARY OF GRIDPP FUNDED ACTIVITIES
The new version of BOSS (V4) has been committed to the CVS repository and its public release is imminent. A critical piece of CMS middleware (PhEDEx) is now working at Imperial College and being configured at Brunel for the London Tier2 contribution to CMS computing. Current activity is strongly focused towards the forthcoming major tests of the whole CMS computing model later in 2006.
T. Barrass (Bristol, 100% FTE)
SUMMARY
I have continued weekly visits to RAL, and continue to send weekly summaries of CMS status to RAL computing management.
I have helped form closer links between CMS UK computing and UK physicists and RAL computing. I have formed plans to manage changes to CMS storage at RAL in the UK over the next year, encompassing reduced disk storage resources and stress testing and acceptance of Castor for UK operations. At RAL I have also engaged in discussion and study of CMS job efficiency on RAL worker nodes in collaboration with Matt Hodges, our CMS RAL liaison.
Deliverable D2.3 (Contribution to CMS data management system) – L1 IIb and IIIb
I remain closely involved with PhEDEx as a lead developer. Recently I have made a significant contribution to new file routing and transfer prioritisation algorithms, and have devised and undertaken a series of acceptance tests for these. The new version of PhEDEx-- which incorporates global use of FTS as the low level file transfer tool-- looks good for release on a month timescale.
I travelled to CHEP06, presenting a PhEDEx technical architecture poster and as author of paper for a general PhEDEx talk. I regularly attend CMS technical and integration meetings, and attended the RAL Atlas Data Store meeting as a CMS representative.
I have continued weekly visits to RAL, and continue to send weekly summaries of CMS status to RAL computing management.
As part of the CMS integration task I formed closer links with the GridPP storage group, sharing information and experience about SRM v2, and the operation of PhEDEx and CMS' requirements of storage. It also helps CMS keep tabs on the development and performance of FTS.
Deliverable D2.6 (Rollout of transfer mechanism to Tier 2 centres) – L1 IIb
In progress with London T2 sites as first adopters. Encountering site-specific problems with CMS SW installation; storage provision. Delayed partly by inability of CMS SW to read from DPM.
Deliverable D2.7 (Tuning of local transfer infrastructure) – L1 IId
This deliverable has been completed this quarter. The CMS switch to FTS means that the management load for CMS staff will be significantly reduced. Good results have been seen in GridPP storage group tests in preparation for SC4 (SC06). As a result of the UK actively promoting this change CMS has now delegated control to FTS admins for managing tuning, and the FTS network is becoming established (~6 UK sites) and offers tuning controls. The WLCG people in the UK are currently running weekly stress tests on FTS to get used to running/tuning these systems; their experience is documented. A new PhEDEx release integrating PhEDEx with FTS is imminent, in accordance with the recent adoption of FTS by the CMS collaboration.
Deliverable D2.8 (Enhanced data management system planning) – L1 IIe
I have helped form closer links between CMS UK computing and UK physicists and RAL computing. I have formed plans to manage changes to CMS storage at RAL in the UK over the next year, encompassing reduced disk storage resources and stress testing and acceptance of Castor for UK operations. At RAL I have also engaged in discussion and study of CMS job efficiency on RAL worker nodes in collaboration with Matt Hodges, our CMS RAL liaison.
I have been transferring 'old Event Data Model' (EDM) data to RAL with PhEDEx. These transfers were mostly shut down at the end of January, as the sites we were drawing files from were unable to provide them for a variety of reasons. Efforts to publish this data mostly failed, partly because of changes in the publishing software and partly because of as-yet-unresolved problems, apparently in reading the data from dCache.
I also installed a number of CMS software packages at RAL, and helped trigger a discussion within CMS and EGEE about how we should make and track requests for installations.
CONCERNS
B. MacEvoy (Imperial College, 50% FTE) and Y-J Zhang (Imperial College, 100% FTE)
SUMMARY
During this quarter the funded effort of Barry MavEvoy was delivered by David Colling (with half of David's GridPP funded WMS work being delivered by Barry). 
Deliverables (D1.6, 1.8 and 1.9, CMS Job submission and monitoring) – L1 Ib, Id, Ie
BOSS is a CMS computing tool for bookkeeping, monitoring and job  submission. It is originally developed by Bologna (Italy) group, until  version 3. Imperial College joined the BOSS development team at end of the  last year and is taking part in develop BOSS version 4.
1)	Imperial side had developed a new XML based architecture prototype, in this new architecture, functionalities are separated from data and much simplified and generalized while data are represented in form of XML. However, during the Bologna meeting last October, BOSS project coordinator decided to freeze this prototype due to the worries about the XML support from Work Node on Grid where the BOSS jobs will be running. The final agreement is that BOSS team Bologna side was to extend BOSS version 3 prototype to be used by BOSS version 4 which has features such as Task/Chain/Program job hierarchical structure, while the Imperial BOSS team side continues to work on CLI and API. 
2)	BOSS team Bologna side developed BOSS version 4 underlying functionalities with hard coded Schema. This brought difficulties for an XML based CLI and API. To compromise, BOSS team Imperial side revised the API to variables to read from XML file and then pass the values to underlying functionalities. 
3)	The new BOSS version 4 has been committed to CVS. Its first official public release is expected before end of April 2006.
CONCERNS
The commitment of BOSS V4 to CVS reduces the concern that this delayed deliverable would fail to be provided in advance of SC4.
Deliverable D4.1 (Contribution to testing prototype CMS production model) – L1 IVa
Setup of CMS software for London Tier2 (Imperial):
1)	The installation of PhEDEx failed during this quarter. This has been fixed, the version of PhEDEx updated and 5 TB data have been transferred.
2)	The production publishing service CMSGlide has been installed to generate the dataset catalogue for publishing.
3)	PubDB was updated and some datasets were published
J. J. Nebrensky (Brunel, 100%FTE)
SUMMARY
Nebrensky continues to work closely with Barrass and Colling to plan the appropriate strategy for ensuring that Brunel will be able to participate fully in SC06. He has completed the installation of PhEDEx, a critical CMS middleware component. He has increased our local storage capacity (SE) to ~ 2 Tb. Our Tier-2 manager (Duncan Rand) has also contributed unfunded (i.e. HEFCE funded) effort to this task. In this quarter several thousand CMS jobs were submitted and run on both of the Brunel clusters (Argo and Green-Stripe).
He contributed to an poster which was presented at CHEP 2006 : Failure Management in the London Distributed Tier 2, Aggarwal et al (with Kyberd, Nebrensky and Rand from Brunel)
Deliverable D4.2 (Contribution to SC06) – L1 IVb
He attended GridPP 15 and the CMS-UK collaboration meeting, and participates in both the CMS Computing and CMS Integration/Operations weekly meetings.
The Brunel site (part of the London Tier2) is now running CMS MC production as well as Atlas and LHCb (see this GOC entry). A ticket was raised with GOC regarding the continuing issue of accounting properly for separate CEs  at a single site.
We started the upgrade to LCG 2.7.0 within the requested 3 weeks.
He has managed to recover crictical data after separate DNS server and RAID array failures, and has been working on some other system support, such as log rotation and the general question of site bandwidth. Brunel managed to participate in the data transfer challenge stressing our site firewall in the process.
PhEDEx, the CMS data management tool, has been installed and we are just finalising the configuration now.
We are negotiating with Brunel University to rapidly upgrade the WAN to 1Gbit in advance of SC4. Nebrensky has played an important role in demonstrating internal bottlenecks and assisting in data transfer tests and assessing their impact on other Brunel traffic.
CONCERNS
Our principle concern is the current lack of adequate external connectivity at Brunel.
2006 Q2 Comments
B. MacEvoy (Imperial College, 50% FTE) and Y-J Zhang (Imperial College, 100% FTE)
GRIDPP Project Map Deliverable 3.4.4
Installed PhEDEx at Imperial and been performing tests that allow us to tune the transfer configuration. We are approaching the 50MB/s that we did in Service Challenge 3, which is a significant improvement  on the 1->2MB that we had in early June 2006 ago. This is work in progress.

Working to understand the transfers to Brunel (currently only a 100Mbit/s WAN link at Brunel) so that PhEDEx can be installed their as well. This installation will physically be at Imperial but will be controlling transfers to Brunel.
Imperial is now a certified CMS service challenge site for analysis and making good progress towards full certification for production
Attended the Tier 2 Workshop  at CERN (talk presented by D Colling on the T2 analysis Models of the different experiments) and the service challenge workshop at CERN
D Colling gave the CMS talk at GridPP16 (see https://www.gridpp.ac.uk/gridpp16/gridpp16_CMS.ppt).
GRIDPP Project Map Deliverable 3.4.13
The work of Y-J Zhang contributed to the further development of the production version of BOSS (see CVS repository at http://cmsdoc.cern.ch/swdev/viewcvs/viewcvs.cgi/COMP/BOSS/BossPython/?cvsroot=CMSSW)
Finalized first version of BOSS4 C++ API and Python API and tested them in a stand alone mode.
Set up CLI which is based on C++ API.
Prepared an example and instructions of how to use Python API to which other two key CMS projects "CRAB" and "ProdAgent" are going to migrate.
Tested code written by the Bologna BOSS team and helped to find bugs and fix them.
T. Barrass (Bristol, 100% FTE)
GRIDPP Project Map Deliverable 3.4.6
Tim Barrass left the project on 19 May 2006. This has caused a major planning challenge within CMS as a whole (still ongoing) about how the delay in the development and roll-out of PhEdEx should be resolved. We anticipate changes to GRIDPP deliverables and timescales.
J. J. Nebrensky (Brunel, 100%FTE)
GRIDPP Project Map Deliverable 3.4.4
All of the work this quarter was motivated by the forthcoming 2006 Service Challenge for CMS
Attended the NGS Grid tutorial held at Brunel. Attended GRIDPP16. 
Assisted by Duncan Rand upgraded local farm to Glite 3.0.0 and then 3.0.1. Deployed the “OPS” VO, and the new GridMon Box. 
Participated in London Tier-2 discussions about deployment of CMS middleware “PhEdEx”, and attended CMS Integration/Operation meetings. Fixed the CMS transfer “heartbeat” for the Brunel T2 site. Prepared a local internal security audit and action plan. 
Worked with Simon Metson to install latest releases of CMS software on Brunel clusters. We are running CMS jobs which accounted for 16.5% of all Brunel jobs in this quarter. 
Efficiencies of CMS jobs running at Brunel were studied and problems due to non-optimal  PBS queue settings resolved. AFS, to allow direct copying of CMSSW client, was provided to local CMS Physicists.
Henry Nebrensky left the GRIDPP project on 30 June 2006.
2006 Q3 Comments
B. MacEvoy (Imperial College, 50% FTE) and Y-J Zhang (Imperial College, 100% FTE)
GRIDPP Project Map Deliverable 3.4.4

Preparation for CSA 2006. We have been trying to enable the whole of the London Tier 2 for CMS. This has involved, for example, running PhEDEx agents for all the London sites (apart from UCL at the moment) from Imperial, configuring the trivial file catalogue at each site, installing the CMS software etc. Debugging these set ups over the grid is extremely time consuming. Imperial was one of the first sites certified for the pre-CSA MC production and although the current IC resources are 5 years old (meaning that they are under powered compared to modern resources) they actually ran at with an efficiency of 98% meaning that IC was able to make a significant contribution even with these aging resources. Working in collaboration with Duncan Rand we were able to get the more modern resources at Brunel certified for the final part of production and they made a significant contribution (24670 kSI2k cpu hours for CMS). RHUL and QMUL were also finally certified but this was too late to be useful for the pre-production.

London Tier-2 CPU time used by various VO in this quarter; the large increase in CMS use in September is noticeable.
It is anticipated that Imperial and Brunel will actually participate in CSA itself. Data files have already started arriving at Imperial. Although Imperial's ageing resources are sufficient to participate in CSA fully, we are working to be able bring our new resources --400 Intel   Intel (50 dual core dual slots servers) cores and 60TB (50 after raiding) of disk space -- into this activity.
Unfunded effort from IC was provided for the preparation of skimming jobs for CSA and examining their profile (timing on different datasets, output size etc). Also  analyses for the last stage of the CSA. Kostas is preparing the (Z->tau tau analysis and fake tau Analysis). The ability to run analysis jobs on Tier-2 resources is an important part of CSA.
GRIDPP Project Map Deliverable 3.4.13
Yong  Zhang designed, wrote and tested a p2p monitoring system for BOSS (see CVS repository at http://cmsdoc.cern.ch/swdev/viewcvs/viewcvs.cgi/COMP/BOSS/BossPython/?cvsroot=CMSSW). This was his final deliverable in the current work plan. This monitoring is now is part of the BOSS4 build and has been shown to work well. However, its adoption is likely to be given a low priority because of Yong leaving the project and monitoring not being seen as being on the critical path for CMS as a whole.
Yong Zhang left GRIDPP at the end of this quarter.
ANO (Bristol, 100% FTE)
In this quarter there was no replacement for Tim Barrass who left the project in Q2, 2006. 

This is still causing a major planning challenge within CMS as a whole about how the delay in the development and roll-out of PhEdEx should be resolved. The changes to GRIDPP deliverables and timescales which will occur will be resolved in third week of October 2006 at meetings in CERN.

2006 Q4 Comments
B. MacEvoy (Imperial College, 50% FTE) and S Wakefield (Imperial College, 100% FTE)
GRIDPP Project Map Deliverable 3.4.4
Stuart Wakefield was appointed on 24/11/2006.
      CSA06 completed with Imperial providing all functionality of a fully participating Tier-2. These activities included data import/export, MC production and analysis jobs. Peak data rates of 80MB/s have been achieved. ~3,500 analysis jobs have been run over the imported CSA06 data. 
      Support was provided to ensure that CMS jobs from remote users ran without problems.  For the new cluster this required significant work experimenting with and testing a new dCache version to bypass various firewall issues. CMS software installations were performed manually by instead of by the automatic CMS system due to various incompatibilities between the new cluster OS and CMS software. Together with CMS Imperial College is looking for a way that this incompatibility can be eliminated. 
      As well as remote users Imperial has a significant local user community which required support and instruction on how to use the resources available i.e. dCache storage and computational cluster.

Graph shows data rates during one week in November 2006 at Imperial College part of London Tier 2. Peaks rates exceeding 80 Mbytes per second are seen.

Figure taken from "Dashboard": data per site over the period 29/10/2006 until 9/11/2006
S.Metson (Bristol, 100% FTE)
GRIDPP Project Map Deliverable 3.4.6
Relating to new set of milestones agreed with CMS:
1. User interface (Web site 3.0)
	* First set of improvements to the current pages almost complete.
		* Subscription management online now possible
		* To be released in line with the 2.5.x PhEDEx release in Jan 07
	* Work on the "next generation" web pages has begun, in line with the Data Management Web Tools group (which I lead)

2. Centralised routing
	* This has been investigated and tested.
	* It has been shown to work well (at least as performant as the previous routing system, without any database locks, and with more efficient use of the CERN DB cluster).
	* Will be used in the next PhEDEx release (2.5.x series) due January 07.

3. Investigate Data Placement System (DPS)
	* This has been postponed due to changes in the CMS management  structure
	* Needs further discussion with in the Data Management group
	* Current system fine now and in the future

2007 Q1 Comments
B. MacEvoy (Imperial College, 50% FTE) and S Wakefield/M Takahashi (Imperial College, 100% FTE)
GRIDPP Project Map Deliverable 3.4.4
Stuart Wakefield
MC production. First period of Spring07 production LCG6 (UK/Imperial) produced the highest number of events. Over entire period we produced 4.8 million events.

Supporting CMS activities at London Tier2: i.e. software installs, Data transfer system (PhEDEx) installation and upgrades, Maintenance of CMS config files controlling file stage out/in and access to CMS
conditions database.

CMS MC production development:  Develop new global production monitoring system to increase reliability, scalability and functionality over new system. We will use the Dashboard as a secure
information repository and create a new web application under the CMS webtools project  A plan for the architecture exists with active participation of the other production developers and Dashboard teams
in https://twiki.cern.ch/twiki/bin/view/CMS/ProdAgentProdMon.

Maiko Takahashi
Production operation- I have submitted a number of MC production jobs that were recently assigned to LCG6.- I have encountered problems that needed Stuarts help, but am progressing towards  self-sufficiency in running the production jobs in the near future.

Documentation
I am investigating what is already there, what is redundant, and what is missing. Now planning changes to the workbook, Twiki pages etc.

Seb Greder (unfunded effort)

• Users' support service:
1. Continuous improvement/update of user's documentation in for CMS and UK CMS.
2. Presentation of GRID-CMS (CRAB) interface tools in tutorial sessions at CMS UK meeting at Bristol in January.
3. Member of the CMS tutorial task force.
• Computing integration and management:
I have been appointed to CMS UK Data Manager role since early march 2007. This consists of managing CMS data operations like data transfers, production and re-processing on a CMS UK perspective.
• Future involvement:
1. Data production monitoring at Imperial College and London Tier 2 centers
2. Data management role
S.Metson (Bristol, 100% FTE)
GRIDPP Project Map Deliverable 3.4.6
The milestone for this quarter is II.f  “Data management system ready for LHC startup”. The metric is system sign-off by CMS. 
Considerable work has been going into this, most notably (recently) has been migration to support DBS2 in PhEDEx, and the corresponding consistency checking between DBS1, PhEDEx and DBS2. The next major set of work for this deliverable is providing a stable PhEDEx system for CSA07. Planning for this is underway (see https://twiki.cern.ch/twiki/bin/view/CMS/PhedexPlanOld - Goals_and_roadmap_as_of_Quarter).

2007 Q2 Comments
S.Metson (Bristol, 100% FTE); D. Newbold (Bristol, 25%, unfunded effort); J. Jackson (Bristol, 25%, unfunded effort).
GRIDPP Project Map Deliverable 3.4.4
"Final-scale deployment and testing of the CMS Grid computing system worldwide, in preparation for LHC start up. Metric: System meets data taking requirements for both scale and throughput as defined in Computing TDR."
This milestone, due this quarter, has been addressed in the context of the CMS CSA07 challenge. CSA07 will exceed the 2007 targets set down in the CTDR, which assumed data-taking this year. We will remain on course for CTDR targets in 2008 by maintaining this ramp-up. UK work has addressed the PhEDEx data management system, which the we lead. The system is now in a testing phase, with no perceived major problems in the current release. New releases continue to be made, addressing configuration / monitoring functions and interfaces to new releases of underlying middleware (e.g. FTS2.0). The project is on track, though CSA07 has now slipped to Q307.
A major remaining concern in the UK is the validation and acceptance of the RAL CASTOR storage system, which is being commissioned and tested with full input from CMSUK.

GRIDPP Project Map Deliverable 3.4.6
A stable release of PhEDEx has been provided and installed at CMS sites. The current release is available from the CVS repository here:
http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/COMP/PHEDEX/. Documentation on the installation, management and use of PhEDEx is available from the Twiki https://twiki.cern.ch/twiki/bin/view/CMS/PhedexDraftDocumentation#PhEDEx_Developer_Documentation
A paper outlining the performance achieved with PhEDEx was accepted for presentation at the CHEP2007 conference. The paper and slides are available from this link: http://indico.cern.ch/contributionDisplay.py?contribId=258&amp;sessionId=31&amp;confId=3580

GRIDPP Project Map Deliverable 3.4.10
"3.4.10, 07Q3: Data management system ready for LHC startup. Metric is system sign-off by CMS."
Work continues in this area; new requirements on the data management system and a commensurate workplan will be drawn up after CSA07 results are known. This is likely to continue into Q407
B. MacEvoy (Imperial College, 50% FTE) and S Wakefield/M Takahashi/S Greder (Imperial College, 100% FTE in total)
GRIDPP Project Map Deliverables 3.4.4, 3.4.9
Deliverable 3.4.9 (advanced system management planning): Planning information relating to this deliverable is available here: https://twiki.cern.ch/twiki/bin/view/CMS/PhedexPlanOld - Past_PhEDEx_development_plans
CMS Data Production
We have run one of the 6 CMS production teams. The main resources used by this production team include the UK resources, FNAL, ASGC and FZK. The Monte Carlo currently being produced is part of CSA07 exercise. So far we have produced ~5M GenSim and ~5M DigiReco events, making up 55 datasets (see http://tinyurl.com/2mjj8k). 
Monte Carlo Development 
We have developed a new global production monitoring system to increase reliability, scalability and functionality over new system. 
•	The ProdAgent code is now undergoing testing by 3rd parties.
•	Central database was implemented and also being tested.
•	The existing web based monitoring being adapted to use new central database.
Operations
Support for London T2 Sites 
•	Data transfer system (PhEDEx) installation and upgrades
•	Maintenance of CMS configuration files controlling file stage out/in and access to CMS conditions database.
•	Working with London Tier-2 co-ordinator to bring shared Imperial resources online for CMS production and analysis use.
•	Extensive debugging of recent dcache problems including missing freespace and problems with VOMS proxies.
Data Manager 
This post involves:
•	Coordination with different institutes/groups for datasets transfer requests, skims, deletion at T1/T2s
•	Preparation/coordination of CSA07 exercise (stress test of CMS computing model at 50% level, ~10 Sep-10 Oct ) with the CMS UK community and CMS physics groups
Information on the status of UK Data management can be found here https://twiki.cern.ch/twiki/bin/view/CMS/CMSUKDataManagement and documentation, to which we have contributed, on data sets here https://twiki.cern.ch/twiki/bin/view/CMS/PhedexDataManDocsFAQ

GRIDPP Project Map Deliverable 3.4.14, 3.4.15
The final release of BOSS has now been published and is available here: http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/COMP/BOSS/

2007 Q3 Comments
S.Metson (Bristol, 100% FTE); D. Newbold (Bristol, 25%, unfunded effort); J. Jackson (Bristol, 25%, unfunded effort).
GRIDPP Project Map Deliverable 3.4.4
"Final-scale deployment and testing of the CMS Grid computing system worldwide, in preparation for LHC start up. Metric: System meets data taking requirements for both scale and throughput as defined in Computing TDR."
This milestone has been addressed in the context of the CMS CSA07 challenge. A programme of loadtesting of the network / storage systems was put in place in order to support debugging of T1-T2 and T1-T1 links.

Simon Metson has been appointed as the CMS DM/WM manager.

GRIDPP Project Map Deliverable 3.4.6
Updates to PhEDEx has been provided and installed at CMS sites. The current release is available from the CVS repository here: http://cmssw.cvs.cern.ch/cgi-bin/cmssw.cgi/COMP/PHEDEX/.

GRIDPP Project Map Deliverable 3.4.10
"3.4.10, 07Q3: Data management system ready for LHC startup. Metric is system sign-off by CMS." With advent of CSA07 the data management system is now essentially finished.

B. MacEvoy (Imperial College, 50% FTE) and S Wakefield/M Takahashi/S Greder (Imperial College, 100% FTE in total)
GRIDPP Project Map Deliverables 3.4.4, 3.4.9
Imperial contributed to CSA07 and the preCSA production, provided the UK data manager (S Greder) and helped to develop monitoring within ProdAgent. In particular individual contributions are:

M Takahashi contributed to the CMS Monte Carlo production for CSA07
- produced and merged a few million events used many of the London Tier-2 and RALPP nodes
- book keeping made easier in the last few weeks from a new feature in the production software which automates some of the job creation and submission steps

S Wakefield provided support for CMS use of London Tier 2. Primarily involving running/debugging PhEDEx load tests.
https://cmsdoc.cern.ch:8443/cms/aprom/phedex/debug/Activity::RatePlots?graph=quantity_rates&entity=link&src_filter=&dest_filter=London&no_mss=true&period=l120d&upto=&.submit=Update

CMS production: Mainly using UK sites (plus Estonia which has RAL as its Tier1)
https://cmsweb.cern.ch/prod-mon/worker_count_bar?endtime=2007-12-20+19%3A02%3A44&span=86400&starttime=2007-07-1+19%3A02%3A44&workflow=.*&team=LCG6&site=.*&dataset=.*
During CSA07 activity was reduced due to the lack of requests, software installation problems and the CMS decision to dedicate Tier1 resources to reconstruction.
 
The new ProdAgent monitoring system was completed and deployed (see https://cmsweb.cern.ch/prod-mon/). The next version, which integrates with the CMS web framework and has improvements to workflow monitoring, is scheduled for release in January 2008.

5. Meetings & Papers
5.1	List of Conference Papers
5.2	List of Conference Talks/Posters
CHEP04, 27 September to 1 October 2004, Interlaken Switzerland:
“Software agents in data and workflow management”, Barrass T. et al
“LHC data files meet mass storage and networks: going after the lost performance”, Tuura L, Barrass T, Innocente V
“Replaying a data challenge to optimise resource usage”, Tuura L, Barrass T
“GROSS: an end user tool for carrying out batch analysis of CMS data on the LCG-2 Grid”, Tallini H et al
“HEP Applications Experience with the European DataGrid Middleware and Testbed”, Burke S et al
“Distributed Computing Grid Experiences in CMS DC04”, Fanfani A et al
IEEE NSS/MIC/SNPS and RTSD Conference, October 2004, Rome Italy:
“Performance of R-GMA based Grid Job Monitoring System for CMS Data Production”, Byrom R. et al
Frontier Science 2005 Conference September 2005, Milan Italy:
“PhEDEx and the data transfer infrastracture for CMS”, Bonacorsi D. et al (POSTER)
IEEE NSS/MIC/Conference, October 2005, Puerto Rico:
“Performance of R-GMA for Monitoring Grid Jobs for CMS Data Production”, R Byrom et al (POSTER)
CHEP06 Conference, February 2006, Mumbai, India:
“Failure Management in the London Distributed Tier 2”, M Aggarwal et al (POSTER)
CHEP07 Conference, September 2007, Victoria, Canada:
“Scaling CMS data transfer system for LHC start-up”, L Tuura et al (ORAL)
Available: http://indico.cern.ch/contributionDisplay.py?contribId=258&sessionId=31&confId=3580

“CMS Offline Web Tools”, S Metson et al (ORAL)
Available: http://indico.cern.ch/contributionDisplay.py?contribId=266&amp;sessionId=33&amp;confId=3580

5.3	List of publications
Barrass T A, Maroney O J E, Metson S, Newbold D “Integrating the SRB with the GIGGLE framework” Nucl.Instrum. and Meth. A534 (2004) 42-46

Bonacorsi D, Colling D, Field L., Fisher S M, Grandi C, Hobson P R, Kyberd P, MacEvoy B, Nebrensky J J, Tallini H, Traylen S “Scalability Tests of R-GMA-Based Grid Job Monitoring System for CMS Monte Carlo Data Production”, IEEE Trans.Nuc.Sci. 51 (2004) 3026-3029

LHC Computing Grid Technical Design Report
Robertson, Les; Knobloch, Jürgen (ed.); Eck, Chris (resource manager)
LCG-TDR-001; CERN-LHCC-2005-024.- Geneva : CERN, 10 Jun 2005

CMS Computing Technical Design Report /Della Negra, Michel; Petrilli, Achille (resource manager); Herve, Alain (tech. coordinator); Foà, L (chairperson Inst. Board) 
CERN. Geneva. LHC Experiments Committee .- LHCC , 2005.- mult. p.- (Tech. Design Rep. CMS; 7).- CMS-TDR-007.- CERN-LHCC-2005-023

R. Byrom et al, “Performance of R-GMA for Monitoring Grid Jobs for CMS Data Production” 2005 IEEE Nuclear Science Symposium Conference Record pp. 860-864 (2005)

Barrass T et al, “The CMS analysis chain in a distributed environment”,  Nucl.Instrum. and Meth. A559 (2006) 38-42

5.4	Dissemination Activities
AHM2005 Nottingham, 20-22 September 2005
Barrass gave a talk on PhEDEx and Barrass, Colling & Hobson assisted on the GRIDPP stand.