A production system for massive data processing in ILCSoft

Abstract

This memo presents a new production system for massive data processing on the Grid and other large computing facilities. The system is primarily targeted at being used with programs from the ILCSoft software framework developed partly within the EUDET project. The new system is written in Python and centered around a MySQL database. It provides the functionality that is needed for the submission, monitoring and bookkeeping of many thousands of jobs on the Grid. The same database that is used for the bookkeeping of the production serves also as the basis for a data catalog that can be queried through a webinterface. ∗Deutsches Elektronen-Synchrotron, Hamburg, Germany

3 Figures and Tables

Cite this paper

@inproceedings{Aplin2009APS, title={A production system for massive data processing in ILCSoft}, author={Stephen Aplin and John Engels and Frank Gaede}, year={2009} }