Connecting arbitrary data resources to the grid
Date
2010
Authors
Zhang, S.
Coddington, P.
Wendelborn, A.
Editors
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Conference paper
Citation
Proceedings of 2010 11th IEEE/ACM International Conference on Grid Computing, 2010: pp.185-192
Statement of Responsibility
Shunde Zhang, Paul Coddington, Andrew Wendelborn
Conference Name
IEEE/ACM International Conference on Grid Computing (11th : 2010 : Brussels, Belgium)
Abstract
Many scientific grid systems have been running and serving researchers for many years around the world. Among them, Globus Toolkit and its variants are playing an important role as the basis of most of those existing grid systems. However, the way data is stored and accessed varies. Proprietary protocols have been designed and developed to serve data by different storage systems or file systems. One example is the integrated Rule Oriented Data System (iRODS), which is a data grid system with the non-standard iRODS protocol and has its own client tools and API. Consequently, it is difficult for the grid to connect to it directly and stage data to computers in the grid for processing. It is usually an ad hoc process to transfer data between two data systems with different protocols. In addition, existing data transfer services are mostly designed for the grid and do not understand proprietary protocols. This requires users to transfer data from the source to a temporary space, and then transfer it from the temporary space to the destination, which is complex, inefficient and error-prone. Some work has been done on the client side to address this issue. In order to address the issues of data staging and data transfer in one solution, this paper describes a different but easy and generic approach to connect any data systems to the grid, by providing a service with an abstract framework to convert any underlying data system protocol to the GridFTP protocol, a de facto standard of data transfer for the grid.
School/Discipline
Dissertation Note
Provenance
Description
Access Status
Rights
© 2010 IEEE