CSGrid is a system for management and integration of applications in a distributed and heterogeneous computing environment. It provides a collaborative and extensible environment to abstract the use of the computational resources, and providing to end-users and domain-specific applications functionalities for creating, sharing and accessing distributed resources, such as projects’ data and executable programs. Figure 1 shows an overview of the CSGrid middleware.
Figure 1 - CSGrid middleware overview
CSGrid Server and Middleware Services
The CSGrid Server is a component that intermediates requests from users and client applications to data repositories and execution nodes. The CSGrid data repository is an area where the CSGrid server stores the user database, user projects and algorithms execution code. The user database keeps data about users and their permissions. The projects area has a directory for the projects of each user and keeps descriptor files that hold information about projects and the projects files. The algorithm repository has a hierarchical structure that uses directories to represent each algorithm. Each directory contains a configuration file, the binaries and subfolders containing the algorithm versions.
Provides facilities for user management and authentication. Validates user authorization for the execution of the provided functionalities and access to resources.
The SGA is a component that exposes interfaces to allow remote execution of algorithms on execution host machines, and the monitoring of their status by the middleware (e.g. current or historical processing and storage usage). It also allows querying the machines’ configurations, their execution platforms, environment attributes and their current status. The SGA interface is flexible and can be instantiated in different scenarios. It can be deployed as a daemon in host machines to provide an execution interface to these hosts individually, or it can be set as a gateway to 3rd party Resource Management Systems. In the former case, each host machine becomes a server for the execution of algorithms. In the latter case, a module inside the SGA implementing the integration with a specific system is used, and the SGA intermediates execution requests from CSGrid to the other systems and manages these executions. Currently, there are SGA gateways implementations for the Sun Grid Engine, TORQUE/PBS, OurGrid, SLURM and Amazon EC2.
Provides mechanisms for the configuration of an optional sandbox area used for the input and output data of algorithms executions.
OpenBus is a CORBA-based middleware used to integrate multi-platform and multi-language systems based on a service-oriented architecture. It offers a service directory, access control mechanisms and peer-to-peer communication. External applications use the OpenBus development kit (OpenBus SDK) to authenticate, publish, discover and access services components, based on the usage policies configured by the OpenBus administrator.