[gin-data] FW: [gin-ops] Draft GIN Paper for e-Science 2007

[Cindy Zheng]

My previous cc bounced. I'm resending this, in case other group interested
in the issues gin-ops encountered and worked.  ...Cindy

-----Original Message-----
From: Cindy Zheng [mailto:zhengc at sdsc.edu]
Sent: Sunday, July 15, 2007 2:26 PM
To: 'Morris Riedel'
Cc: 'gin-auth at ggf.org'; 'gin-data at ogf.org'; 'gin-info at ogf.org';
'gin-jobs at ogf.org'; 'gin-ops at ogf.org'; 'gin at ogf.org'
Subject: RE: [gin-ops] Draft GIN Paper for e-Science 2007

Hi, Morris,

Here are some key points and thoughts regarding gin-ops work.
(I'm cc'ing to all gin lists to take this opportunity of communication)

You can find references to most of the points below at
http://forge.gridforum.org/sf/wiki/do/viewPage/projects.gin/wiki/GinOps.
Also, all gin-ops members can add and correct.

The goal of gin-ops work is to discover issues, test ideas and provide
insights to standardization effort.

The methods we used so far are:

1. Run real science applications in gin testbed. Each production grid join
one or more their production clusters to form a gin testbed. So far, we have
run TDDFT and also started on a data-intensive application - Savannah fire
simulation. 

2. Near term implementation and long term research on cross-grid monitoring
in collaboration with gin-info. 

3. Peer-grids interoperation experiments. Run real science applications
across a pair of grids. The difference compare to gin testbed is that there
less number of grids but more resources of each grid are involved, which
gives different dimensions of understanding and perspectives to the issues.
The application ran in this environment so far is FMO. For more detail info,
see
http://goc.pragma-grid.net/wiki/index.php/OSG-PRAGMA_Grid_Interoperation_Exp
eriments

The main issues learned so far can be categorized in 3 areas:

1. Trust and access: This is the least foggy among the 3. :-) IGTF and VOMS
help to provide the mechanism for inter-grid trust, although more
simplification of user interface is desired.

2. Job submission: Among the 3 areas, this has been the biggest hurtle in
our experiments. 
There are 2 main issues:
A. The applications we ran so far all used to submit jobs via the original
GRAM method. Those grids provide the GRAM worked easily. Those grids with
only modified or non-GRAM services, an interface has to be developed for
each to enable the application run. This has tried with one grid and worked.
But, we feel that peer-grid interface development is not a scalable nor
desired solution for the long term.
A better solution would be to develop standards which all grids want to
interoperate with others can work with.
B. Some grids are setup to serve certain types of applications, other
applications may encounter difficulties. One example is that job scheduling
systems at some grids work well with embarrassingly-parallel applications,
but not applications require synchronization or communication among
processes.

3. Software support environment. Some grids require site admins to install
the software which the application requires; some grids require application
be self-contained - to package all software it needed (sandbox). 2 thoughts
have been discussed, one is to adopt the sandbox method, although not easy
for naïve users; second is to provide community software area (CSA) where
users can install and share software, but there are difficulties in
management and performance issues.

Hope this will help with the gin-ops section in the paper.

Thanks,

Cindy