[saga-rg] some principle comments on SAGA

[Andre Merzky]

Hi Alexander, 

thanks for your thoughts!  In fact, that goes along the
lines of several ongoing discussions - please see inserted
comments below.


Quoting [Alexander Beck-Ratzka] (Jun 21 2006):
> 
> Hello SAGA developers,
> 
> first of all: I am not taking part in developing SAGA, but I try to distribute 
> GAT within the german D-Grid community project, and therefore I know about 
> the SAGA efforts, and - as I feel - I also see some problems in the 
> realisation...
> 
> Okay, let's start with the problems of GAT.
> 
> As GAT has started, it missed a lot of adaptors to the mostly ditributed grid 
> middleware products. This is - at least in Germany - a fundamental reason for 
> the very low acceptance of GAT in the Grid Community.

Understandable.  What use is an API if it does not run on
the environment in use...


> As far as I know SAGA seems to inherit this deficiency of GAT. So Andre statet 
> recently in a Email to me: "... we can definitively support only some 
> adaptors, and we need the help of those projects, which have to develop AND 
> SUPPORT adaptors to their middleware..."

That statement was in respect to the people who develop the
C++ reference implementation.  So, that group of people
(currently Hartmut, Stephan and I) just don't have the
resources secured to allow definite plans for any set of
(well supoported) adaptors.

OTOH, we are aware of that problem, and very much so, as we
hade to deal with the same situaltion for GAT, as you
mention.

Now, there are two general ways to get middleware bindings:)
(a) find funding to do it yourself, or (b) convince other
groups to provide those.

 (a) Hartmut is currently funded from CCT, Stephan is
     writing his masters thesis, and has no secured funding
     after that right now.  I will be funded for SAGA work from
     now on, but for a specific EU project.  

     That obviously does not leave us much room for writing
     globus, EGEE, Unicore etc. bindings, and to support them

     OTOH, we try to get additional funding _dedicated_ to
     adaptor development: Shantenu tries to secure a small grant
     for that, and CCT tries to provide internal funding for
     some adaptors, to be used in their environment.

     It is doubtful if that will allow to cover support for more
     than one or two major middlewares.


 (b) How do you get other groups and projects to provide MW
     bindings?  Two ways: 

     (1) talk to their users: if their users demand SAGA for
         their middleware, they will provide and support
         bindings.

     (2) convince them that providing bindings is in their
         own interest: they find more users, and have an
         easier time supporting them.

     We try to address both: lobbying in application groups,
     and middleware provider groups.  Currently, Ed and
     others are considering to found a 'SAGA Alliance' or
     something similar, to have a visible set of SAGA
     supporters, which might help to get critical mass.
     Also, we go to conferences and present SAGA, and plan
     to provide tutorials etc. to bring the API closer to
     application groups.


> Please dont understand me wrong: I know about the small personel resources in 
> the SAGA project, and that therefore only the development of a "SAGA Engine" 
> and some local adaptors is possible. However, the missing of adaptors to 
> Globus, Unicore or Condor etc.., is an urgent problem. The acceptance of 
> software increases with its possibilities ot use it. 

Absolutely agree. We are somewhat disappointed from DGrid in
that respect: Yes, it is great that they consider to use
GAT, but what we actually have been hoping for are some
resources to develop GAT adaptors...  If DGrid just looks at
the API, and sees it useful, w/o acknowledging the need for
support for the actual bindings, it won't fly.

The same holds for other projects.  Neither GAT nor SAGA are
something which is 'ready to use' - right now its rather a
framework, which you can use to get a SAGA API for your
users in place.

The story is different for Java CoG, hence their success...

> If SAGA delivers now 
> only an engine without a connection to any grid middleware, I would also not 
> use SAGA to access e.g. Globus grdi functionality. Why should I develop an 
> adaptor to my grid middleware (and even support it later on) for enabling the 
> usage of SAGA; in such a case it's easier to develop directly against the 
> middleware API.

No, that is wrong.  Well, at least it is wrong if you
consider the target community of SAGA: Grid Unaware
Application developers!  For you it might be the same, for
some biology student, which tries to get his software
running, it is not!  Please don't confuse the groups we try
to convince to provide SAGA bindings and implementations,
with the group of people which is supposed to use it - in
general, these groups are totally different...


> If I develop an adaptor, I'll have to possible regions where 
> problems can arise: SAGA and the access to the middleware API. Developing 
> diretyl against the middleware will only bring problems with the middleware 
> API. I believe that such preconditions will not lead to a wide acceptance of 
> SAGA!
> 
> How to get rid of this problem? I like to suggest to deliver a SAGA toolkit. 
> Beside the engine there also should be delivered a set of adaptors to the 
> mostly distributed Grid middleware, as Globus, Unicore, Condor, etc... But 
> how can this bed realised with the restricted resources? I have had the idea 
> that the CCT should increase its support of the SAGA development, in Germany, 
> there is no money available (at least right now). I don't know, if this is 
> possible, but I'm quite shure that it must be a SAGA toolkit instead of a 
> SAGA engine with local adaptors, if SAGA should be successful.

Hehe, nice - we would love to do that!  Helas, you under
estimate the amount of funding neccessary.  The problem is
actually not writing the adaptors, but to support and
maintain them.  

In GAT, we had a number of adaptors written by students,
which worked pretty well for a while.  However, students
leave, and after a while all you have left is a set of badly
documented adaptors in mediocre quality, which break all
over the place, and which eat your support resouces...
We don't want to repeat that expereince (from GAT).

So, you need more than a couple of students, and for more
than a year!  CCT does not have that kind of funding, nor
has any institution alone.  

Also, we thing that the most proficient groups to write
adaptors to some middleware are the developers of that
middleware!  For them, support of the adaptors is _much_
easier, and its also much simplier to maintain them over the
development cycles of that middleware.


> I hope, you understand my words as a helpful comment. I want SAGA to be 
> successful, and therefore we have to think about a SAGA toolkit.

Alexander, thanks for your comments, and they truly point on
the biggest acceptance problem we'll face: solid
implementations.  However, as said, we beleave that the
ultimate goal should be to have the middleware developers
supporting SAGA bindings to their API, and move that burden
away from the application groups and projects.  

Cheers, Andre.


> Cheers
> 
> Alexander

-- 
"So much time, so little to do..."  -- Garfield