[saga-rg] Fwd (hupfeld at zib.de): Re: SAGA Strawman API Version 1.0

[Jon MacLaren]

Hi Andre, all,

Actually, that wasn't the point that I was making.  My point is that  
the code writer has to be aware of whether they are writing code for  
a local filesystem, or a remote one.  (Most application writers  
assume that they *are* writing for POSIX filesystems, thinking that  
their code will only be run on local POSIX filesystems.)

If I am writing code for a system that's distributed, then I should  
try to make the code "loosely coupled".  In other words, to have as  
infrequent interactions with the remote services as possible, and to  
make these more coarse grained.  I will also try to allow for  
failures in the network, knowing that even if it is not possible to  
communicate with the remote component now, it might be possible in  
the future.  (As an example, I have some recent code which tries to  
push logical file mappings into Globus RLS.  If the RLS service is  
not up, the mappings are cached locally, and will be uploaded later.)

With filesystems, if you are writing code to work on a local  
filesystem, then the way you would behave in an error condition is  
different.  If you can't write to your local filesystem, the program  
should probably try to quit (unless perhaps the disk is full, and  
there are some temporary files you can clean out - but this is  
already complex).  However, if the filesystem might be AFS, or  
another remote filesystem, I should think about writing the code so  
that the operations can be retried later.  (With AFS, the writes will  
"succeed", it's the close file operation that you have to be careful  
with.)

Having (briefly) read the discussion on the mailing list, I don't  
believe that trying to enforce POSIX semantics on remote filesystems  
solves the above problem.  You might get a filesystem that has POSIX  
semantics, but it would behave like a very unreliable POSIX  
filesystem.  The point, which is made well in the paper I cited, is  
that you should not try to make remote things look like local things.

In terms of practical advice for SAGA, I don't know how important  
this point is.  The application writer, if using the SAGA interface,  
will probably know that the API is designed to communicate with  
remote components, and will hopefully code accordingly.

I do believe that deciding upon single-system-type POSIX semantics  
everywhere would be a mistake.  But I agree with the point that SAGA  
has to say something about (consistency) semantics too.  If the GGF  
schedule permits, I hope to attend a lively discussion on this next  
week.

For the meeting, it might be worth trying to contact the Legion (and  
AVAKI?) people.  They have been doing this stuff for around ten years  
now.  They support multiple underlying filesystem semantics, I  
believe.  Perhaps this is the voice of experience?  We should try to  
get someone from their team to come along to the discussion.

Jon.


On Jun 20, 2005, at 3:21 AM, Andre Merzky wrote:

> Not w/o semantics, but the programmer has to assume worst
> case.
>
> And, as Jon & John point out, that is not so different from
> what you have 'locally'.  If you open a file for write, you
> SHOULD assume worst case, that is the file is on NFS/AFS,
> and POSIX consistency is NOT guaranteed.
>
> A.
>
>
> Quoting [Thorsten Schuett] (Jun 20 2005):
>
>>
>> I see the point that it is hard guarantee a specific consistency  
>> model in SAGA
>> or make it a configuration option to choose one. But if you  
>> completely ignore
>> this topic in the specs you will end up with a good spec of the  
>> syntax but
>> without semantics.
>>
>> My 0.02???,
>> Thorsten
>>
>
> -- 
> +-----------------------------------------------------------------+
> | Andre Merzky                      | phon: +31 - 20 - 598 - 7759 |
> | Vrije Universiteit Amsterdam (VU) | fax : +31 - 20 - 598 - 7653 |
> | Dept. of Computer Science         | mail: merzky at cs.vu.nl       |
> | De Boelelaan 1083a                | www:  http://www.merzky.net |
> | 1081 HV Amsterdam, Netherlands    |                             |
> +-----------------------------------------------------------------+
>
>