[SAGA-RG] missing(?) method reporting last modification time

[Sylvain Reynaud]

Andre Merzky a écrit :
> Hi again, 
>   
Hi again,

> Quoting [Sylvain Reynaud] (Jun 05 2009):
>   
>>>  
>>>       
>>>>  - Queue: this attribute makes the job description dependent on the 
>>>>    targeted
>>>>    execution site, this information should be put in the URL instead.
>>>>    
>>>>         
>>> Interesting point.  The problem I see is that its hard to
>>> define a standard way on *how* to encode it in the URL, as
>>> each URL component (host, path, query, ...) may already be
>>> interpreted by the backend.
>>>
>>> For example, a globus job manager URL may well look like
>>>
>>> https://some.remote.host:9443/wsrf/services/ManagedExecutableJobService?65e59770-35e1-11da-8810-a04185b6c7ae
>>>
>>> Where would you put the queue?
>>>  
>>>       
>> In JSAGA, such URL is used internally, user gives this URL:
>> wsgram://some.remote.host:9443/Fork
>>     
>
> sure, that will mostly work.  The point is however, that we
> can't assure that it breaks for other backends which require
> a path specification on the URL.
>   
Then it can be added to the query part of the URL...

But anyway, I think that the main point is not to know if we should put 
it in the URL or not, it is rather to know if the queue is part of the 
job description or part of the targeted resource.

IMHO, the answer is "targeted resource", because if the service 
discovery extension does not provide this information (either in the URL 
or in the service_data object), you can not guess it by yourself.

>
>   
>> If encoding the queue in the URL is not an acceptable
>> solution, then I think the queue should be moved from
>> attributes of job description to arguments of method
>> job_service.create_job.
>>     
>
> Thats also an option.  What would be the difference however
> to keeping it in the job description?  The info arrives at
> the same call, once in the description, once separate. 
>   
The difference is that other attributes in job description do not depend 
on a particular execution site or a particular grid. Hence the same job 
description object could be used to run jobs on different hosts (and 
even on different grids) if it has no attribute "Queue".

> I understand that having only JSDL approved keys in the job
> description is a clean solution - but that is mostly for the
> benefit of the SAGA implementors.  For the SAGA users, that
> makes not much of the difference, IMHO.
>   
Since they are not in the JSDL specification, these attributes are 
likely to be put at stake... Moreover, the SAGA specification says these 
attributes "might disappear in future versions of the SAGA API".

But I agree, if their usefulness is confirmed, they must be kept.

> Well, I guess there are pros and cons for both versions.
> Maybe others on the list have any preference one qay or the
> other?
>
>
>   
>>>>  - JobStartTime and JobContact: IMHO, these attributes are not in the 80%
>>>>    of the 80/20 rule, they are not supported by most middlewares, and 
>>>>    they
>>>>    can be implemented by the user application.
>>>>    
>>>>         
>>> Those have been included because DRMAA has these, and the
>>> DRMAA folx found them dead useful.  
>>>
>>> JobStartTime I expect to get defined in JSDL when they get
>>> started on scheduling (which is not in the near future
>>> AFAIK).  In SAGA, it probably should be move to the resource
>>> management package, when that emerges.
>>>  
>>>       
>> When this feature is not available in middleware, should SAGA 
>> implementations delay job submission by itself (i.e. on client-side) or 
>> throw a NotImplemented exception ?
>>     
>
> To throw or not to throw, both would be valid, IMHO, and up
> to the implementation.  For example, specifying a startup
> time far in the future would most likely trigger a
> BadParameter exception, as it cannot be expected that the
> job service instance is still alive by then.  Having a very
> short delay may be acceptable for some implementations, and
> not for others.
>
> Anyway, a BadParameter exception is more appropriate than a
> NotImplemented, as the latter would suggest that the method
> create_job() is not implemented at all.  BadParameter
> suggests that the interpretation of the method parameters
> (the job description) failed, which is in fact what happens.
>   
OK.

>
>   
>>> JobContact is a tricky one: it seems dead useful to be able
>>> to specify a custom name to a job, and to identify the job
>>> thus in an easy way, but (a) I have yet to see a backend
>>> which supports it, (b) SAGA does not provide a way to
>>> actually *use* this name, e.g. as jobid, and (c) such a
>>> mapping can trivially be implemented on application level.
>>>       
>> I think you are talking about JobName, which is currently supported by 
>> JSAGA (I plan to remove it for portability) but is not defined in the 
>> SAGA specification.
>>     
>
> Uhoh, bad mixup on my part - my apologies!
>
>
>   
>> JobContact is "set of endpoints describing where to report
>> job state transitions": I think your comments (a) and (c)
>> also apply to this one.
>>     
>
> Agree.
>
>
>   
>>> [Queued...]
>>>       
>> OK. I was thinking that state_detail was supposed to contain only 
>> middleware-specific states, but indeed nothing in the specification is 
>> saying that...
>> So I agree with you, I will remove my sub_state attribute and use the 
>> state_detail instead !
>>     
>
> Great!
>
>
>   
>>>> SAGA Name Spaces:
>>>> ================
>>>> * add a flag to disable checking existence of entry in constructor and 
>>>> open methods, because the cost for this check is not negligible with 
>>>> some protocols (then subsequent method calls on this object may throw an 
>>>> IncorrectState exception
>>>> if the entry does not exist).
>>>>     
>>>>         
>>> Makes sense.  We could also overload 'Exclusive', which, at
>>> the moment, is only evaluated if 'Create' is specified.  It
>>> has the same semantic meaning so (inversed): if 'Exclusive'
>>> is not specified on 'Create', an existing file is ignored.
>>>
>>> Would it make sense to allow Exclusive to be evaluated on
>>> all c'tors and open calls?
>>>       
>
> Any feedback on this one? :-)
>   
Good idea IMHO, but then I think the name of this flag should be changed 
to one suitable for both use-cases : exclusive creation and no file 
existence check.

>
>   
>>>> SAGA Session:
>>>> ============
>>>> * An exception should be thrown when several contexts of the session can 
>>>> be used for a specific method call. Else, some files may be created with 
>>>> unexpected owner, some jobs may fail at the end of execution because of 
>>>> unexpected permissions, some accounts may be locked because of too many 
>>>> failed connection attempts.
>>>>    
>>>>         
>>> Uh, this would badly break our use cases!  In our
>>> implementation, you may very well copy a file instance with
>>> http, but read it with ftp, and delete it with nfs!  And we
>>> need the respective contexts for these backends on the same
>>> session...
>>>
>>> Yes, results are not completely predictable if multiple
>>> backends can be used, but the user has always the option to
>>> create a specific session with exactly one context attached.
>>>
>>> So, that's a tough one.
>>>  
>>>       
>> OK. Can we consider that this is up to the implementation, or do you 
>> think these different behaviors would break portability ?
>>     
>
> I think it makes sense to leave that to the implementation,
> as (a) we have use cases for both versions, and (b)
> implementations are not required to bind to multiple
> backends: if they bind to a single backend, the behaviour
> would be very much like what you describe.
>   
OK.

> I am still not sure about introducing an additional
> exception here, but that is another issue...
>   
Maybe the right exception to be thrown is AuthenticationFailed.
Then its description should be changed to something like this (page 40) :
<< An operation failed because session could not successfully be used 
for authentication (none of the available contexts could be used, or can 
not determine which context to use). >>

>
>   
>> I think creating default session is very convenient, and I was 
>> frustrated not being able to use this possibility with my use-cases! ;-)
>>     
>
> Hehe, I see.  Good point.  Might not justfy an errata, but
> would be good to keep in mind for the next version...
>   
OK.

Best regards,
Sylvain
> Cheers, Andre.
>
>
>