[SAGA-RG] missing(?) method reporting last modification time

[Sylvain Reynaud]

Andre Merzky a écrit :
> Dear Sylvain,
>   
Dear Andre,

> Quoting [Sylvain Reynaud] (Jun 04 2009):
>   
>> SAGA Job Management:
>> ===================
>> * CPUArchitecture and OperatingSystemType should be scalar (instead of 
>> vector) attributes for consistency with JSDL specification.
>>     
>
> Good find!  These are indeed errors in the spec, and should
> be scalar attributes.  Will fix.
>
>
>   
>> * page 166 says: "Attributes marked as 'not supported by JSDL' might 
>> disappear in future versions of the SAGA API".
>>   - Queue: this attribute makes the job description dependent on the 
>>     targeted
>>     execution site, this information should be put in the URL instead.
>>     
>
> Interesting point.  The problem I see is that its hard to
> define a standard way on *how* to encode it in the URL, as
> each URL component (host, path, query, ...) may already be
> interpreted by the backend.
>
> For example, a globus job manager URL may well look like
>
> https://some.remote.host:9443/wsrf/services/ManagedExecutableJobService?65e59770-35e1-11da-8810-a04185b6c7ae
>
> Where would you put the queue?
>   
In JSAGA, such URL is used internally, user gives this URL:
wsgram://some.remote.host:9443/Fork

If encoding the queue in the URL is not an acceptable solution, then I 
think the queue should be moved from attributes of job description to 
arguments of method job_service.create_job.

>   
>>   - JobStartTime and JobContact: IMHO, these attributes are not in the 80%
>>     of the 80/20 rule, they are not supported by most middlewares, and they
>>     can be implemented by the user application.
>>     
>
> Those have been included because DRMAA has these, and the
> DRMAA folx found them dead useful.  
>
> JobStartTime I expect to get defined in JSDL when they get
> started on scheduling (which is not in the near future
> AFAIK).  In SAGA, it probably should be move to the resource
> management package, when that emerges.
>   
When this feature is not available in middleware, should SAGA 
implementations delay job submission by itself (i.e. on client-side) or 
throw a NotImplemented exception ?

> JobContact is a tricky one: it seems dead useful to be able
> to specify a custom name to a job, and to identify the job
> thus in an easy way, but (a) I have yet to see a backend
> which supports it, (b) SAGA does not provide a way to
> actually *use* this name, e.g. as jobid, and (c) such a
> mapping can trivially be implemented on application level.
>   
I think you are talking about JobName, which is currently supported by 
JSAGA (I plan to remove it for portability) but is not defined in the 
SAGA specification.
JobContact is "set of endpoints describing where to report job state 
transitions": I think your comments (a) and (c) also apply to this one.

> I am with you here.  Anyway, I think it is kind of dangerous
> to remove stuff from the spec simply because we don't see a
> use at the moment.  Not sure how representative we are...
>
>
>   
>>   - Interactive: although this attribute may not be in the 80% of the 
>> 80/20 rule,
>>     I think it is usefull and should be kept.
>>     
>
> Many of our original use cases had a visualization
> background, where interactive jobs are more likely to be
> useful than in other fields...
>
>
>   
>> * job are missing a state "QUEUED" in order to enable timeout on jobs 
>> queued for a long time, synchronization with job start, or just because 
>> many users want to know if their jobs are running or queued. This can 
>> not be done with the job.state_detail attribute because its content is 
>> not uniform. As it was said at OGF23, some job services don't have any 
>> queue, but I think this is not the usual case.
>>     
>
> I don't see why job.state_detail can't help you here: your
> SAGA implementation should always be able to bap the
> respective backend queue state to something like
> 'jsaga:queued'.  
>
> The spec says that job_detail SHOULD be formatted as
> 'model:state', but you seem to have a very valid use case
> for defining your own backend model for jsaga.
>   
OK. I was thinking that state_detail was supposed to contain only 
middleware-specific states, but indeed nothing in the specification is 
saying that...
So I agree with you, I will remove my sub_state attribute and use the 
state_detail instead !

> Reason why I am hesitant to agree with you, despite of the
> validity of your use case, is: while defining the spec and
> the state model, we tried only to expose those state at the
> top level which are the result of a SAGA API call, or can be
> left with a SAGA API call.  
>
> There is not saga.job.queue()  or saga::job::unqueue, so
> moving Queued on the top state model would break that rule
> of thumb.
>
>
>   
>> SAGA Name Spaces:
>> ================
>> * add a flag to disable checking existence of entry in constructor and 
>> open methods, because the cost for this check is not negligible with 
>> some protocols (then subsequent method calls on this object may throw an 
>> IncorrectState exception
>> if the entry does not exist).
>>     
>
> Makes sense.  We could also overload 'Exclusive', which, at
> the moment, is only evaluated if 'Create' is specified.  It
> has the same semantic meaning so (inversed): if 'Exclusive'
> is not specified on 'Create', an existing file is ignored.
>
> Would it make sense to allow Exclusive to be evaluated on
> all c'tors and open calls?
>
>
>   
>> SAGA Session:
>> ============
>> * An exception should be thrown when several contexts of the session can 
>> be used for a specific method call. Else, some files may be created with 
>> unexpected owner, some jobs may fail at the end of execution because of 
>> unexpected permissions, some accounts may be locked because of too many 
>> failed connection attempts.
>>     
>
> Uh, this would badly break our use cases!  In our
> implementation, you may very well copy a file instance with
> http, but read it with ftp, and delete it with nfs!  And we
> need the respective contexts for these backends on the same
> session...
>
> Yes, results are not completely predictable if multiple
> backends can be used, but the user has always the option to
> create a specific session with exactly one context attached.
>
> So, that's a tough one.
>   
OK. Can we consider that this is up to the implementation, or do you 
think these different behaviors would break portability ?

>   
>> * AFAIK, existing SAGA implementations support many technologies and may 
>> be used with several grids. Default session is very convenient, but it 
>> is not usable when several grids can be accessed, because a single 
>> default session for all is not enough. Argument of session constructor 
>> could be an identifier (e.g. a grid identifier) instead of a boolean.
>>     
>
> Again, you can create a context for a Grid type, and attache
> just that one context to a session.  Isn't that what you
> need?
>   
I think creating default session is very convenient, and I was 
frustrated not being able to use this possibility with my use-cases! ;-)
However, I can indeed create my own default session builder on top of 
the SAGA specification.

Best regards,
Sylvain

> I may be missing something here...
>
> Thanks, Andre.
>
>
>