[SAGA-RG] missing(?) method reporting last modification time
Andre Merzky
andre at merzky.net
Wed Sep 23 08:47:28 CDT 2009
Dear Sylvain,
I dropped the ball on this thread I think. Also, I think we
came a conclusion about a number of issues already. So, let
me try to summarize where we stand. I'd loke to use this as
a last call for the list for the closed items, and as a call
for feedback for the items still open.
Closed items
------------------------------------------------------------
- add a LastModiefied timestamp to namespace entries (in
addition to Created timestamp)
-> added as get_mtime() to namespace::entry
- IncorrectType for
task t = f.get_size <Sync> ();
size_t s = t.get_result <char> ();
-> added to the spec
- context.toString():
-> this is a language binding issue - no change in spec
- NSEntry.remove() should allow for rmdir
-> corrected in spec (Recursive flag only required for
non-empty dirs)
- CPUArchitecture and OperatingSystemType should be scalar
-> needs to be fixed in spec
- job are missing a state "QUEUED"
-> this is a state_detail of the Running state - no
change in spec.
- removing the Queue attrib
-> resolution unclear, possibly postponed to next JSDL
version, or to a SAGA resource package, whichever
comes first
- avoid check for existence on open/creation of ns entries
-> two possible solutions
(a) overload Exclusive
// entry exists
open (name, Create | Exclusive) : fail
open (name, Create ) : success
open (name, Exclusive) : success
open (name ) : success
// entry does not exist
open (name, Create | Exclusive) : success (creates)
open (name, Create ) : success
open (name, Exclusive) : no check (later IncorrectState)
open (name ) : fail
(b) add new flag 'DoNotFailIfDoesNotExist' (better
name needed)
// entry exists
open (name, Create | Exclusive) : fail
open (name, Create ) : success
open (name, DNFIDNE ) : success
open (name ) : success
// entry does not exist
open (name, Create | Exclusive) : success (creates)
open (name, Create ) : success
open (name, DNFIDNE ) : no check (later IncorrectState)
open (name ) : fail
I vote for (a), because I think its simplier and
because I can't think of a good name for the new flag.
Sylvain votes for (b) IIRC, but does not have a good
name either ;-)
Group should consider this to be a last call!
So, I hope I covered all items - let me know if not!
Best, Andre.
Quoting [Sylvain Reynaud] (Jun 06 2009):
> Date: Sat, 06 Jun 2009 20:29:10 +0200
> From: Sylvain Reynaud <Sylvain.Reynaud at in2p3.fr>
> To: Andre Merzky <andre at merzky.net>
> CC: Thilo Kielmann <kielmann at cs.vu.nl>, saga-rg at ogf.org
> Subject: Re: [SAGA-RG] missing(?) method reporting last modification time
>
> Andre Merzky a écrit :
> >Quoting [Sylvain Reynaud] (Jun 05 2009):
> >
> >>Andre Merzky a écrit :
> >>
> >>
> >>>Hi again,
> >>>
> >>Hi again,
> >>
> >>
> >>>Quoting [Sylvain Reynaud] (Jun 05 2009):
> >>>
> >>>>>
> >>>>>
> >>>>>>- Queue: this attribute makes the job description dependent on the
> >>>>>> targeted
> >>>>>> execution site, this information should be put in the URL instead.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>Interesting point. The problem I see is that its hard to
> >>>>>define a standard way on *how* to encode it in the URL, as
> >>>>>each URL component (host, path, query, ...) may already be
> >>>>>interpreted by the backend.
> >>>>>
> >>>>>For example, a globus job manager URL may well look like
> >>>>>
> >>>>>https://some.remote.host:9443/wsrf/services/ManagedExecutableJobService?65e59770-35e1-11da-8810-a04185b6c7ae
> >>>>>
> >>>>>Where would you put the queue?
> >>>>>
> >>>>>
> >>>>>
> >>>>In JSAGA, such URL is used internally, user gives this URL:
> >>>>wsgram://some.remote.host:9443/Fork
> >>>>
> >>>>
> >>>sure, that will mostly work. The point is however, that we
> >>>can't assure that it breaks for other backends which require
> >>>a path specification on the URL.
> >>>
> >>But anyway, I think that the main point is not to know if we should put
> >>it in the URL or not, it is rather to know if the queue is part of the
> >>job description or part of the targeted resource.
> >>
> >>IMHO, the answer is "targeted resource", because if the service
> >>discovery extension does not provide this information (either in the URL
> >>or in the service_data object), you can not guess it by yourself.
> >>
> >
> >Hi Sylvain,
> >
> Hi,
>
> >yes, excellent description of the problem: it should be part
> >of the resource specification, not part of the job
> >description. Alas, we don't have a resource description
> >(yet). BTW, the same holds IMHO for CPUArchitecture for
> >example, doesn't it?
> >
> I think CPUArchitecture and other resource specification attributes are
> part of the job description, since they describe the job requirements.
> But IMHO, attribute queue is not part of resource specification, it is
> part of *resource location* (like URL).
>
> Although queues are often configured with names "short" or "long", they
> can be used for very different purposes (e.g. queues by VO, by SLA, by
> feature...), they can have different names even when used for the same
> purpose, and when discovering job services, the queue is always in the
> response rather than in the query.
>
> >
> >
> >>>>If encoding the queue in the URL is not an acceptable
> >>>>solution, then I think the queue should be moved from
> >>>>attributes of job description to arguments of method
> >>>>job_service.create_job.
> >>>>
> >>>>
> >>>Thats also an option. What would be the difference however
> >>>to keeping it in the job description? The info arrives at
> >>>the same call, once in the description, once separate.
> >>>
> >>>
> >>The difference is that other attributes in job description do not depend
> >>on a particular execution site or a particular grid. Hence the same job
> >>description object could be used to run jobs on different hosts (and
> >>even on different grids) if it has no attribute "Queue".
> >>
> >
> >Ideally that may be true, but in practice, CPUArchitecture,
> >OperatingSystem, and others pose similar limitations.
> >
> IMHO the limitations are not similar :
> * If a job requires a specific OperatingSystem to run, then we can
> assume this requirement is the same for grid A and grid B.
> * If the user wants to submit his job on a specific queue on grid A, he
> can not expect to have the same queue on grid B.
>
> >Anyway, don't get me wrong: I think I mostly agree with you
> >about the problem statement, and the cause. I am not 100%
> >about the proposed solution,
> I have no preference on the proposed solution (URL, create_job argument,
> or other solution...), I just think queue must be removed from job
> description.
>
> >but that may be just me, being
> >hesitant to change (I'm known for that I'm afraid)...
> >
> I think you are right to be hesitant; specifications must not change too
> much!
>
> >
> >
> >>>I understand that having only JSDL approved keys in the job
> >>>description is a clean solution - but that is mostly for the
> >>>benefit of the SAGA implementors. For the SAGA users, that
> >>>makes not much of the difference, IMHO.
> >>>
> >>Since they are not in the JSDL specification, these attributes are
> >>likely to be put at stake... Moreover, the SAGA specification says these
> >>attributes "might disappear in future versions of the SAGA API".
> >>
> >>But I agree, if their usefulness is confirmed, they must be kept.
> >>
> >
> >I think, in the long run, further versions of JSDL, and JSDL
> >extensions, will make our live much easier...
> >
> >
> >
> >>>>>>SAGA Name Spaces:
> >>>>>>================
> >>>>>>* add a flag to disable checking existence of entry in constructor
> >>>>>>and open methods, because the cost for this check is not negligible
> >>>>>>with some protocols (then subsequent method calls on this object may
> >>>>>>throw an IncorrectState exception
> >>>>>>if the entry does not exist).
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>Makes sense. We could also overload 'Exclusive', which, at
> >>>>>the moment, is only evaluated if 'Create' is specified. It
> >>>>>has the same semantic meaning so (inversed): if 'Exclusive'
> >>>>>is not specified on 'Create', an existing file is ignored.
> >>>>>
> >>>>>Would it make sense to allow Exclusive to be evaluated on
> >>>>>all c'tors and open calls?
> >>>>>
> >>>>>
> >>>Any feedback on this one? :-)
> >>>
> >>Good idea IMHO, but then I think the name of this flag should be changed
> >>to one suitable for both use-cases : exclusive creation and no file
> >>existence check.
> >>
> >
> >Ah, well, naming - you are opening a bottomless pit! ;-)
> >Any proposal?
> No proposal yet... I am thinking about it!
>
> >I throw in 'FailIfExists' ...
> >
> FailIfExists match the first use-case (exclusive creation), the second
> use-case needs DoNotFailIfDoesNotExist ! ;-)
>
> Best regards,
> Sylvain
>
> >
> >
> >>>I am still not sure about introducing an additional
> >>>exception here, but that is another issue...
> >>>
> >>>
> >>Maybe the right exception to be thrown is AuthenticationFailed.
> >>Then its description should be changed to something like this (page 40) :
> >>
> >><< An operation failed because session could not successfully be used
> >>for authentication (none of the available contexts could be used, or can
> >>not determine which context to use). >>
> >>
> >
> >I think thats an excellent proposal.
> >
> >
> >Best regards,
> >
> > Andre.
> >
> >
> >
>
--
Nothing is ever easy.
More information about the saga-rg
mailing list