[dmis-bof] Comments on the charter?

[William E. Allcock]

> -----Original Message-----
> From: Peter Kunszt [mailto:peter.kunszt at cern.ch] 
> Sent: Monday, December 19, 2005 9:29 AM
> To: allcock at mcs.anl.gov
> Cc: 'Dave Berry'; 'Michel Drescher'; 'Hiro Kishimoto'; 
> dmis-bof at ggf.org; 'Ian Foster'
> Subject: RE: [dmis-bof] Comments on the charter?
> 
> 
> hi bill
> 
> so you're saying that unless everyone agrees to use either 
> WS-I or WSRF,
> it does not make sense to discuss the interface at all? that 

No, that is not what I said.  I said that *my* main goal for this group was
inter-operability.  If we don’t achieve that, then, for me, much of the
benefit is gone, and the cost / benefit analysis shifts dramatically.  I had
hoped that we could achieve some compromise.  I had assumed that we (the
Globus implementation of this spec) would end up implementing a bunch of
methods for polling state so that your clients could use our services, even
though our clients didn’t need them.  I was hoping you would agree to make
the Resource Properties available in your services, but leave the transition
of your clients till later.

> was not our
> experience elsewhere. discussing semantics is most of the work. the
> rendering is important but should not be the main driver. 

IMHO, the main driver should be inter-operability.  Also, perhaps I am being
naïve, but I think the semantics of this are fairly well understood,
particularly since FTS, RFT, and srm_copy are more similar than different
when it comes to the interface.

> 
> of course the aim is that clients can be written such that
> it does not matter what transfer service will do the transfer. 
> what's your count of 4? globus rft, condor stork, glite fts, srb?

RFT, FTS, srmcopy, and the Unicore stuff that Michel and company sent around
were the 4 I was counting.  Condor and SRB are not participating in this WG
(at least to my knowledge) and are not web service based, so I had not
considered them, though we can if the group thinks it makes sense.

> 
> 
> anyway, if the agreement here is to support only a WSRF rendering from
> the beginning by everyone, of course we can accept that but would be
> limited in our ability to participate in an implementation in 
> the short
> term. it is our plan to become fully WSRF compliant 
> eventually, but you
> know our constraints... so for us to be interoperable on the 
> short term,
> we of course will prefer WS-I. but we don't want to block the
> standardization process.

I am, of course, in the same boat.  We are all over committed and want to
get the most "bang for our buck" when it comes to where we expend our
effort.  If we are not going to achieve inter-operability, I will be limited
in my ability to participate.  I honestly don’t want this to sound like I am
trying to strong arm anyone into WSRF or anything else.  I am just jealous
of my time and the commitments I make and I have to be convinced the benefit
justifies my effort.

Going back to my questions:

 - what is the benefit of doing the functional spec?

Your answer was agreement on semantics and I can see that, particularly
since it can overcome issues like WS-I vs. WSRF.  However, my argument is,
and feel free to try and show me the error in this, that the semantics of
data transfer are actually much simpler and more well understood than say
the whole of SRM and are actually quite well understood.  I am not sure how
much benefit this brings.

 - does forming this working group make sense?

It has value in simply providing a formal communication channel between us
if nothing else.  However, I am not sure that is sufficient benefit for me
to justify the time it would require for me to be the chair.

I am curious whether people agree with me on these points or not.

Bill
> 
> cheers
> peter
> 
> 
> 
> On Mon, 2005-12-19 at 08:51 -0600, William E. Allcock wrote:
> > I have actually been putting some thought into this as 
> well, and I have to
> > ask this question:
> > 
> > What is the real benefit of doing a functional spec?
> > 
> > WS-I and WSRF renderings will NOT interoperate, so I am 
> sort of missing the
> > point.  I can see that it is a way of discussing needed 
> functionality
> > without getting caught up in religious wars over things 
> like WS-I vs. WSRF,
> > but I don't think that is the issue here.  We have 4 
> implementations, 3 of
> > which I know are deployed in "production" environments (I 
> suspect the 4th is
> > in Unicore, but I don't know that for a fact).  We know 
> *how* to do this
> > stuff, the goal was, at least in my mind, 
> inter-operability.  However, if we
> > can't find a compromise, I have to ask another question:
> > 
> > Does forming this WG make sense?
> > 
> > I am willing to put the effort forth to drive this WG if I 
> see a tangible
> > benefit at the end.  However, if the end benefit is going to be
> > non-interoperable implementations, that happen to use the 
> same names for
> > state or some such, then I can't personally justify 
> expending the time on
> > this, unless someone can convince me that there is benefit that I am
> > missing.
> > 
> > Bill
> > 
> > > -----Original Message-----
> > > From: Peter Kunszt [mailto:peter.kunszt at cern.ch] 
> > > Sent: Monday, December 19, 2005 8:09 AM
> > > To: Dave Berry
> > > Cc: Michel Drescher; allcock at mcs.anl.gov; Hiro Kishimoto; 
> > > dmis-bof at ggf.org; Ian Foster
> > > Subject: RE: [dmis-bof] Comments on the charter?
> > > 
> > > 
> > > 
> > > hi dave
> > > 
> > > thanks! i admit that i didn't have the time to follow the latest
> > > developments in the past year. for the open source stuff,
> > > i have not found e.g. the jboss and apache 
> implementations in any of
> > > the mainstream (say axis) code yet... or did i miss it?
> > > 
> > > but back on topic: i think we all agree that we need 
> > > standardisation on
> > > the transfer interface. the question is now what is the 
> > > context we embed
> > > this standardisation into? if it is OGSA/WSRF then for 
> > > production grids
> > > as EGEE we cannot expect to have robust off-the-shelf 
> implementations
> > > within the next 18 months (that is my personal assessment of the
> > > current state of the art, maybe i'm wrong). if it is WS-I 
> then there
> > > is a possibility to have implementations very soon, and thus have
> > > interoperability within a short time.
> > > 
> > > we agreed here that the functional spec of the interface 
> can be done
> > > first, and then the rendering into WS-I or WSRF can come 
> as needed. 
> > > so all i'm saying is: don't use a concept like 
> WS-Addressing in the
> > > functional spec as it is rendering-specific! however, we can use
> > > a "URL" as a concept as it is generic enough. keeping the 
> functional
> > > spec independent of the implementation is a good idea. 
> then of course
> > > the WSRF rendering can and should use concepts that are within its
> > > context, including WS-Addressing.
> > > 
> > > of course how the different implementations/renderings will 
> > > interoperate
> > > is a different topic altogether. if the functional 
> semantics are the
> > > same then at least you can write clients that understand 
> both flavors.
> > > 
> > > cheers,
> > > 
> > > peter
> > > 
> > > 
> > > On Mon, 2005-12-19 at 09:39 +0000, Dave Berry wrote:
> > > > Hi Peter,
> > > > 
> > > > There are implementations of WS-A by at least MS, IBM, BEA, 
> > > Sun, JBOSS &
> > > > Apache.  
> > > > 
> > > > Dave.  
> > > > 
> > > > > -----Original Message-----
> > > > > From: Peter Kunszt [mailto:peter.kunszt at cern.ch] 
> > > > > Sent: 19 December 2005 08:38
> > > > > To: Dave Berry
> > > > > Cc: Michel Drescher; allcock at mcs.anl.gov; Hiro Kishimoto; 
> > > > > dmis-bof at ggf.org; Ian Foster
> > > > > Subject: RE: [dmis-bof] Comments on the charter?
> > > > > 
> > > > > hi dave
> > > > > 
> > > > > can you point me to the implementation of ws-addressing 
> > > that you refer
> > > > > to as 'just about everybody'?
> > > > > 
> > > > > cheers
> > > > > peter
> > > > > 
> > > > > On Sat, 2005-12-17 at 16:23 +0000, Dave Berry wrote:
> > > > > > I'd push for WS-Addressing EPRs where possible.  The 
> > > simplest EPR is
> > > > > > just a wrapped URI (actually an IRI, I believe), so it is a
> > > > > > straightforward extension of using URLs.  WS-Addressing is 
> > > > > implemented
> > > > > > by just about everybody, I believe.
> > > > > > 
> > > > > > If you use WS-A then it should be straightforward to 
> > > generalise or
> > > > > > extend to include other specs based on WS-A.  These 
> > > include WSRF and
> > > > > > WS-Naming, of course, but also other specs being developed 
> > > > > within the
> > > > > > industry.
> > > > > > 
> > > > > > Having said that, I would add the caveat that I'm not 
> > > > > expert on WS-A and
> > > > > > it would be good to get input from someone who really 
> > > > > understands this
> > > > > > stuff in detail.
> > > > > > 
> > > > > > Dave.
> > > > > > 
> > > > > > -----Original Message-----
> > > > > > From: owner-dmis-bof at ggf.org 
> > > > > [mailto:owner-dmis-bof at ggf.org] On Behalf
> > > > > > Of Michel Drescher
> > > > > > Sent: 16 December 2005 11:04
> > > > > > To: Peter Kunszt
> > > > > > Cc: allcock at mcs.anl.gov; Hiro Kishimoto; dmis-bof at ggf.org; 
> > > > > Ian Foster
> > > > > > Subject: Re: [dmis-bof] Comments on the charter?
> > > > > > 
> > > > > > 
> > > > > > Hi all,
> > > > > > 
> > > > > > On 14 Dec 2005, at 17:06, Peter Kunszt wrote:
> > > > > > 
> > > > > > >>>>  - we need to address naming.  What will we accept as 
> > > > > source and 
> > > > > > >>>> destination names? URLs? URIs? any string? EPRs?
> > > > > > >>>
> > > > > > >>> URL seems like a good choice, this would 
> probably suit all
> > > > > > >> use cases.
> > > > > > >>> this is what we use in the GSM-WG. we have storage URLs.
> > > > > > >>> an EPR is nothing more than a decorated URL so 
> a simple URL
> > > > > > >> would suit
> > > > > > >>> that too.
> > > > > > >>
> > > > > > >> URLs work fine for files, and I *suspect* it will be 
> > > > > what we use for 
> > > > > > >> V1.0, but if we take our file blinders off and try and 
> > > > > think about 
> > > > > > >> source other than files, i.e. a service that is 
> > > > > virtualizing some 
> > > > > > >> non-file data source, we might want EPRs.  This is 
> > > > > related to the 
> > > > > > >> next issue.
> > > > > > >
> > > > > > > well, if you take rich URLs, you can get away 
> with anything. 
> > > > > > > ?query=...
> > > > > > 
> > > > > > I strongly support URLs. Note that, opposed to the very 
> > > popular  
> > > > > > perception, URLs describe *resources*, not (only) files. 
> > > > > Apart from  
> > > > > > the scheme and authority parts of URLS (i.e. "http" and  
> > > > > > "forge.gridforum.org[:80]") URLs describe resources in a  
> > > > > > hierarchically ordered domain. THe query part is just a 
> > > piece of  
> > > > > > information that further transports information 
> > > specific to the  
> > > > > > addressed resource.
> > > > > > 
> > > > > > So if you want to publish for example the coffee 
> > > > > temperature in your  
> > > > > > mug using HTTP, you could do that with the following 
> > > imaginary URL:
> > > > > > 
> > > > > > 	
> > > http://users.mcs.anl.gov/~allock/resources/sensors/coffem-mug? 
> > > > > > refresh=10
> > > > > > 
> > > > > > Requesting that URL may result in a nice data stream 
> > > sending the  
> > > > > > current temperature in 10 second intervals.
> > > > > > Note that the path element does not necessarily have to map 
> > > > > to a file  
> > > > > > system path - refer i.e. to the J2EE Servlet 
> > > specification that  
> > > > > > splits the path information into three elements:
> > > > > > - context path (a J2EE enterprise application)
> > > > > > - servlet path (a locator for servlets within a J2EE 
> > > enterprise  
> > > > > > application)
> > > > > > - path info    (the residual URL path element)
> > > > > > All three elements concatenated form the original URL path 
> > > > > element;  
> > > > > > and the context path and servlet path may be null. The 
> > > mapping is  
> > > > > > purely virtual and is specified in the EAR deployment 
> > > descriptors.
> > > > > > 
> > > > > > >> I guess now that I think about it, your way works too.  
> > > > > As long as 
> > > > > > >> the query mechanism is there, as long as you don't send 
> > > > > a block the 
> > > > > > >> site doesn't support, you wont fail validation... I 
> > > > > don't like it, 
> > > > > > >> but unless we discover a bigger problem than that, I 
> > > > > guess it is an 
> > > > > > >> option :-).
> > > > > > >
> > > > > > > the trick is easy. you put in an element that you don't 
> > > > > define. you 
> > > > > > > then extend it and play around the way you like it, and 
> > > > > once you know 
> > > > > > > what you want, you just take it into your mainstream WSDL 
> > > > > as a proper 
> > > > > > > part of it. see our discussion/example in 
> > > > > > > https://uimon.cern.ch/twiki/bin/view/SRMDev/XmlSolution
> > > > > > 
> > > > > > This URL requires authentication which I cannot provide 
> > > for AFS...
> > > > > > 
> > > > > > >>
> > > > > > >>>
> > > > > > >>>>  - what about scheduling / planning aspects?  
> Do we want
> > > > > > >> to include
> > > > > > >>>> elements in this WSDL that specify rate (bandwidth),
> > > > > > >> quantity (file
> > > > > > >>>> size), and timing (START BY, FINISH BY, etc)
> > > > > > >>>
> > > > > > >>> our experience: individually for each transfer job: no.
> > > > > > >> that is very
> > > > > > >>> complicated and of questionable use.
> > > > > > >>> on the scale of the service itself as a service
> > > > > > >>> parameter/configuration: yes.
> > > > > > >>
> > > > > > >> I half agree with you.  Clearly the service, may want to 
> > > > > have limits 
> > > > > > >> that can be set.  In that regard, it is acting as a 
> > > > > resource manager 
> > > > > > >> and protecting the resource for which it is responsible, 
> > > > > much like a 
> > > > > > >> compute scheduler.  Btw, I would argue that this is 
> > > > > outside the scope
> > > > > > 
> > > > > > >> of this working group, since an external entity has no 
> > > > > need to be 
> > > > > > >> involved with that.  At most we might want to agree on 
> > > > > some state 
> > > > > > >> that such a service MAY (in the SPEC sense of the 
> > > word) expose.
> > > > > > >>
> > > > > > >> However, it is not clear to me that requests per 
> job are of 
> > > > > > >> questionable use.  Clearly they should be optional 
> > > > > elements, but if 
> > > > > > >> the submission has no such information, on what 
> > > basis does the 
> > > > > > >> service make decisions when it has a resource 
> > > > > constraint?  I would 
> > > > > > >> also add priority to the list.  If I have multiple jobs 
> > > > > submitted, I 
> > > > > > >> may want to make sure that one of them gets service over 
> > > > > the others.
> > > > > > >
> > > > > > > ok, i was sloppy. it is true that you should be able to 
> > > > > influence the 
> > > > > > > priority of your transfers and maybe even their 
> > > > > speed/bandwidth within
> > > > > > 
> > > > > > > your allocated 'quota'.
> > > > > > 
> > > > > > Yes, I agree here as well. We're on the right track I 
> > > > > think, so let's  
> > > > > > fight over the details in the meetings. ;-)
> > > > > > 
> > > > > > Cheers,
> > > > > > Michel
> > > > > > 
> > > > > 
> > > 
> > 
>