[dmis-bof] Vote for submission of Charter

[James Casey]

Bill,

It seems not to have some of Hiro's changes (like correcting the name of
the mailing list, and the types of the  documents.  Is this the correct
version to comment on as a final draft ?

James.

> -----Original Message-----
> From: owner-dmis-bof at ggf.org [mailto:owner-dmis-bof at ggf.org] 
> On Behalf Of William E. Allcock
> Sent: 21 March 2006 18:19
> To: 'Michel Drescher'; shoshani at lbl.gov
> Cc: dmis-bof at ggf.org; Peter Kunszt; 'Alex Sim'; Timur Perelmutov
> Subject: RE: [dmis-bof] Vote for submission of Charter
> 
> Version 8, which changes to WS-I v1.1 and incorporates the 
> suggested wording changes from Arie.  I REALLY want to get 
> this submitted.  I am about to go on two weeks of travel and 
> I believe there is an SG meeting at months end that I would 
> like to get this in front of so that we can become 
> established as a WG.
> 
> PLEASE comment ASAP if you have concerns on this version.
> 
> Bill 
> 
> > -----Original Message-----
> > From: Michel Drescher [mailto:Michel.Drescher at uk.fujitsu.com]
> > Sent: Tuesday, March 21, 2006 1:54 AM
> > To: shoshani at lbl.gov
> > Cc: allcock at mcs.anl.gov; dmis-bof at ggf.org; 'Peter Kunszt'; 
> 'Alex Sim'; 
> > 'Timur Perelmutov'
> > Subject: Re: [dmis-bof] Vote for submission of Charter
> > 
> > Hi Ari, others,
> > 
> > On 20 Mar 2006, at 22:05, Arie Shoshani wrote:
> > 
> > > Hi, Bill,
> > >
> > > See embedded responses.
> > 
> > ditto.
> > 
> > > William E. Allcock wrote:
> > >>> 1) Regarding the sentence:
> > >>> "Setting up a data movement includes the selection of a
> > transport
> > >>> protocol, for example GridFTP, and parameters for reliability, 
> > >>> timing, scheduling, resource usage, accounting, billing,
> > etc. The
> > >>> Working Group will explore existing mechanisms to reach such 
> > >>> agreement, e.g. WS-Agreement and use them where appropriate."
> > >>>
> > >>
> > >> Perhaps we need to reword this section, but I think the 
> intent of 
> > >> this part was in response to comments about providing 
> information 
> > >> for the service to make prioritization decisions amongst files 
> > >> within a request and between requests.  So, reliability might be 
> > >> parameters associated with number of retries, backoff, 
> etc.; timing 
> > >> could be fail if it takes
> > more than
> > >> x hours;
> > >> scheduling could be have it done by x time; resource 
> usage could be 
> > >> use no more that 50 Mbs; accounting/billing could be 
> don't exceed z 
> > >> cost.
> > >>
> > > I see.  So, you expect other layers (components, services) to 
> > > provides such parameters to the DMIS, right?
> > 
> > Perhaps, perhaps not. This is simply out of scope from where the 
> > service gets its information. However, before a data 
> movement can take 
> > place, all surrounding parameters need to be clarified: A data 
> > movement inflicts temporary changes in the serivce's environment:
> > bandwidth usage, system load, etc. When requesting a data 
> movement, I 
> > have certain expectations that I would like to be met. It is simply 
> > not always true that a service can meet these expectations; for 
> > example if I request a data transfer to be carried out 
> using GridFTP 
> > but the service supports only MTOM, then the request would fail.
> > 
> > This section expresses that the DMIS WG wants to have the 
> parameters 
> > that directly affect the data movement itself sorted out before the 
> > data movement will actually take place. (Don't know it this 
> sentence 
> > makes it any clearer, but on the other hand, I don't know how to 
> > express this better.)
> > 
> > THe DMIS WG will start with a relatively simple model to 
> meet this use 
> > case. It it turns out that this is a more fundamental 
> issue, we always 
> > have the option to re-charter and spin off another Working 
> Group. In 
> > fact, this strategy was discussed at the GGF16 BoF.
> > 
> > >> The issue is that once something has decided the files 
> need to move 
> > >> (like
> > >> SRM) and it gives the service a request for say 100 files, the 
> > >> service could, in theory, move those in any order within 
> a request 
> > >> (perhaps we should have an option for in order...), or 
> may need to 
> > >> prioritize between requests.  So, a user might want to put 
> > >> priorities within a request (though I could argue against that), 
> > >> but they certainly might put priorities between requests 
> of their 
> > >> own.
> > > OK.  This makes sense to me.
> > >> This service is intended to be a file movement service, 
> along the 
> > >> lines of the existing RFT, FTS, etc..
> > We have no
> > >> intention of doing SRM type functionality.  It is something SRM 
> > >> might call.
> > >>
> > > Good.  This is where I found the above sentence unclear.  
> It sounded 
> > > to me like the DMIS will be responsible to call all the other 
> > > services and coordinate various activities including 
> allocating and 
> > > reserving space.  But, as long as it is intended as a 
> "file movement 
> > > service" without expectations that it will
> > perform
> > > space negotiations, etc. it sounds fine.
> > 
> > Yes, 'bull's eye".
> > 
> > >> So, I would argue that we need verbiage that says we will
> > want extra
> > >> information available to make prioritization decisions 
> within the 
> > >> service, and the information is of the type listed.  Do 
> you have a 
> > >> suggestion for a wording change that makes it clearer?
> > >>
> > > I'll try a little later, by the end of the day ...
> > >>
> > >>> Thus, in the setup phase, all kinds of services will have to be 
> > >>> contacted, such as components that manage resources, components 
> > >>> that perform scheduling, keep track of accounting, etc.  For 
> > >>> example, to reserve storage space, the data movement 
> service may 
> > >>> want to interact with SRMs.  I think this is a huge 
> undertaking, 
> > >>> and too large of a scope.  Even if you go through 
> WS-agreement, it 
> > >>> will have to interact with various resource managers, 
> accounting, 
> > >>> billing, etc.  You may want to leave all the
> > setup to
> > >>> a separate component, like WS-agreement, and then focus on the 
> > >>> data movement part.
> > >>>
> > >>
> > >> WS-Agreement is not a component, it is just a pattern for
> > negotiating
> > >> agreements, but as I noted above, our intent is that 
> this is just a 
> > >> file movement service.  The one point I might argue on 
> this would 
> > >> be reporting to such services .vs. negotiating with them.  For 
> > >> instance, if there were some standard message that could 
> be used to 
> > >> report "resource consumption", it might be useful to 
> include some 
> > >> kind of EPR or such which
> > the data
> > >> movement
> > >> service could inform about the resources consumed during the 
> > >> request.  I am not sure this is a good idea, nor that 
> fits in the 
> > >> WS-I
> > basic profile
> > >> because it is a sort of notification, but it is a thought. 
> >  In the
> > >> WSRF
> > >> rendering, you have the alternative of exposing such info
> > as an RP
> > >> and the
> > >> interested service could be directed to subscribe to that RP.
> > >>
> > > Again, I am confused about term"resource consumption".  A 
> DMIS can 
> > > report on file movement, how many files were moved successfully, 
> > > total size moved, hoe many to go, etc.  It cannot report on
> > storage
> > > consumed, because it does not know what is done with the 
> files after 
> > > they are moved (for example they may be used quickly and 
> removed, or 
> > > archived).  Am I missing something?
> > 
> > Yes, a DMIS can and should report on the resources it has 
> consumed to 
> > fulfil a data movement request, e.g. bandwidth usage, CPU 
> cycles(?), 
> > total bytes moved, etc. It is a different set of resources 
> a storage 
> > management system may report, though, but still a report on which a 
> > bill/invoice may be filed later-on.
> > 
> > >>> 2) A related point, in the "Seven questions: Evaluation 
> Criteria 
> > >>> (from GFD.3)" document it says:
> > >>> "The most direct overlap would be with the gsm-wg, but they are 
> > >>> participating in this group and will, presumably, use what is 
> > >>> developed here for the transport portion of their interface."
> > >>>
> > >>> Yes, it makes sense for SRMs to make use of powerful transport 
> > >>> services, since SRMs rely on transport services.  But, when a 
> > >>> request is made to an SRM, they take care of managing storage 
> > >>> space, keeping track of usage and accounting, as well 
> as perform 
> > >>> scheduling, so it does not make sense for the transport 
> services 
> > >>> to negotiate storage allocations (including lifetime), 
> scheduling, 
> > >>> etc, again, as is suggested in the "setup
> > phase" above.
> > >>>
> > >>
> > >> I think I addressed this above.  We have no intention of 
> the data 
> > >> movement service to be doing storage allocation.  I strongly 
> > >> disagree with you though that it does not make sense for the 
> > >> service to be able do scheduling of its own data movement tasks.
> > >>
> > > I do not disagree with you.  I did not mean to say that the DMIS 
> > > should not be able to do its own scheduling of data
> > movement.  What
> > > I meant is that if a request such as srmCopy (which later could be
> > > renamed) is made, and the SRMs reserved space and make their own 
> > > scheduling decisions, that this is now passed to the DMIS.  For 
> > > example, if you include in the DMIS space allocation, then we get 
> > > into a circular situation: call SRM which calls DMIS which
> > calls SRM.
> > 
> > The model would be hierarchical: SRM utilises DMIS to carry out the 
> > data movement, passing on its constraints down to a DMIS 
> (like "Hey, 
> > service, I need to have the following files transferred 
> with such and 
> > such constraints, can you do that?"). If you have only one DMIS in 
> > your domain, that this is usually a piece of cake, basically 
> > obliterating the need of some sort of negotiating/agreement phase.
> > But consider a horde of DMISs, then you may need or want to contact 
> > more than one DMIS and see which one may serve you best.
> > 
> > 
> > Cheers,
> > Michel
> > 
> > 
>