[dmis-bof] Vote for submission of Charter

[William E. Allcock]

Yikes!  No, I forgot to make the type of document change and apparently
completely missed the mailing list.  I am about to board a flight, but will
try and send out an updated version when I land in about 8 hours.

Bill 

> -----Original Message-----
> From: owner-dmis-bof at ggf.org [mailto:owner-dmis-bof at ggf.org] 
> On Behalf Of James Casey
> Sent: Wednesday, March 22, 2006 6:35 AM
> To: allcock at mcs.anl.gov; Michel Drescher; shoshani at lbl.gov
> Cc: dmis-bof at ggf.org; Peter Kunszt; Alex Sim; Timur Perelmutov
> Subject: RE: [dmis-bof] Vote for submission of Charter
> 
> Bill,
> 
> It seems not to have some of Hiro's changes (like correcting 
> the name of
> the mailing list, and the types of the  documents.  Is this 
> the correct
> version to comment on as a final draft ?
> 
> James.
> 
> > -----Original Message-----
> > From: owner-dmis-bof at ggf.org [mailto:owner-dmis-bof at ggf.org] 
> > On Behalf Of William E. Allcock
> > Sent: 21 March 2006 18:19
> > To: 'Michel Drescher'; shoshani at lbl.gov
> > Cc: dmis-bof at ggf.org; Peter Kunszt; 'Alex Sim'; Timur Perelmutov
> > Subject: RE: [dmis-bof] Vote for submission of Charter
> > 
> > Version 8, which changes to WS-I v1.1 and incorporates the 
> > suggested wording changes from Arie.  I REALLY want to get 
> > this submitted.  I am about to go on two weeks of travel and 
> > I believe there is an SG meeting at months end that I would 
> > like to get this in front of so that we can become 
> > established as a WG.
> > 
> > PLEASE comment ASAP if you have concerns on this version.
> > 
> > Bill 
> > 
> > > -----Original Message-----
> > > From: Michel Drescher [mailto:Michel.Drescher at uk.fujitsu.com]
> > > Sent: Tuesday, March 21, 2006 1:54 AM
> > > To: shoshani at lbl.gov
> > > Cc: allcock at mcs.anl.gov; dmis-bof at ggf.org; 'Peter Kunszt'; 
> > 'Alex Sim'; 
> > > 'Timur Perelmutov'
> > > Subject: Re: [dmis-bof] Vote for submission of Charter
> > > 
> > > Hi Ari, others,
> > > 
> > > On 20 Mar 2006, at 22:05, Arie Shoshani wrote:
> > > 
> > > > Hi, Bill,
> > > >
> > > > See embedded responses.
> > > 
> > > ditto.
> > > 
> > > > William E. Allcock wrote:
> > > >>> 1) Regarding the sentence:
> > > >>> "Setting up a data movement includes the selection of a
> > > transport
> > > >>> protocol, for example GridFTP, and parameters for 
> reliability, 
> > > >>> timing, scheduling, resource usage, accounting, billing,
> > > etc. The
> > > >>> Working Group will explore existing mechanisms to reach such 
> > > >>> agreement, e.g. WS-Agreement and use them where appropriate."
> > > >>>
> > > >>
> > > >> Perhaps we need to reword this section, but I think the 
> > intent of 
> > > >> this part was in response to comments about providing 
> > information 
> > > >> for the service to make prioritization decisions amongst files 
> > > >> within a request and between requests.  So, 
> reliability might be 
> > > >> parameters associated with number of retries, backoff, 
> > etc.; timing 
> > > >> could be fail if it takes
> > > more than
> > > >> x hours;
> > > >> scheduling could be have it done by x time; resource 
> > usage could be 
> > > >> use no more that 50 Mbs; accounting/billing could be 
> > don't exceed z 
> > > >> cost.
> > > >>
> > > > I see.  So, you expect other layers (components, services) to 
> > > > provides such parameters to the DMIS, right?
> > > 
> > > Perhaps, perhaps not. This is simply out of scope from where the 
> > > service gets its information. However, before a data 
> > movement can take 
> > > place, all surrounding parameters need to be clarified: A data 
> > > movement inflicts temporary changes in the serivce's environment:
> > > bandwidth usage, system load, etc. When requesting a data 
> > movement, I 
> > > have certain expectations that I would like to be met. It 
> is simply 
> > > not always true that a service can meet these expectations; for 
> > > example if I request a data transfer to be carried out 
> > using GridFTP 
> > > but the service supports only MTOM, then the request would fail.
> > > 
> > > This section expresses that the DMIS WG wants to have the 
> > parameters 
> > > that directly affect the data movement itself sorted out 
> before the 
> > > data movement will actually take place. (Don't know it this 
> > sentence 
> > > makes it any clearer, but on the other hand, I don't know how to 
> > > express this better.)
> > > 
> > > THe DMIS WG will start with a relatively simple model to 
> > meet this use 
> > > case. It it turns out that this is a more fundamental 
> > issue, we always 
> > > have the option to re-charter and spin off another Working 
> > Group. In 
> > > fact, this strategy was discussed at the GGF16 BoF.
> > > 
> > > >> The issue is that once something has decided the files 
> > need to move 
> > > >> (like
> > > >> SRM) and it gives the service a request for say 100 files, the 
> > > >> service could, in theory, move those in any order within 
> > a request 
> > > >> (perhaps we should have an option for in order...), or 
> > may need to 
> > > >> prioritize between requests.  So, a user might want to put 
> > > >> priorities within a request (though I could argue 
> against that), 
> > > >> but they certainly might put priorities between requests 
> > of their 
> > > >> own.
> > > > OK.  This makes sense to me.
> > > >> This service is intended to be a file movement service, 
> > along the 
> > > >> lines of the existing RFT, FTS, etc..
> > > We have no
> > > >> intention of doing SRM type functionality.  It is 
> something SRM 
> > > >> might call.
> > > >>
> > > > Good.  This is where I found the above sentence unclear.  
> > It sounded 
> > > > to me like the DMIS will be responsible to call all the other 
> > > > services and coordinate various activities including 
> > allocating and 
> > > > reserving space.  But, as long as it is intended as a 
> > "file movement 
> > > > service" without expectations that it will
> > > perform
> > > > space negotiations, etc. it sounds fine.
> > > 
> > > Yes, 'bull's eye".
> > > 
> > > >> So, I would argue that we need verbiage that says we will
> > > want extra
> > > >> information available to make prioritization decisions 
> > within the 
> > > >> service, and the information is of the type listed.  Do 
> > you have a 
> > > >> suggestion for a wording change that makes it clearer?
> > > >>
> > > > I'll try a little later, by the end of the day ...
> > > >>
> > > >>> Thus, in the setup phase, all kinds of services will 
> have to be 
> > > >>> contacted, such as components that manage resources, 
> components 
> > > >>> that perform scheduling, keep track of accounting, etc.  For 
> > > >>> example, to reserve storage space, the data movement 
> > service may 
> > > >>> want to interact with SRMs.  I think this is a huge 
> > undertaking, 
> > > >>> and too large of a scope.  Even if you go through 
> > WS-agreement, it 
> > > >>> will have to interact with various resource managers, 
> > accounting, 
> > > >>> billing, etc.  You may want to leave all the
> > > setup to
> > > >>> a separate component, like WS-agreement, and then 
> focus on the 
> > > >>> data movement part.
> > > >>>
> > > >>
> > > >> WS-Agreement is not a component, it is just a pattern for
> > > negotiating
> > > >> agreements, but as I noted above, our intent is that 
> > this is just a 
> > > >> file movement service.  The one point I might argue on 
> > this would 
> > > >> be reporting to such services .vs. negotiating with them.  For 
> > > >> instance, if there were some standard message that could 
> > be used to 
> > > >> report "resource consumption", it might be useful to 
> > include some 
> > > >> kind of EPR or such which
> > > the data
> > > >> movement
> > > >> service could inform about the resources consumed during the 
> > > >> request.  I am not sure this is a good idea, nor that 
> > fits in the 
> > > >> WS-I
> > > basic profile
> > > >> because it is a sort of notification, but it is a thought. 
> > >  In the
> > > >> WSRF
> > > >> rendering, you have the alternative of exposing such info
> > > as an RP
> > > >> and the
> > > >> interested service could be directed to subscribe to that RP.
> > > >>
> > > > Again, I am confused about term"resource consumption".  A 
> > DMIS can 
> > > > report on file movement, how many files were moved 
> successfully, 
> > > > total size moved, hoe many to go, etc.  It cannot report on
> > > storage
> > > > consumed, because it does not know what is done with the 
> > files after 
> > > > they are moved (for example they may be used quickly and 
> > removed, or 
> > > > archived).  Am I missing something?
> > > 
> > > Yes, a DMIS can and should report on the resources it has 
> > consumed to 
> > > fulfil a data movement request, e.g. bandwidth usage, CPU 
> > cycles(?), 
> > > total bytes moved, etc. It is a different set of resources 
> > a storage 
> > > management system may report, though, but still a report 
> on which a 
> > > bill/invoice may be filed later-on.
> > > 
> > > >>> 2) A related point, in the "Seven questions: Evaluation 
> > Criteria 
> > > >>> (from GFD.3)" document it says:
> > > >>> "The most direct overlap would be with the gsm-wg, 
> but they are 
> > > >>> participating in this group and will, presumably, use what is 
> > > >>> developed here for the transport portion of their interface."
> > > >>>
> > > >>> Yes, it makes sense for SRMs to make use of powerful 
> transport 
> > > >>> services, since SRMs rely on transport services.  But, when a 
> > > >>> request is made to an SRM, they take care of managing storage 
> > > >>> space, keeping track of usage and accounting, as well 
> > as perform 
> > > >>> scheduling, so it does not make sense for the transport 
> > services 
> > > >>> to negotiate storage allocations (including lifetime), 
> > scheduling, 
> > > >>> etc, again, as is suggested in the "setup
> > > phase" above.
> > > >>>
> > > >>
> > > >> I think I addressed this above.  We have no intention of 
> > the data 
> > > >> movement service to be doing storage allocation.  I strongly 
> > > >> disagree with you though that it does not make sense for the 
> > > >> service to be able do scheduling of its own data 
> movement tasks.
> > > >>
> > > > I do not disagree with you.  I did not mean to say that 
> the DMIS 
> > > > should not be able to do its own scheduling of data
> > > movement.  What
> > > > I meant is that if a request such as srmCopy (which 
> later could be
> > > > renamed) is made, and the SRMs reserved space and make 
> their own 
> > > > scheduling decisions, that this is now passed to the DMIS.  For 
> > > > example, if you include in the DMIS space allocation, 
> then we get 
> > > > into a circular situation: call SRM which calls DMIS which
> > > calls SRM.
> > > 
> > > The model would be hierarchical: SRM utilises DMIS to 
> carry out the 
> > > data movement, passing on its constraints down to a DMIS 
> > (like "Hey, 
> > > service, I need to have the following files transferred 
> > with such and 
> > > such constraints, can you do that?"). If you have only 
> one DMIS in 
> > > your domain, that this is usually a piece of cake, basically 
> > > obliterating the need of some sort of negotiating/agreement phase.
> > > But consider a horde of DMISs, then you may need or want 
> to contact 
> > > more than one DMIS and see which one may serve you best.
> > > 
> > > 
> > > Cheers,
> > > Michel
> > > 
> > > 
> > 
> 
>