[dmis-bof] Vote for submission of Charter

[Hiro Kishimoto]

Thanks Bill,

It looks good to me.
----
Hiro Kishimoto

William E. Allcock wrote:
> Ok, lets try THIS version 
> 
> 
>>-----Original Message-----
>>From: owner-dmis-bof at ggf.org [mailto:owner-dmis-bof at ggf.org] 
>>On Behalf Of James Casey
>>Sent: Wednesday, March 22, 2006 6:35 AM
>>To: allcock at mcs.anl.gov; Michel Drescher; shoshani at lbl.gov
>>Cc: dmis-bof at ggf.org; Peter Kunszt; Alex Sim; Timur Perelmutov
>>Subject: RE: [dmis-bof] Vote for submission of Charter
>>
>>Bill,
>>
>>It seems not to have some of Hiro's changes (like correcting 
>>the name of
>>the mailing list, and the types of the  documents.  Is this 
>>the correct
>>version to comment on as a final draft ?
>>
>>James.
>>
>>
>>>-----Original Message-----
>>>From: owner-dmis-bof at ggf.org [mailto:owner-dmis-bof at ggf.org] 
>>>On Behalf Of William E. Allcock
>>>Sent: 21 March 2006 18:19
>>>To: 'Michel Drescher'; shoshani at lbl.gov
>>>Cc: dmis-bof at ggf.org; Peter Kunszt; 'Alex Sim'; Timur Perelmutov
>>>Subject: RE: [dmis-bof] Vote for submission of Charter
>>>
>>>Version 8, which changes to WS-I v1.1 and incorporates the 
>>>suggested wording changes from Arie.  I REALLY want to get 
>>>this submitted.  I am about to go on two weeks of travel and 
>>>I believe there is an SG meeting at months end that I would 
>>>like to get this in front of so that we can become 
>>>established as a WG.
>>>
>>>PLEASE comment ASAP if you have concerns on this version.
>>>
>>>Bill 
>>>
>>>
>>>>-----Original Message-----
>>>>From: Michel Drescher [mailto:Michel.Drescher at uk.fujitsu.com]
>>>>Sent: Tuesday, March 21, 2006 1:54 AM
>>>>To: shoshani at lbl.gov
>>>>Cc: allcock at mcs.anl.gov; dmis-bof at ggf.org; 'Peter Kunszt'; 
>>>
>>>'Alex Sim'; 
>>>
>>>>'Timur Perelmutov'
>>>>Subject: Re: [dmis-bof] Vote for submission of Charter
>>>>
>>>>Hi Ari, others,
>>>>
>>>>On 20 Mar 2006, at 22:05, Arie Shoshani wrote:
>>>>
>>>>
>>>>>Hi, Bill,
>>>>>
>>>>>See embedded responses.
>>>>
>>>>ditto.
>>>>
>>>>
>>>>>William E. Allcock wrote:
>>>>>
>>>>>>>1) Regarding the sentence:
>>>>>>>"Setting up a data movement includes the selection of a
>>>>
>>>>transport
>>>>
>>>>>>>protocol, for example GridFTP, and parameters for 
>>
>>reliability, 
>>
>>>>>>>timing, scheduling, resource usage, accounting, billing,
>>>>
>>>>etc. The
>>>>
>>>>>>>Working Group will explore existing mechanisms to reach such 
>>>>>>>agreement, e.g. WS-Agreement and use them where appropriate."
>>>>>>>
>>>>>>
>>>>>>Perhaps we need to reword this section, but I think the 
>>>
>>>intent of 
>>>
>>>>>>this part was in response to comments about providing 
>>>
>>>information 
>>>
>>>>>>for the service to make prioritization decisions amongst files 
>>>>>>within a request and between requests.  So, 
>>
>>reliability might be 
>>
>>>>>>parameters associated with number of retries, backoff, 
>>>
>>>etc.; timing 
>>>
>>>>>>could be fail if it takes
>>>>
>>>>more than
>>>>
>>>>>>x hours;
>>>>>>scheduling could be have it done by x time; resource 
>>>
>>>usage could be 
>>>
>>>>>>use no more that 50 Mbs; accounting/billing could be 
>>>
>>>don't exceed z 
>>>
>>>>>>cost.
>>>>>>
>>>>>
>>>>>I see.  So, you expect other layers (components, services) to 
>>>>>provides such parameters to the DMIS, right?
>>>>
>>>>Perhaps, perhaps not. This is simply out of scope from where the 
>>>>service gets its information. However, before a data 
>>>
>>>movement can take 
>>>
>>>>place, all surrounding parameters need to be clarified: A data 
>>>>movement inflicts temporary changes in the serivce's environment:
>>>>bandwidth usage, system load, etc. When requesting a data 
>>>
>>>movement, I 
>>>
>>>>have certain expectations that I would like to be met. It 
>>
>>is simply 
>>
>>>>not always true that a service can meet these expectations; for 
>>>>example if I request a data transfer to be carried out 
>>>
>>>using GridFTP 
>>>
>>>>but the service supports only MTOM, then the request would fail.
>>>>
>>>>This section expresses that the DMIS WG wants to have the 
>>>
>>>parameters 
>>>
>>>>that directly affect the data movement itself sorted out 
>>
>>before the 
>>
>>>>data movement will actually take place. (Don't know it this 
>>>
>>>sentence 
>>>
>>>>makes it any clearer, but on the other hand, I don't know how to 
>>>>express this better.)
>>>>
>>>>THe DMIS WG will start with a relatively simple model to 
>>>
>>>meet this use 
>>>
>>>>case. It it turns out that this is a more fundamental 
>>>
>>>issue, we always 
>>>
>>>>have the option to re-charter and spin off another Working 
>>>
>>>Group. In 
>>>
>>>>fact, this strategy was discussed at the GGF16 BoF.
>>>>
>>>>
>>>>>>The issue is that once something has decided the files 
>>>
>>>need to move 
>>>
>>>>>>(like
>>>>>>SRM) and it gives the service a request for say 100 files, the 
>>>>>>service could, in theory, move those in any order within 
>>>
>>>a request 
>>>
>>>>>>(perhaps we should have an option for in order...), or 
>>>
>>>may need to 
>>>
>>>>>>prioritize between requests.  So, a user might want to put 
>>>>>>priorities within a request (though I could argue 
>>
>>against that), 
>>
>>>>>>but they certainly might put priorities between requests 
>>>
>>>of their 
>>>
>>>>>>own.
>>>>>
>>>>>OK.  This makes sense to me.
>>>>>
>>>>>>This service is intended to be a file movement service, 
>>>
>>>along the 
>>>
>>>>>>lines of the existing RFT, FTS, etc..
>>>>
>>>>We have no
>>>>
>>>>>>intention of doing SRM type functionality.  It is 
>>
>>something SRM 
>>
>>>>>>might call.
>>>>>>
>>>>>
>>>>>Good.  This is where I found the above sentence unclear.  
>>>
>>>It sounded 
>>>
>>>>>to me like the DMIS will be responsible to call all the other 
>>>>>services and coordinate various activities including 
>>>
>>>allocating and 
>>>
>>>>>reserving space.  But, as long as it is intended as a 
>>>
>>>"file movement 
>>>
>>>>>service" without expectations that it will
>>>>
>>>>perform
>>>>
>>>>>space negotiations, etc. it sounds fine.
>>>>
>>>>Yes, 'bull's eye".
>>>>
>>>>
>>>>>>So, I would argue that we need verbiage that says we will
>>>>
>>>>want extra
>>>>
>>>>>>information available to make prioritization decisions 
>>>
>>>within the 
>>>
>>>>>>service, and the information is of the type listed.  Do 
>>>
>>>you have a 
>>>
>>>>>>suggestion for a wording change that makes it clearer?
>>>>>>
>>>>>
>>>>>I'll try a little later, by the end of the day ...
>>>>>
>>>>>>>Thus, in the setup phase, all kinds of services will 
>>
>>have to be 
>>
>>>>>>>contacted, such as components that manage resources, 
>>
>>components 
>>
>>>>>>>that perform scheduling, keep track of accounting, etc.  For 
>>>>>>>example, to reserve storage space, the data movement 
>>>
>>>service may 
>>>
>>>>>>>want to interact with SRMs.  I think this is a huge 
>>>
>>>undertaking, 
>>>
>>>>>>>and too large of a scope.  Even if you go through 
>>>
>>>WS-agreement, it 
>>>
>>>>>>>will have to interact with various resource managers, 
>>>
>>>accounting, 
>>>
>>>>>>>billing, etc.  You may want to leave all the
>>>>
>>>>setup to
>>>>
>>>>>>>a separate component, like WS-agreement, and then 
>>
>>focus on the 
>>
>>>>>>>data movement part.
>>>>>>>
>>>>>>
>>>>>>WS-Agreement is not a component, it is just a pattern for
>>>>
>>>>negotiating
>>>>
>>>>>>agreements, but as I noted above, our intent is that 
>>>
>>>this is just a 
>>>
>>>>>>file movement service.  The one point I might argue on 
>>>
>>>this would 
>>>
>>>>>>be reporting to such services .vs. negotiating with them.  For 
>>>>>>instance, if there were some standard message that could 
>>>
>>>be used to 
>>>
>>>>>>report "resource consumption", it might be useful to 
>>>
>>>include some 
>>>
>>>>>>kind of EPR or such which
>>>>
>>>>the data
>>>>
>>>>>>movement
>>>>>>service could inform about the resources consumed during the 
>>>>>>request.  I am not sure this is a good idea, nor that 
>>>
>>>fits in the 
>>>
>>>>>>WS-I
>>>>
>>>>basic profile
>>>>
>>>>>>because it is a sort of notification, but it is a thought. 
>>>>
>>>> In the
>>>>
>>>>>>WSRF
>>>>>>rendering, you have the alternative of exposing such info
>>>>
>>>>as an RP
>>>>
>>>>>>and the
>>>>>>interested service could be directed to subscribe to that RP.
>>>>>>
>>>>>
>>>>>Again, I am confused about term"resource consumption".  A 
>>>
>>>DMIS can 
>>>
>>>>>report on file movement, how many files were moved 
>>
>>successfully, 
>>
>>>>>total size moved, hoe many to go, etc.  It cannot report on
>>>>
>>>>storage
>>>>
>>>>>consumed, because it does not know what is done with the 
>>>
>>>files after 
>>>
>>>>>they are moved (for example they may be used quickly and 
>>>
>>>removed, or 
>>>
>>>>>archived).  Am I missing something?
>>>>
>>>>Yes, a DMIS can and should report on the resources it has 
>>>
>>>consumed to 
>>>
>>>>fulfil a data movement request, e.g. bandwidth usage, CPU 
>>>
>>>cycles(?), 
>>>
>>>>total bytes moved, etc. It is a different set of resources 
>>>
>>>a storage 
>>>
>>>>management system may report, though, but still a report 
>>
>>on which a 
>>
>>>>bill/invoice may be filed later-on.
>>>>
>>>>
>>>>>>>2) A related point, in the "Seven questions: Evaluation 
>>>
>>>Criteria 
>>>
>>>>>>>(from GFD.3)" document it says:
>>>>>>>"The most direct overlap would be with the gsm-wg, 
>>
>>but they are 
>>
>>>>>>>participating in this group and will, presumably, use what is 
>>>>>>>developed here for the transport portion of their interface."
>>>>>>>
>>>>>>>Yes, it makes sense for SRMs to make use of powerful 
>>
>>transport 
>>
>>>>>>>services, since SRMs rely on transport services.  But, when a 
>>>>>>>request is made to an SRM, they take care of managing storage 
>>>>>>>space, keeping track of usage and accounting, as well 
>>>
>>>as perform 
>>>
>>>>>>>scheduling, so it does not make sense for the transport 
>>>
>>>services 
>>>
>>>>>>>to negotiate storage allocations (including lifetime), 
>>>
>>>scheduling, 
>>>
>>>>>>>etc, again, as is suggested in the "setup
>>>>
>>>>phase" above.
>>>>
>>>>>>I think I addressed this above.  We have no intention of 
>>>
>>>the data 
>>>
>>>>>>movement service to be doing storage allocation.  I strongly 
>>>>>>disagree with you though that it does not make sense for the 
>>>>>>service to be able do scheduling of its own data 
>>
>>movement tasks.
>>
>>>>>I do not disagree with you.  I did not mean to say that 
>>
>>the DMIS 
>>
>>>>>should not be able to do its own scheduling of data
>>>>
>>>>movement.  What
>>>>
>>>>>I meant is that if a request such as srmCopy (which 
>>
>>later could be
>>
>>>>>renamed) is made, and the SRMs reserved space and make 
>>
>>their own 
>>
>>>>>scheduling decisions, that this is now passed to the DMIS.  For 
>>>>>example, if you include in the DMIS space allocation, 
>>
>>then we get 
>>
>>>>>into a circular situation: call SRM which calls DMIS which
>>>>
>>>>calls SRM.
>>>>
>>>>The model would be hierarchical: SRM utilises DMIS to 
>>
>>carry out the 
>>
>>>>data movement, passing on its constraints down to a DMIS 
>>>
>>>(like "Hey, 
>>>
>>>>service, I need to have the following files transferred 
>>>
>>>with such and 
>>>
>>>>such constraints, can you do that?"). If you have only 
>>
>>one DMIS in 
>>
>>>>your domain, that this is usually a piece of cake, basically 
>>>>obliterating the need of some sort of negotiating/agreement phase.
>>>>But consider a horde of DMISs, then you may need or want 
>>
>>to contact 
>>
>>>>more than one DMIS and see which one may serve you best.
>>>>
>>>>
>>>>Cheers,
>>>>Michel
>>>>
>>>>
>>>
>>