[dmis-bof] Vote for submission of Charter

[William E. Allcock]

> -----Original Message-----
> From: owner-dmis-bof at ggf.org [mailto:owner-dmis-bof at ggf.org] 
> On Behalf Of Arie Shoshani
> Sent: Saturday, March 18, 2006 5:29 PM
> To: allcock at mcs.anl.gov
> Cc: dmis-bof at ggf.org; Peter Kunszt; Alex Sim; Timur Perelmutov
> Subject: Re: [dmis-bof] Vote for submission of Charter
> 
> Bill,
> 
> I sent an email yesterday expressing some questions (not sure 
> if to call 
> them objections, but I am concerned that the setup phase is 
> too large of 
> a scope for this BOF and this activity).  I have not seen a 
> response.  
> Here it is again.
> 
> Arie
> --------------
> I saw a couple of items that I'd like to comment on.
> 
> 1) Regarding the sentence:
> "Setting up a data movement includes the selection of a transport 
> protocol, for example GridFTP, and parameters for 
> reliability, timing, 
> scheduling, resource usage, accounting, billing, etc. The 
> Working Group 
> will explore existing mechanisms to reach such agreement, e.g. 
> WS-Agreement and use them where appropriate."

Perhaps we need to reword this section, but I think the intent of this part
was in response to comments about providing information for the service to
make prioritization decisions amongst files within a request and between
requests.  So, reliability might be parameters associated with number of
retries, backoff, etc.; timing could be fail if it takes more than x hours;
scheduling could be have it done by x time; resource usage could be use no
more that 50 Mbs; accounting/billing could be don't exceed z cost.

The issue is that once something has decided the files need to move (like
SRM) and it gives the service a request for say 100 files, the service
could, in theory, move those in any order within a request (perhaps we
should have an option for in order...), or may need to prioritize between
requests.  So, a user might want to put priorities within a request (though
I could argue against that), but they certainly might put priorities between
requests of their own.  This service is intended to be a file movement
service, along the lines of the existing RFT, FTS, etc..  We have no
intention of doing SRM type functionality.  It is something SRM might call.

So, I would argue that we need verbiage that says we will want extra
information available to make prioritization decisions within the service,
and the information is of the type listed.  Do you have a suggestion for a
wording change that makes it clearer?

> 
> Thus, in the setup phase, all kinds of services will have to be 
> contacted, such as components that manage resources, components that 
> perform scheduling, keep track of accounting, etc.  For example, to 
> reserve storage space, the data movement service may want to interact 
> with SRMs.  I think this is a huge undertaking, and too large of a 
> scope.  Even if you go through WS-agreement, it will have to interact 
> with various resource managers, accounting, billing, etc.  
> You may want 
> to leave all the setup to a separate component, like 
> WS-agreement, and 
> then focus on the data movement part.

WS-Agreement is not a component, it is just a pattern for negotiating
agreements, but as I noted above, our intent is that this is just a file
movement service.  The one point I might argue on this would be reporting to
such services .vs. negotiating with them.  For instance, if there were some
standard message that could be used to report "resource consumption", it
might be useful to include some kind of EPR or such which the data movement
service could inform about the resources consumed during the request.  I am
not sure this is a good idea, nor that fits in the WS-I basic profile
because it is a sort of notification, but it is a thought.  In the WSRF
rendering, you have the alternative of exposing such info as an RP and the
interested service could be directed to subscribe to that RP.



> 
> 2) A related point, in the "Seven questions: Evaluation 
> Criteria (from 
> GFD.3)" document it says:
> "The most direct overlap would be with the gsm-wg, but they are 
> participating in this group and will, presumably, use what is 
> developed 
> here for the transport portion of their interface."
> 
> Yes, it makes sense for SRMs to make use of powerful 
> transport services, 
> since SRMs rely on transport services.  But, when a request 
> is made to 
> an SRM, they take care of managing storage space, keeping 
> track of usage 
> and accounting, as well as perform scheduling, so it does not 
> make sense 
> for the transport services to negotiate storage allocations 
> (including 
> lifetime), scheduling, etc, again, as is suggested in the 
> "setup phase" 
> above.

I think I addressed this above.  We have no intention of the data movement
service to be doing storage allocation.  I strongly disagree with you though
that it does not make sense for the service to be able do scheduling of its
own data movement tasks.

> 
> 3) In the charter document, it says:
> "To accomplish 3^rd party data transfer, a uniform, yet 
> abstract naming 
> scheme for resources (data in general, files in particular) 
> is required. 
> This working group will provide such abstract uniform naming scheme."
> 
> I think this was the goal of the GFS-WG group, to the extent that I 
> understand what they doing.  It might be good to coordinate 
> with them.  
> Also, isn't the issue if a uniform name space (also referred to as 
> logical namespace) and its mapping into physical names fall 
> into the RLS 
> domain (and future related activities).  Do you really want to keep 
> track of this mapping as part of the data movement service?  
> Perhaps I 
> don't understand the concept.  Please clarify.

I don't believe we intend to create any namespace tech, but use what others
are creating.  This sentence is there because we want this service to be
able to move any named piece of data, not just a file.  We just want to make
sure that, where possible, we keep data other than files in mind and allow
for that in our schema, WSDL, etc..

As to someone to handle the srm_copy WSDL, in theory anyone who can read
WSDL could do that, but it will be the ensuing discussion that will be more
important, and where the SRM communities viewpoint would be useful.  Could
whoever is leading the gsm WG meeting handle this?  Or is there not going to
be a gsm WG meeting at this GGF?  

It would be best if someone was present from all the concerned parties, but
if not, there will be discussions via email and future GGFs.  I don't really
see an option other than proceeding as best we can with those that can make
it.

Bill
> 
> --------------------
> 
> See additional embedded responses below.
> 
> William E. Allcock wrote:
> > DOH!  I thought this went out yesterday morning.
> >
> > ---------------------------------------------
> >
> > All,
> >
> > The deadline for getting our charter in is fast approaching 
> (it may be
> > today, I have sent mail to ask).  I would like to call for 
> a vote on the
> > current charter (attached).  If anyone has any objections 
> to submitting as
> > is, please speak up immediately.  Note that there is now 
> the concept of a
> > living charter, so we can make changes if necessary.
> >   
> See above.
> > Also, need to know if anyone objects violently to going 
> with DMIS as the
> > name.
> >   
> Sounds like a good name to me.
> > Also, per our proposed schedule we would have two sessions 
> in Tokyo, one of
> > which is the WSDL presentation, the second is discussion 
> and a start at
> > trying to extract the common elements for the functional spec. 
> >
> > I will handle the presentation for RFT WSDL.
> > Can we have a volunteer that will be at GGF17 volunteer for:
> >
> >   - srm_copy?  Alex?
> >   
> Alex and I did not plan to attend this next GGF, and neither is Peter 
> Kunszt.
> I wonder if Timur or another person involved with SRMs is planning 
> attend the
> Tokyo meeting.
> >   - FTS? Erwin?
> >   - Unicore (not even sure that is the right way to 
> reference it)?  Michel?
> >
> > I look forward to working with you on this.
> >
> > Bill
> >
> > ---------------------------------------------------------------
> > William E. Allcock
> > Argonne National Laboratory
> > Bldg 221, Office B-139
> > 9700 South Cass Ave
> > Argonne, IL 60439-4844
> > Email:           allcock at mcs.anl.gov
> > Office Phone:    +1-630-252-7573
> > Office Fax:      +1-630-252-1997
> > Cell Phone:      +1-630-854-2842
> >   
> 
>